luciano/vmware-go-kcl-v2
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

KCL: KCL should ignore deleted parent shard

Browse source 
After a few days of shard splitting, the parent shard will be
deleted by Kinesis system. KCL should ignore the error caused
by deleted parent shared and move on.

Test:
Manuall split shard on kcl-test stream in photon-infra account
Currently, shard3 is the parent shard of shard 4 and 5. Shard 3
has a parent shard 0 which has been deleted already. Verified
the test can run and not stuck in waiting for parent shard.

Jira CNA-2089

Change-Id: I15ed0db70ff9836313c22ccabf934a2a69379248
This commit is contained in:
Tao Jiang 2018-09-04 14:40:46 -07:00
parent 9addbb57f0
commit 3163d31f28
1 changed files with 5 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
clientlibrary/worker/shard-consumer.go
View file

@ -125,8 +125,11 @@ func (sc *ShardConsumer) getRecords(shard *shardStatus) error {
// If the shard is child shard, need to wait until the parent finished.
if err := sc.waitOnParentShard(shard); err != nil {
log.Errorf("Error in waiting for parent shard: %v to finish. Error: %+v", shard.ParentShardId, err)
return err
// If parent shard has been deleted by Kinesis system already, just ignore the error.
if err != ErrSequenceIDNotFound {
log.Errorf("Error in waiting for parent shard: %v to finish. Error: %+v", shard.ParentShardId, err)
return err
}
}
shardIterator, err := sc.getShardIterator(shard)