7c631ba8c0
In some cases we'll want to save the data from the stream directly with no transformation needed. This will allow us to return the raw data string from the stream * Add new StringToStringTransformer * Remove Record from codebase in favor of more generic interface
16 lines
650 B
Go
16 lines
650 B
Go
package connector
|
|
|
|
// Buffer defines a buffer used to store records streamed through Kinesis. It is a part of the
|
|
// Pipeline utilized by the Pipeline.ProcessShard function. Records are stored in the buffer by calling
|
|
// the Add method. The buffer has two size limits defined: total total number of records and a
|
|
// time limit in seconds. The ShouldFlush() method may indicate that the buffer is full based on
|
|
// these limits.
|
|
type Buffer interface {
|
|
ProcessRecord(record interface{}, sequenceNumber string)
|
|
FirstSequenceNumber() string
|
|
Flush()
|
|
LastSequenceNumber() string
|
|
NumRecordsInBuffer() int
|
|
Records() []interface{}
|
|
ShouldFlush() bool
|
|
}
|