Overview
Amazon Kinesis Data Streams is AWS's managed real-time data streaming service. The Automize AWS Kinesis connector covers stream + shard management, put/get records, retention configuration, and tagging.
- Regions: Regional. Each stream lives in one region; cross-region streaming via separate streams + DynamoDB Streams or MSK.
- Plans: Pay-per-shard-hour + per-million-records. Provisioned + On-Demand modes.
Setting up the connection
Auth type: aws_sigv4.
- AWS IAM → user + AmazonKinesisFullAccess (or scoped per-stream).
- In Automize → Settings → Connectors → AWS Kinesis. Paste keys + region.
Find your credentials at https://us-east-1.console.aws.amazon.com/iam/home.
Rate limits
Per-shard: 1 MB/sec or 1000 records/sec write, 2 MB/sec read. Connector retries ProvisionedThroughputExceededException.
Data model
Streams + shards:
Stream ─── Shards (each: ordered append-log with partition key range)
├─ Records (data + partition key + sequence number)
└─ Iterators (read positions: AT_SEQUENCE_NUMBER, LATEST, TRIM_HORIZON)Available operations
17 operations available. Click any row to jump to its detail.
| Operation | Category | What it does |
|---|---|---|
| Add Tags | General | Add tags to a stream |
| Create Stream | General | Create a new Kinesis stream |
| Decrease Retention Period | General | Decrease stream data retention period |
| Delete Stream | General | Delete a Kinesis stream |
| Describe Stream | General | Get stream details and shard information |
| Describe Stream Summary | General | Get stream summary without shard details |
| Get Records | General | Read records from a shard iterator |
| Get Shard Iterator | General | Get a shard iterator for reading records |
| Increase Retention Period | General | Increase stream data retention period |
| List Shards | General | List shards in a stream |
| List Streams | General | List all Kinesis streams |
| List Tags | General | List tags for a stream |
| Merge Shards | General | Merge two adjacent shards |
| Put Record | General | Write a single record to a stream |
| Put Records | General | Write multiple records to a stream |
| Remove Tags | General | Remove tags from a stream |
| Split Shard | General | Split a shard into two |
Operations
Add Tags
Add tags to a stream
No input parameters.
Create Stream
Create a new Kinesis stream
No input parameters.
Tips
- ShardCount + StreamModeDetails (PROVISIONED or ON_DEMAND).
Decrease Retention Period
Decrease stream data retention period
No input parameters.
Delete Stream
Delete a Kinesis stream
No input parameters.
Describe Stream
Get stream details and shard information
No input parameters.
Tips
- Returns shards + status + retention period.
Describe Stream Summary
Get stream summary without shard details
No input parameters.
Get Records
Read records from a shard iterator
No input parameters.
Tips
- Use ShardIterator from Get Shard Iterator. Returns up to 10 MB or 10,000 records.
Get Shard Iterator
Get a shard iterator for reading records
No input parameters.
Tips
- ShardIteratorType: LATEST (new records only), TRIM_HORIZON (oldest available), AT_TIMESTAMP, AT/AFTER_SEQUENCE_NUMBER.
Increase Retention Period
Increase stream data retention period
No input parameters.
List Shards
List shards in a stream
No input parameters.
List Streams
List all Kinesis streams
No input parameters.
List Tags
List tags for a stream
No input parameters.
Merge Shards
Merge two adjacent shards
No input parameters.
Put Record
Write a single record to a stream
No input parameters.
Tips
- StreamName + Data (max 1 MB) + PartitionKey (determines shard).
- Same partition key → same shard → strict order.
Put Records
Write multiple records to a stream
No input parameters.
Tips
- Bulk – up to 500 records (5 MB total) per call. Returns per-record success.
Remove Tags
Remove tags from a stream
No input parameters.
Split Shard
Split a shard into two
No input parameters.
FAQ
- Why are records out of order?
- Order is per-shard. Records with different partition keys may land in different shards. Use a fixed PartitionKey for strict order (but it serialises to one shard's throughput).
- Kinesis vs SQS vs SNS?
- Kinesis: ordered replayable stream, multiple consumers. SQS: queue with single consumer per message. SNS: pub-sub fan-out. Use Kinesis for analytics + event sourcing.