Kafka consumer app written in Python. Prints messages with their partition and offset.
158
Generic Kafka consumer app written in Python that prints out message with their partition and offset. Works with the Confluent schema registry to deserialize Avro and JSON messages when provided such config.
The app's source code is on GitHub.
The configuration can be changed by setting environment variables.
| Name | Default | Description |
|---|---|---|
KAFKA_BOOTSTRAP_SERVERS | localhost:19092 | Kafka broker connection string |
SCHEMA_REGISTRY_URL | http://localhost:8084 | URL for the Schema Registry service |
SERIALIZATION | None | Type of serialization to use for message handling |
SCHEMA_LOC | None | Location of the schema |
SCHEMA_FILE_PATH | None | File path to the schema definition |
TOPICS | customers | Comma-separated list of Kafka topics to consume (whitespace will be trimmed) |
SLEEP_TIME | 1 | Time to sleep between processing each message (in seconds) |
CLIENT_ID | my-python-consumer | Identifier for the Kafka consumer client |
CONSUMER_GROUP_ID | my-mostly-pythonic-consumer-group | Consumer group identifier for managing partition assignments |
AUTO_OFFSET_RESET | earliest | Controls where to start reading messages when no offset is stored |
Security has not been built into this PoC app, must be an Kafka cluster that does not require authentication.
Content type
Image
Digest
sha256:8620013b7…
Size
65.6 MB
Last updated
over 1 year ago
docker pull stuzanne/kafka-consumer