Kafka Consumer Test Example

Example consumer configuration using the Kafka client tools. Update the identified <variables> accordingly.

Prerequisites:

• valid client cert w/pkey
• pkey password/passphrase

Example configuration steps:

1. Create certfile with client cert: cert.pem
2. Create key file: key.pem
3. Create PKCS12 from pem files: openssl pkcs12 -export -in cert.pem -inkey key.pem -name <cert DN> -out <filename>.p12
4. Create keystore using PKCS12 file: keytool -importkeystore -deststorepass <store password> -destkeystore kafka.client.keystore.jks -srckeystore <filename>.p12 -srcstoretype PKCS12
5. Copy Java truststore file to same dir as keystore: cp /usr/lib/jvm/java-<version>/jre/lib/security/cacerts kafka.client.truststore.jks
6. Copy and edit Kafka consumer.properties file:
   1. group.id=<consumer group>
   2. security.protocol=SSL
   3. ssl.truststore.location=kafka.client.truststore.jks
   4. ssl.keystore.location=kafka.client.keystore.jks
   5. ssl.keystore.password=<store password>
   6. ssl.key.password=<key password>

7. Update local hosts file with all broker FQDNS & IP addresses for correct name resolution (if external)

8. Run Kafka consumer command to confirm connectivity: bin/kafka-console-consumer.sh --bootstrap-server <broker FQDN>:9094 --consumer.config config/consumer.properties --topic <topic> --max-messages 10

Completion of the above steps will enable streaming of the requested data feed from the Cboe Global Cloud.