Publish Messages to Kafka Topic

Thanh Nguyen

Publish Messages to Kafka Topic

1. Create a new Mule Project by clicking on File > New > Mule Project.

2. Navigate through the project’s structure and double-click on src/main/app/project-name.xml and follow the steps below:

3. Drag and drop a new HTTP element onto the canvas. This element is going to be the entry point for the flow and will provide data to be sent to the topic

Publish Messages to Kafka Topic

4. Drag and drop a new Apache Kafka element after the HTTP listener.

5. Drag and drop a new Set Payload element after Apache Kafka. This Set Payload element is going to set the response to the HTTP request

Publish Messages to Kafka Topic

5. Drag and drop a new Set Payload element after Apache Kafka. This Set Payload element is going to set the response to the HTTP request

Publish Messages to Kafka Topic

6. Double click on flow header (blue line) and change the name of the flow to "producer-flow"

Publish Messages to Kafka Topic

7. Select the HTTP element.8. Click on the plus sign next to the "Connector

Configuration" dropdown.9. A pop-up appears, accept the default

configurations and click OK.10.Set Path to push

Publish Messages to Kafka Topic

11.Set Display Name to Push http endpoint

Publish Messages to Kafka Topic

12.Select the Apache Kafka connector and set its properties as below:

Display Name Kafka producerConsumer Configuration "Apache_Kafka__Configuratio

n" (default name of a configuration, or any other configuration that you configured as explained in Configuring the Kafka Connector Global Element section)Operation Producer

Topic #[payload.topic]Key #[server.dateTime.getMilliSec

onds()]Message #[payload.message]

Publish Messages to Kafka Topic

13.For the Set Payload element:• Set Display Name to Set push response• Set Value to Message successfully sent

Publish Messages to Kafka Topic

14.Now we have to provide values for placeholders.15.Open /src/main/app/mule-app.properties and

provide values for following properties:– If you configured the Kafka global element as explained

within the Configuration section then provide values for config.bootstrapServers, config.consumerPropertiesFile and config.producerPropertiesFile

16.Now you can deploy the app. (Run As > Mule Application)

Publish Messages to Kafka Topic

17.In order to trigger the flow and push a message to a topic, use an HTTP client app and send a POST request with content-type "application/x-www-form-urlencoded" and body in urlencoded format to localhost:8081/push. The request should contain values for topic and message

Publish Messages to Kafka Topic - XML

Question and answer