Sample Kafka Project

Radek Stolarczyk
5 days ago
2 min read

Overview

This project is a beginner-friendly Kafka + Spring Boot example demonstrating an event-driven data pipeline.

The application reads raw person data, sends it through Kafka, standardizes the data, and outputs the transformed result.

The project is structured as a multi-module Maven project inside a single IntelliJ workspace.

Architecture

What the Project Does

The processor service standardizes incoming data:

Example Input

{  "firstName": "anna",  "lastName": "kowalska",  "dateOfBirth": "1988-12-03"}

Example Output

{  "firstName": "Anna",  "lastName": "Kowalska",  "dateOfBirth": "03 December 1988"}

Technologies Used

Java 24
Spring Boot 3
Spring Kafka
Apache Kafka
Docker
Maven
IntelliJ IDEA

Project Modules

kafka-standardizer/│├── producer-service├── processor-service├── consumer-service├── common├── data└── docker-compose.yml

Module Responsibilities

producer-service

Publishes raw person messages to Kafka topic:

raw-people

processor-service

Consumes raw messages, standardizes data, and publishes transformed messages to:

standardized-people

consumer-service

Consumes standardized messages and prints final output.

common

Contains shared DTO/model classes:

PersonRawPersonStandardized

How to Run

1. Start Kafka

From project root:

docker compose up

2. Verify Kafka Container

Open new terminal:

docker ps

Expected container:

kafka-standardizer

3. Create Kafka Topics

Create raw topic:

docker exec -it kafka-standardizer /opt/kafka/bin/kafka-topics.sh \--create \--topic raw-people \--bootstrap-server localhost:9092

Create standardized topic:

docker exec -it kafka-standardizer /opt/kafka/bin/kafka-topics.sh \--create \--topic standardized-people \--bootstrap-server localhost:9092

4. Start Spring Boot Applications

Start applications in IntelliJ in this order:

1. ProcessorServiceApplication2. ConsumerServiceApplication3. ProducerServiceApplication

5. Send Test Message to Kafka

Open new terminal:

docker exec -it kafka-standardizer /opt/kafka/bin/kafka-console-producer.sh \--topic raw-people \--bootstrap-server localhost:9092

Paste sample message:

{"firstName":"anna","lastName":"kowalska","dateOfBirth":"1988-12-03"}

Press ENTER.

6. Expected Result

Processor service standardizes the data:

anna -> Annakowalska -> Kowalska1988-12-03 -> 03 December 1988

Consumer service output:

Anna Kowalska - 03 December 1988

Useful Kafka Commands

List Topics

docker exec -it kafka-standardizer /opt/kafka/bin/kafka-topics.sh \--list \--bootstrap-server localhost:9092

Read Raw Topic

docker exec -it kafka-standardizer /opt/kafka/bin/kafka-console-consumer.sh \--topic raw-people \--from-beginning \--bootstrap-server localhost:9092

Read Standardized Topic

docker exec -it kafka-standardizer /opt/kafka/bin/kafka-console-consumer.sh \--topic standardized-people \--from-beginning \--bootstrap-server localhost:9092

Stop Everything

Stop Spring Boot applications in IntelliJ.

Then stop Kafka:

docker compose down