The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Need a server to try this on? Antihost offers Linux VPS plans with NVMe storage, full root access and a 14-day free trial.
To install Apache Kafka on a Linux VPS, install Java 17 or 21, download and unpack a Kafka release, initialize its KRaft storage, start the broker, and verify it with a test topic. A default installation is suitable only for a private test machine: Kafka starts with unauthenticated PLAINTEXT listeners and no authorization. Before permitting Internet or cross-host traffic, configure listener addresses, TLS or SASL, ACLs, firewall rules, and a private controller listener.
What you need before installing Kafka
The commands below use the Apache Kafka 4.3.0 quick-start layout. Pinning a release makes paths and configuration reproducible; if you choose another release, use the configuration files shipped with that archive and adjust the directory name in every command.
| Requirement | What to prepare | Why it matters |
|---|---|---|
| Operating system | Supported 64-bit Linux on a VPS | Kafka and its bundled scripts expect a standard 64-bit Unix environment. |
| Java | Java 17 or Java 21; Java 21 is the current recommended LTS in Kafka documentation | The broker, storage tool, and command-line clients run on the Java runtime. |
| Service account | A non-root account named kafka (or an equivalent dedicated account) |
Running the broker as root increases the impact of a compromise. |
| Storage | Persistent disk mounted at a stable path for Kafka logs | Topic data and the KRaft metadata log must survive a process restart. |
| Network identity | A DNS name that resolves to the VPS address, if remote clients will connect | Clients use the broker’s advertised address after metadata discovery. |
| Firewall policy | Only the client, administration, and internal peer traffic you actually need | Kafka has no protective network policy when first installed. |
Install Java from your Linux distribution or a trusted OpenJDK build, then verify the runtime before touching Kafka:
java -versionKafka’s current Java guidance fully supports Java 17 and Java 21 and recommends the latest long-term-support release available for performance, efficiency, and support. The 4.3.0 quick-start example requires Java 17 or newer.
Choose KRaft for a new VPS installation
KRaft is Kafka’s built-in metadata quorum and is the normal choice for a new deployment. ZooKeeper is an older operating model documented for compatibility with existing environments; it adds a separate service and a second set of failure and security concerns.
Combined mode for a lab or single VPS
The standalone configuration starts one process that acts as both broker and controller. It is convenient for development, a functional test, or a small private service, but the VPS itself remains a single point of failure. Losing that host loses both roles at once.
Free tools Windows power users keep installed
One-click scans. No signup required.
Separate roles for critical deployments
For a production system, separate broker and controller roles and place controllers on independent hosts or failure domains. A controller quorum that must tolerate N concurrent controller failures needs 2N+1 controllers. One VPS cannot provide host-level fault tolerance, regardless of how carefully Kafka is configured.
Prepare the Linux VPS
Create a dedicated account and directories
Log in with an administrative account, create the service user, and give it ownership of Kafka’s installation and data paths. The exact commands vary by distribution; the following is a typical Linux layout:
sudo useradd --system --create-home --home-dir /var/lib/kafka --shell /usr/sbin/nologin kafka
sudo mkdir -p /opt/kafka /var/lib/kafka
sudo chown -R kafka:kafka /opt/kafka /var/lib/kafkaKeep the archive and the active installation under a versioned, stable path such as /opt/kafka. Put the log directory on persistent storage rather than a temporary filesystem. If your VPS provider offers separate block storage, ensure it is mounted before Kafka starts and that the mount survives reboot.
Set the host name and DNS
Choose the name clients will use, for example kafka.example.net, and verify forward resolution from every client network. A name that resolves only on the VPS or resolves to an unreachable private address will make bootstrap appear to work while metadata requests fail later.
Install and verify Java
Install OpenJDK 21 when your distribution provides it; Java 17 is also supported. Confirm both the version and the executable visible to the service account. If multiple Java installations exist, set the system default or specify JAVA_HOME in the service definition so systemd and interactive shells use the same runtime.
Plan the firewall before opening Kafka
Decide which addresses need client access, which administrators need management access, and which hosts may act as broker or controller peers. Do not open every Kafka port to the Internet. Apply the host firewall and any provider security-group rules before changing a listener from loopback to a public interface.
Download and extract Kafka
Download the binary archive from the official Apache Kafka release channel and verify it according to that release’s published checksum or signature instructions. Copy the archive to the VPS, then extract it under the stable installation path:
cd /tmp
tar -xzf kafka_2.13-4.3.0.tgz
sudo mv kafka_2.13-4.3.0 /opt/kafka
sudo chown -R kafka:kafka /opt/kafkaThe Scala suffix in the filename is part of the archive name; use the filename you actually downloaded. Before proceeding, confirm that /opt/kafka/bin/kafka-storage.sh, kafka-server-start.sh, and kafka-topics.sh exist.
Recommended Free Tools
Initialize KRaft storage
KRaft storage must be formatted with a cluster identifier before the broker can start. Run the storage commands as the kafka user and use the configuration bundled with the release:
sudo -u kafka -H bash
cd /opt/kafka
KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"
bin/kafka-storage.sh format --standalone -t "$KAFKA_CLUSTER_ID" -c config/server.propertiesThe --standalone option creates a single combined broker/controller for the quick-start topology. The configuration path can differ between releases, so use the KRaft configuration shipped in your archive if it is named differently.
Formatting writes the cluster identity and metadata-log structure to the configured log directory. Do not point a formatted directory at a different cluster identity or casually reuse it for another cluster. If you are rebuilding a disposable test installation, stop Kafka and remove only the data directory you have explicitly decided to destroy; deleting it destroys the topics stored there.
Configure listeners before remote access
Kafka has two separate address concepts. listeners controls where the broker binds sockets. advertised.listeners tells clients which addresses to use after they receive cluster metadata. Both must be reachable from the relevant network.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallKeep the controller private
The controller listener is for KRaft quorum traffic, not application clients. Bind it to loopback for a single-host combined deployment or to a private interface shared only by controllers and brokers. Never advertise or firewall-open the controller listener to client networks.
Use a deliberate listener map
A minimal private test configuration can look like this, subject to the property names and defaults in your release’s bundled file:
process.roles=broker,controller
node.id=1
controller.quorum.voters=1@127.0.0.1:9093
listeners=PLAINTEXT://0.0.0.0:9092,CONTROLLER://127.0.0.1:9093
advertised.listeners=PLAINTEXT://kafka.example.net:9092
listener.security.protocol.map=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT
controller.listener.names=CONTROLLER
inter.broker.listener.name=PLAINTEXTFor a loopback-only test, advertise localhost:9092 instead of a public DNS name. For a remote client, advertise the DNS name and port that the client can resolve and route to; advertising localhost causes a remote client to connect to itself.
Understand Kafka’s four security protocols
| Protocol | Encryption | Authentication | Typical use |
|---|---|---|---|
PLAINTEXT |
None | None | Closed, disposable testing only |
SSL |
TLS | Certificate-based TLS identity | Encrypted clients and broker links without SASL credentials |
SASL_PLAINTEXT |
None | SASL mechanism | Authenticated private networks where encryption is supplied elsewhere |
SASL_SSL |
TLS | SASL mechanism plus TLS | Internet or cross-network clients that need confidentiality and identity checks |
When TLS or SASL is enabled, ensure the controller and inter-broker listener names, security-protocol map, trust stores, and quorum settings agree. A mismatch often allows the process to start but prevents brokers, controllers, or clients from completing handshakes.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Start Kafka
Run it in the foreground first
Start the broker interactively to catch configuration and permission errors immediately:
cd /opt/kafka
bin/kafka-server-start.sh config/server.propertiesLeave this process attached while you perform the first validation. Stop it with Ctrl+C after confirming the test workflow, or use the same configuration in a service manager.
Use systemd for a persistent VPS service
A VPS deployment should restart Kafka after a reboot and recover from a transient process failure. Create a unit such as /etc/systemd/system/kafka.service, changing paths if your installation differs:
[Unit]
Description=Apache Kafka broker
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=kafka
Group=kafka
WorkingDirectory=/opt/kafka
Environment=JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
ExecStart=/opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties
Restart=on-failure
RestartSec=5
LimitNOFILE=100000
[Install]
WantedBy=multi-user.targetCheck the actual Java path with your distribution’s package tools. Then load and start the unit:
sudo systemctl daemon-reload
sudo systemctl enable --now kafka
sudo systemctl status kafka
sudo journalctl -u kafka -n 100 --no-pagerThe unit is an operational packaging choice, not a special Kafka mode. Keep logs in a controlled location, rotate them according to your retention policy, and ensure the service account can read and write the configured Kafka log directories.
Create and test a topic
With the broker running and a matching client listener available, create a topic and inspect its metadata:
/opt/kafka/bin/kafka-topics.sh --create
--topic quickstart-events
--bootstrap-server localhost:9092
/opt/kafka/bin/kafka-topics.sh --describe
--topic quickstart-events
--bootstrap-server localhost:9092Use the advertised host and port instead of localhost:9092 when testing from another machine. The describe command should return the topic’s partition, replication, leader, and replica metadata.
Produce a record
echo 'Kafka test message' | /opt/kafka/bin/kafka-console-producer.sh
--topic quickstart-events
--bootstrap-server localhost:9092Consume the record
/opt/kafka/bin/kafka-console-consumer.sh
--topic quickstart-events
--from-beginning
--bootstrap-server localhost:9092For an SSL, SASL, or SASL_SSL listener, pass a client-properties file containing the required trust-store, credential, and security-protocol settings to both commands. The client protocol must match the listener it reaches; successful TCP connection alone does not prove that authentication or authorization is correct.
Secure a public-facing VPS
Apache Kafka’s security model is explicit: security is off by default. A fresh cluster accepts unauthenticated PLAINTEXT connections on every listener and applies no authorization. Treat the quick-start broker as private until all of the following controls are in place.
Bind only intended interfaces
Bind the client listener to the private or public interface that clients genuinely need. Do not use a wildcard bind merely because it is convenient. Keep the controller listener on loopback or a private network, and do not include its address in client documentation or load-balancer rules.
Advertise a reachable, stable address
Set advertised.listeners to a DNS name or address that every intended client can resolve and reach. If clients connect through NAT, a reverse proxy, or a private network, validate the complete route rather than assuming that the bootstrap address is enough; Kafka clients fetch subsequent broker addresses from metadata.
Encrypt and authenticate connections
Use SSL or, commonly, SASL_SSL for client traffic and any inter-broker traffic that crosses an untrusted network. Install and rotate certificates or SASL credentials using your organization’s normal secret-management process. Test trust chains and hostname verification from a real client host, not only from the broker itself.
Authorize every principal
Enable Kafka authorization and create least-privilege ACLs. Producers should be allowed to write only to the topics they need; consumers should be allowed to read only their topics and consumer groups; administrative identities should be separate from application identities. Authentication without authorization still leaves an authenticated account potentially over-privileged.
Restrict network paths
Permit the client listener only from approved application networks, management access only from administrative addresses, and controller or broker peer traffic only between the hosts that form the cluster. Apply both the VPS firewall and any cloud security-group policy. Record the allowed ports and sources so a later listener change does not silently expose the broker.
Protect data at rest
Kafka’s network security settings do not encrypt message or log data stored on disk. Use filesystem or block-device encryption, or encrypt messages at the application layer, when the data’s confidentiality requirements demand it. Protect backups and snapshots under the same policy.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitchesSize the VPS from the workload
There is no universal production VPS size for Kafka. Capacity depends on message rate, message size, retention period, replication factor, partition count, disk I/O, network throughput, and the number of producers and consumers.
| Planning item | How to use it |
|---|---|
| KRaft controller metadata log | Kafka guidance gives a typical baseline of 5 GB of main memory and 5 GB of disk for a controller metadata log; this is not a broker-sizing guarantee. |
| Broker disk | Allow for retained data, replication copies, segment overhead, recovery headroom, and operating-system space. |
| Memory | Account for the JVM, page cache, active partitions, clients, and any co-located services; more partitions increase metadata and file-descriptor pressure. |
| Network | Include producer ingress, consumer egress, replication traffic, and TLS overhead rather than sizing only for application payloads. |
| Availability | A single VPS cannot tolerate host failure. Use independent hosts and a redundant controller quorum when uptime requirements exceed a single machine. |
Measure the workload you expect, choose a retention and replication policy, and leave operating headroom. The 5 GB controller baseline is useful for a small metadata log, not a recommendation to run a high-throughput broker on a 5 GB VPS.
Validate the installation
Run these checks before handing the broker to an application:
Quick Recap
- Runtime:
java -versionreports Java 17 or 21, and the systemd service uses the same runtime. - Process:
systemctl status kafkashows an active service, or the foreground process remains running without repeated errors. - Storage: the Kafka user can write to the configured log directory and the persistent mount is present after reboot.
- Sockets: host socket tools such as
ss -lntpshow listeners only on intended addresses and ports. - Metadata:
kafka-topics.sh --describereturns the expected topic and leader information. - Data path: a producer can write and a consumer can read a test record using the same security properties as the application.
- Remote route: a client outside the VPS can resolve the advertised name, reach the client port, complete TLS or SASL authentication, and receive usable broker metadata.
Troubleshoot common failures
| Symptom | Likely cause | What to check |
|---|---|---|
| Kafka exits immediately with a Java error | Unsupported or incorrectly selected Java runtime | Run java -version as the service user and inspect JAVA_HOME in systemd. |
| Storage formatting fails | Wrong configuration path, permissions, or an already formatted directory | Use the release’s bundled KRaft configuration, verify ownership, and confirm whether the data directory contains an existing cluster identity. |
| Local clients work but remote clients hang | Unreachable advertised.listeners, DNS, or firewall rules |
Resolve the advertised name from the client host, test the advertised port, and inspect the metadata addresses returned by Kafka. |
| Connection is refused | Listener bound to loopback, service stopped, or port blocked | Check systemctl status kafka, socket bindings, VPS firewall rules, and provider security groups. |
| TLS handshake fails | Untrusted certificate, hostname mismatch, or inconsistent listener mapping | Verify the client trust store, certificate names, security protocol, and listener.security.protocol.map. |
| SASL authentication fails | Wrong mechanism, username, secret, or client-properties file | Compare the client’s mechanism and credentials with the listener configuration and inspect broker logs without exposing secrets. |
| Client authenticates but receives authorization errors | Missing or overly narrow ACLs | Grant the principal the required topic and consumer-group operations, then retest with the least privilege that works. |
| Broker or controller quorum cannot communicate | Private peer listener, voter address, or protocol mismatch | Check controller quorum settings, peer DNS or IP reachability, listener names, and firewall rules between cluster nodes. |
What a safe first deployment looks like
- Install Java 21 or 17 and verify it as the Kafka service account.
- Create the non-root
kafkaaccount and persistent storage. - Download and verify one pinned Kafka release, then extract it under
/opt/kafka. - Generate a KRaft cluster ID and format the release’s bundled configuration.
- Run the broker locally in the foreground and complete topic, producer, and consumer tests.
- Move the same configuration into systemd only after the foreground test is clean.
- Before remote access, replace public
PLAINTEXTexposure with an authenticated, encrypted listener, apply ACLs, restrict firewall sources, and keep the controller private. - Re-test from the actual client network, including metadata discovery, not just the initial TCP connection.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →
