Free tools Windows power users keep installed
One-click scans. No signup required.
Kafka can carry PDF, ZIP, and other files as binary record values. To keep each filename, send it separately as record metadata—commonly a Kafka header—and ensure every producer, Kafka Connect component, connector, and consumer preserves both the bytes and the metadata. A header alone does not guarantee that a complete pipeline will retain the name or the original file bytes.
How to represent a file in a Kafka record
Treat the file as an opaque sequence of bytes: place those bytes in the record value and put the filename in metadata. Kafka records have a key-value shape with associated metadata, and records support headers, as described in the Kafka protocol documentation and the Kafka 4.2.2 Header API.
For example, a record can have the original file bytes as its value and a header named filename whose value is report.pdf. The header name is an application convention, not a Kafka-mandated name. Your producer and consumer must agree on the header key, how its value is encoded, and how duplicate or missing filename headers are handled.
Choose where the filename belongs
| Approach | Payload representation | Filename location | Key consideration |
|---|---|---|---|
| Binary value with a header | Original file bytes | Kafka record header, such as filename |
Check that every connector and consumer in the route preserves and reads the header. |
| Structured envelope | File bytes plus metadata in an application-defined structured representation | Inside the payload envelope | All participating applications must agree on the envelope format and how the binary data is represented. |
| Base64 text representation | Base64-encoded bytes represented as text | Header or envelope, according to the application contract | This is an encoded representation, not the original byte sequence; the receiver must decode it to reconstruct the file. |
Kafka documentation establishes the mechanisms for binary records and headers, but does not provide performance benchmarks or a universal recommendation between a header and an envelope. Choose the representation your full route can preserve reliably, and document it as part of the producer-consumer contract.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
#1 Best Overall
Configure Kafka Connect without changing the file bytes
Kafka Connect supports transformations for inserting headers and for copying or moving record fields into headers. The current Kafka Connect user guide documents these header transformations. Use them only where they fit your record structure and verify that the destination connector retains the resulting headers.
Do not assume every conversion in a Connect pipeline is byte-transparent. The Kafka Connect 3.6 user guide documents a Cast transformation that can convert binary data to a string using Base64 encoding. If the destination needs the original bytes, avoid converting the value to text. If Base64 is intentionally the contract, ensure the receiving application decodes it before writing the file.
Verify the complete route, not just the producer
Apache Kafka’s design documentation describes a standardized binary message format shared by producer, broker, and consumer so data chunks can pass through that path without modification. That platform design does not, by itself, establish that an application serializer, Connect converter, transformation, or third-party connector will preserve your data and headers. The Kafka design documentation explains the binary-message approach; the behavior of your complete connector chain depends on its components and configuration.
- Producer: Serialize the file as bytes, without converting it to a character string or otherwise changing the sequence. Add the filename using the header key and value format agreed with the consumer.
- Kafka Connect converters and transformations: Check that the value remains binary and that any transformation adding or relocating filename metadata produces the expected header. Avoid a binary-to-string cast unless Base64 text is intended.
- Connectors and intermediate services: Confirm that each component in the actual route supports and preserves record headers as well as the value. Do not infer third-party connector behavior from Kafka’s header API.
- Consumer: Read the record value as bytes, retrieve the expected filename header, and write the value to storage without text decoding or other transformations when raw-byte fidelity is required.
- End-to-end check: Send representative PDF, ZIP, and other binary files through the deployed route. Compare the consumed bytes with the source bytes and verify that the expected filename arrives. Include any required Base64 decoding in that comparison if your contract uses Base64.
Account for file size and operational limits
Kafka’s binary record format does not establish a universal size threshold for sending a whole file as one record, nor does it define a complete architecture for large-file streaming. Before adopting one-record-per-file, check the limits and behavior of your deployed broker, producer, Connect workers and connectors, consumer, retries, and memory usage using representative files. If those constraints make whole-file records unsuitable, select a different design deliberately; the cited documentation does not establish one universally appropriate large-file pattern.
Rank #3
Version and compatibility checks
Kafka configuration and connector behavior can vary by version. The cited materials span the current Kafka Connect guide on the project’s trunk, the Kafka 3.6 Connect guide, and Kafka 4.2.2 header API documentation. Check the documentation for the versions actually deployed, particularly when relying on a transformation, converter, or connector’s header behavior.
Quick Recap
Best Value
Rank #4
- Metamorphosis: Franz Kafka (Little Clothbound Classics)
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
