To change an Avro or Protobuf schema safely, check how each consumer reads the data—not just whether a compatibility checker approves the schema. Avro resolution depends on both the writer and reader schemas; Protobuf binary compatibility depends on stable field numbers and safe wire-level changes; ProtoJSON has different rules again. Test the old/new producer-consumer combinations your rollout can create, including application behavior and stored data.
What “compatible” means depends on the data format
A schema change can parse successfully and still break an application, discard information, or fail during a rolling deployment. Start by identifying every representation in use and the versions that may overlap. A service may exchange Protobuf binary messages internally but expose the same messages as JSON externally; those paths need separate reviews.
| Representation | What identifies fields | What the reader needs to know | Common compatibility trap |
|---|---|---|---|
| Avro | Record field names are matched during schema resolution. | The writer schema as well as the reader schema. | A new reader field has no reader-side default, or a writer emits an enum symbol or union branch the reader cannot resolve. |
| Protobuf binary | Field numbers identify fields on the wire. | The schema for interpreting known fields; unknown fields are generally ignored by older readers. | Reusing a removed field number, or writing values an older reader cannot represent safely. |
| ProtoJSON | JSON field names and JSON representations, not binary field numbers. | Field names, enum representation, and unknown-field handling. | An older JSON client rejects a newly emitted field even though an older binary reader would ignore it. |
These distinctions follow the Apache Avro specification and the Protocol Buffers documentation for binary evolution and ProtoJSON. There is no single compatibility label that covers all three formats, both rollout directions, and application-level meaning.
How to evolve Avro schemas
Keep the writer schema available
Avro resolution uses the schema that wrote the data together with the schema the reader wants to use. Avro binary data does not carry field names and type information in the way a self-describing JSON object does, so systems that store or transmit Avro need a dependable way to retain and supply the writer schema. Apache Avro’s specification also describes Parsing Canonical Form for normalizing schemas and identifying schemas that are equivalent for parsing.
Recommended Free Tools
#1 Best Overall
In practice, keep each writer schema available for as long as records produced with it may still be read. A schema registry or another schema-distribution mechanism can be an operational way to do this; whichever mechanism you use, verify that consumers can retrieve the schema needed for historical and queued data.
Add fields with older records in mind
Avro matches record fields by name, not position. Reordering fields is therefore supported by schema resolution. If the writer has a field the reader does not define, the reader ignores it. If the reader has a field absent from the writer schema, the reader uses a default from its own schema; without that default, resolution fails.
A default is reader-side resolution behavior. It supplies a value when reading data written without that field; it does not instruct an encoder to omit the field when its value equals the default. Check both the schema-resolution result and how your actual writers encode records.
Rank #2
Check type, enum, and union changes in both directions
Avro allows specific writer-to-reader type promotions: int to long, float, or double; long to float or double; float to double; and string and bytes in either direction. These are directional resolution rules, not general permission to change types. Test the exact old-writer/new-reader and new-writer/old-reader combinations that can occur in your rollout.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →For an enum, resolution fails if the writer emits a symbol the reader does not define, unless the reader schema supplies an enum default. For a union, the reader must have a matching branch for the writer’s value. Exercise newly introduced symbols and branches with real records: a schema that passes for common values may still fail on less frequent historical or newly produced data.
How to evolve Protobuf binary schemas
Treat field numbers as permanent
Protobuf’s binary wire format uses field numbers as field identities. Do not change the number of a deployed field. When removing a field, reserve its number so a later field cannot accidentally reuse it; reserve its name as appropriate too. Reuse can cause old data to be interpreted as a different field. The Protocol Buffers binary evolution guide describes adding fields as wire-safe: new readers can read old messages using defaults, while old readers ignore unknown fields.
That wire-level rule does not guarantee safe application behavior. Code may assume a particular set of fields, reject a message during validation, or mishandle data after parsing. Review those assumptions along with the schema change.
Separate parse compatibility from value safety
Some type changes can parse across schemas but are only conditionally safe. For example, changing int32 to int64 may work while values stay within the older type’s range. If a new producer writes a larger value, an old reader can truncate it. Keep writes within the range older readers support until every relevant endpoint has the updated schema; only then expand the range. This strategy depends on coordinated rollout, so it is unsuitable when you cannot establish that all consumers have upgraded.
Adding enum values is wire-safe, but generated application code can still be affected. An exhaustive switch over known enum cases may fail to compile after regeneration or behave incorrectly when it encounters a value it does not recognize. Check the generated code and consumer handling in each target language.
Rank #4
Review presence and required-field assumptions
With implicit presence, Protobuf omits default-valued scalars from serialization. A consumer may therefore be unable to distinguish “not set” from “set to zero,” false, or an empty string. If that distinction matters, use a presence-aware representation and verify the generated API and runtime behavior for your language. Proto3 supports optional for explicit presence; the Protobuf documentation recommends it as a smoother path toward Editions.
Avoid adding required fields to new evolution-friendly schemas. The Protocol Buffers Style Guide cautions against them because future changes may need to stop setting a field, and middleware that forwards messages may not understand every required field.
Why ProtoJSON needs its own review
Do not infer JSON compatibility from binary compatibility. ProtoJSON generally does not propagate unknown fields, so an older JSON client can reject a new field emitted by an updated producer. Deploy readers that understand the field before enabling producers to emit it, or deliberately use a documented ignore-unknown-fields option where the application and runtime support it.
Review JSON field names and representations as well as binary field numbers. The ProtoJSON guide identifies changes that are unsafe for JSON, including changing between string and bytes, between a message and bytes, or from optional to repeated. Changing a field number does not affect ProtoJSON parsing because JSON does not use field numbers, but it remains unsafe for binary messages and should not be treated as an acceptable cross-format change.
A rollout procedure for mixed-version systems
- Inventory the paths. Record whether each producer, consumer, queue, file, API, and intermediary uses Avro, Protobuf binary, ProtoJSON, or more than one representation.
- Map versions that can overlap. Write down current producer and consumer schemas, deployment order, and whether stored or queued data can outlive a rollout.
- Test Avro resolution with actual writer schemas. Check reader defaults, supported promotions, enum symbols, and union branches for historical and planned records.
- Check Protobuf binary invariants. Keep deployed field numbers stable, reserve removed identifiers, and examine type ranges, enum handling, presence, and generated-code assumptions.
- Review ProtoJSON independently. Check field names, enum output, unknown-field behavior, and whether readers will be upgraded before producers emit new fields.
- Exercise every possible direction. Test old producer to new consumer and new producer to old consumer wherever those combinations can occur. Include historical records and intermediary services that parse and reserialize messages.
- Gate new writes. Do not enable a new field, enum value, union branch, or expanded value range until the consumers that may encounter it can handle it safely.
- Review meaning as well as parsing. Confirm that validation, business logic, and generated APIs preserve the intended behavior; passing a schema compatibility check is necessary but not sufficient.
Choose tests based on the change, not a single compatibility label
For each proposed change, record the format, reader/writer direction, field identity rule, default or presence behavior, unknown-field behavior, possible information loss, and whether deployment coordination is realistic. A change that is safe for one of those dimensions can still fail another. The useful release decision is therefore based on the actual representations and overlap states in your system, not on the word “compatible” alone.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
