To add semantic metadata to an Avro field, use a custom schema property for descriptive information and a logicalType when the value needs a defined semantic contract, such as a representation, validation rule, or conversion. A logical type keeps the field’s underlying Avro type—and therefore its serialized representation—intact, so a reader that does not recognize the annotation can still use that underlying type.
Choose between a custom property and a logical type
Avro permits attributes that are not defined by the specification as metadata, provided they do not affect the format of serialized data. This makes custom properties suitable for annotations that describe a field but do not change how its value is encoded or interpreted by Avro itself.
| Use | Best for | Effect on serialized data |
|---|---|---|
| Custom property | Descriptive details such as a business concept, data owner, sensitivity class, quality tier, display unit, vocabulary URI, or deprecation status | Must not change the data format |
logicalType |
A semantic type with a defined underlying Avro type and consistent interpretation, validation, or conversion behavior | Uses the underlying Avro type’s encoding |
Do not use logicalType as a label for arbitrary prose. If a value needs only documentation, use the field’s doc attribute or an application-specific property. If it needs a shared contract that software can recognize, define a logical type and specify what values it permits.
Keep the underlying type as the compatibility fallback
A logical type is an Avro primitive or complex type with additional attributes. It is serialized exactly as its underlying type. Avro implementations must ignore an unknown logical type when reading and use the underlying Avro type instead. That behavior makes a logical type an opt-in semantic layer, not a change to the wire encoding.
Free tools Windows power users keep installed
One-click scans. No signup required.
#1 Best Overall
This fallback does not guarantee that every consumer will understand the value’s business meaning. A consumer that sees an unrecognized annotation may still decode a string, bytes, or other base value, but it cannot be assumed to apply your domain-specific interpretation. Document the fallback clearly and test the behavior of the actual runtimes you support.
Example: annotate fields without changing their base types
This record uses standard logical types where Avro has defined them and namespaced custom properties for application-specific meaning:
{
"type": "record",
"name": "Payment",
"namespace": "com.example.billing",
"fields": [
{
"name": "amount",
"type": {
"type": "bytes",
"logicalType": "decimal",
"precision": 12,
"scale": 2,
"com.example.semantic.unit": "USD",
"com.example.semantic.concept": "gross_amount"
},
"doc": "Gross payment amount in US dollars"
},
{
"name": "customer_id",
"type": {
"type": "string",
"logicalType": "uuid",
"com.example.semantic.identifier": "customer"
}
}
]
}Here, bytes remains the underlying type for the decimal, and string remains the underlying type for the UUID. The namespaced properties communicate application-level context; they do not replace the standard logical-type rules.
Use standard logical types when they fit
Avro defines standard logical types for common semantic values including dates, times, timestamps, UUIDs, decimals, and durations. Prefer a standard type when its meaning and constraints match your data: this gives consumers a known convention instead of requiring every application to invent and implement one.
Decimal
The standard decimal logical type annotates bytes or fixed. Its precision must be positive, and scale must not exceed precision. Choose these values to reflect the data contract; they are not merely display hints.
UUID
The standard uuid logical type annotates either a string or a 16-byte fixed value conforming to RFC 4122. Select the representation your producers and consumers can support consistently.
Rank #3
Define a custom logical type as a contract
A custom logical type is appropriate when no standard type expresses the required semantics and producers and consumers need a stable interpretation or validation rule. Before using one, define its name, its one permitted underlying Avro type, the valid values, and what consumers should do when they do not support it.
- Choose a stable name owned by your application or organization.
- Specify the one allowed underlying type, such as
stringorlong. - Define validation constraints, units, timezone behavior, ranges, and nullability where relevant.
- Provide examples and a fallback interpretation for consumers that do not recognize the logical type.
Use a reverse-DNS or similarly controlled namespace for custom property names and logical-type names. This helps distinguish your annotations from standard Avro attributes and from names used by other applications.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problemsImplement a custom logical type in Java
In Java, create a subclass of LogicalType, validate that the schema uses the compatible underlying Avro type, and attach the logical type with addToSchema. The API sets the schema’s logicalType property to the type name and allows additional type-specific properties.
public final class CustomerIdType extends LogicalType {
public CustomerIdType() { super("customer-id"); }
@Override public void validate(Schema schema) {
if (schema.getType() != Schema.Type.STRING) {
throw new IllegalArgumentException("customer-id requires string");
}
}
}The example enforces that this logical type is attached to a string schema. It is a validation example, not a complete conversion implementation: conversion hooks depend on the language binding and datum reader or writer in use. Check the API for the Avro library version deployed by your application.
For registration in Java, use LogicalTypes.register(...) when your application controls startup. Alternatively, expose a public factory through the service-provider file META-INF/services/org.apache.avro.LogicalTypes$LogicalTypeFactory for service-provider discovery.
Govern semantic metadata as part of the schema contract
An annotation can leave the bytes unchanged and still matter to applications. A consumer may rely on a unit, vocabulary, identifier role, or logical type to make a correct business decision. Treat changes to those meanings as schema-governance changes, even when the underlying Avro type has not changed.
Recommended Free Tools
- Keep the base Avro type stable when relying on unknown-logical-type fallback, and document what that base value means.
- Record units, timezone rules, precision and scale, nullability, vocabulary identifiers, and allowed ranges in
docor namespaced properties. - Review whether producers or consumers depend on an annotation before changing or removing it.
- Test writer-reader resolution across the oldest and newest supported Avro runtimes, including a reader that has not registered your custom logical type.
- For object-container-file metadata, do not use names beginning with
avro.; that prefix is reserved. Choose an application namespace such ascom.example.semantic.*instead.
Schema-field properties and object-container-file metadata are distinct places where metadata can appear. The reservation of the avro. prefix applies to object-container-file metadata; application-owned namespaced properties are a clear choice for your own annotations.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
