The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
To send a large payload through Amazon SQS or SNS from Kotlin without using AWS’s Java extended-client library, store the payload in Amazon S3 and send a pointer to the object as the queue or topic message. Your Kotlin code must implement the compatible upload, pointer, retrieval, and cleanup flow; AWS documents Java extended clients, not a Kotlin-native extended client.
How S3 offloading works
SQS and SNS have a standard message-size limit of 256 KB, while AWS’s extended-client documentation describes handling payloads up to 2 GB. That 2 GB figure is a documented library capability, not a promise about latency, throughput, or behavior in every configuration. The pattern keeps the large content in S3 and sends a smaller reference through SQS or SNS.
- Serialize the payload and decide whether it exceeds your offload threshold.
- Upload the payload to an S3 bucket and retain the bucket and object key.
- Publish a pointer envelope containing the agreed reference format through SQS or SNS.
- Have the consumer recognize that envelope, retrieve the object from S3, and deserialize the payload.
- Delete the object according to an explicit retention and failure policy.
A message consumer that expects the full payload in the message body may instead receive only the pointer. Every producer and consumer must therefore agree on the pointer contract; S3 offloading is not automatically transparent to ordinary clients.
Choose between a Java client and a Kotlin-owned adapter
| Choice | What you get | What your application must own |
|---|---|---|
| AWS Java extended client called from Kotlin | The documented SQS or SNS extended-client behavior and its established reference handling. AWS identifies these libraries as Java libraries. Kotlin/JVM can interoperate with Java, but that interoperability is a language-platform inference, not a claim made in the AWS guides. | Integrating the library into your Kotlin application and ensuring all relevant consumers use compatible handling. |
| Kotlin-owned S3-offload adapter | Control over the Kotlin-facing API and implementation choices, without taking a dependency on the Java extended-client library. | Compatible upload, pointer creation, retrieval, retries, cleanup, and error handling. You must also define and maintain a pointer format all publishers and consumers understand. |
Avoiding the Java extended client does not avoid the extended-client protocol. It means your team takes responsibility for implementing and operating compatible behavior. AWS describes the SQS Java library as using S3 to hold the payload and placing a reference in SQS in its large-message guide; its SNS Java extended-client guide describes the corresponding SNS approach.
#1 Best Overall
Design the Kotlin flow and its failure handling
Choose when to offload
You can offload only messages above a configured threshold or route every payload through S3. AWS documents both threshold-based and always-through-S3 options for its extended clients. A threshold avoids putting smaller messages through the S3 path; always-through-S3 gives the application one storage path but makes every message depend on it.
Define the pointer contract
Choose a stable envelope format and make its meaning explicit to every consumer. Include enough information to locate the object, such as its bucket and key, and any version or integrity metadata your design requires. These are implementation choices, not a Kotlin API or reference format specified by AWS for your custom adapter.
Rank #2
Align S3 access and encryption
Select the bucket and region deliberately, and grant publishers permission to upload and consumers permission to retrieve objects. Configure encryption to match your requirements and ensure the relevant principals can use the encryption key. AWS’s SNS extended-client configuration documents bucket selection and custom KMS key support; your own adapter must implement equivalent choices if needed.
Handle retries, partial failures, and cleanup
The upload and message publish are separate operations. If upload succeeds but publishing fails, the object may be orphaned; if a message is delivered but the object is unavailable, the consumer cannot reconstruct the payload. Define retry and idempotency behavior for both sides, and choose how long objects remain available. Use an explicit deletion or retention policy rather than assuming message deletion also removes the S3 object.
SNS-to-SQS requires compatible consumers
For an SNS topic with an SQS subscription, the SQS consumer must understand the S3 reference and retrieve the object. AWS’s SNS example uses the SQS extended client for retrieval and configures raw message delivery in that example. That setting is part of the documented example, not a universal requirement for every custom integration. If you use your own Kotlin adapter, ensure the delivery format and pointer interpretation match what your consumer expects. See AWS’s SNS large-message example.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What the 2 GB figure does—and does not—mean
AWS describes SQS Java extended-client payloads from 256 KB up to 2 GB, and the AWS Labs SNS Java repository also states a 2 GB maximum. Read that as the documented capability of those extended-client approaches, not as a universal SQS or SNS message limit or a performance guarantee. AWS announced the SNS extended-client library supporting payloads up to 2 GB in 2020. The SQS Java extended-client repository and SNS Java extended-client repository provide project details; repository dependency versions can change and should not be assumed current without checking them.
AWS’s documentation describes Java extended clients rather than a Kotlin-native library. The SNS Python extended-client guide documents a separate language-specific library; it does not establish a Kotlin implementation.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
