Recommended Free Tools
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Use Spring AI’s reactive ChatClient API and expose its chunks as Server-Sent Events (SSE). In a WebFlux application, add spring-boot-starter-webflux, call chatClient.prompt().user(prompt).stream().content(), map each text chunk to ServerSentEvent, and produce text/event-stream. The browser can then append each named event as it arrives instead of waiting for the complete model response.
This approach keeps your controller independent of a particular model provider, supports cancellation and backpressure through Reactor, and works well for ChatGPT-style token delivery. Servlet applications can use Spring MVC’s SseEmitter instead, but the lifecycle and operational trade-offs differ.
What is the Spring AI SSE streaming pattern?
Spring AI separates model interaction from transport. Its ChatClient reactive API returns a stream of model output; your HTTP layer turns that stream into SSE frames. The client receives multiple events over one response, each terminated according to the SSE protocol, while the connection remains open until generation completes, fails, or is cancelled.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsFor text-only output, use .stream().content(), which yields a Flux<String>. When the client needs usage, finish reasons, or other response metadata, use .stream().chatResponse() and serialize a stable event payload of your own. The documented streaming methods are available in the Spring AI 2.0.1 reference; an older 1.1.8 reference also exists, so match the example to the version declared by your project.
The request and response flow
- The browser opens an SSE endpoint and supplies a prompt.
- The controller creates a Spring AI prompt and subscribes to the provider-backed reactive stream.
- Each emitted text fragment is mapped to an SSE event, such as an event named
token. - WebFlux writes each event as soon as it is available; it does not wait for the whole answer.
- Completion closes the stream. An error after data has been sent must be represented by an event-level convention because the HTTP status can no longer be changed reliably.
Which dependency and application stack do I need?
Use Spring WebFlux for the native reactive implementation. Add the WebFlux starter to the application that owns the controller:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
</dependency>You also need the Spring AI starter for the model provider you have configured. The provider-specific starter and credentials are outside the SSE layer; changing providers should not require changing the controller’s event mapping.
Check the version before copying an example
Spring AI’s current reference in the supplied material is 2.0.1, while a 1.1.8 reference remains available. Confirm the Spring AI version in your Maven or Gradle build and use that release’s API and configuration names. Do not combine dependency versions, configuration keys, or response classes from different release lines without checking their documentation.
How do I create a WebFlux SSE endpoint?
Return a multi-value publisher and declare the SSE media type on the mapping. This minimal controller streams text fragments as named token events:
package com.example.llm;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.http.MediaType;
import org.springframework.http.codec.ServerSentEvent;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import reactor.core.publisher.Flux;
@RestController
@RequestMapping("/api/llm")
class LlmController {
private final ChatClient chatClient;
LlmController(ChatClient.Builder builder) {
this.chatClient = builder.build();
}
@GetMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
Flux<ServerSentEvent<String>> stream(@RequestParam String prompt) {
return chatClient.prompt()
.user(prompt)
.stream()
.content()
.map(text -> ServerSentEvent.<String>builder()
.data(text)
.event("token")
.build());
}
}What each part does
ChatClient.Builderis injected and used to create the application’s client..prompt().user(prompt)supplies the user message..stream().content()requests incremental text and returns a reactive publisher.MediaType.TEXT_EVENT_STREAM_VALUEnegotiatestext/event-streamrather than a single JSON document.ServerSentEvent.builder()sets the event data and an event name that browser listeners can select.
If you do not need custom event names or IDs, a controller can return Flux<String> with the same media type and let the framework serialize the values. Explicit ServerSentEvent objects are usually preferable because they make the wire contract visible and leave room for IDs, comments, retry hints, or different event types.
How do I consume the stream in a browser?
For a simple GET endpoint, the browser’s native EventSource API handles connection management and SSE parsing:
const url = '/api/llm/stream?prompt=' + encodeURIComponent(question);
const source = new EventSource(url);
source.addEventListener('token', event => {
appendText(event.data);
});
source.onerror = () => {
source.close();
};Keep the connection open while tokens arrive and append data rather than replacing the whole answer. Close it when the server completes, when the user cancels, or when your application decides that a reconnect would be unsafe.
When EventSource is not enough
Native EventSource is GET-oriented and does not let you set arbitrary authorization headers or send a JSON request body. For a POST prompt, bearer token in a header, or richer request schema, use fetch() with a streaming reader, or accept the request as a job and let the browser subscribe to a separate GET stream.
A fetch-based reader must parse SSE framing: fields such as event:, id:, and one or more data: lines end at a blank line. Do not assume that one network read equals one model chunk; a read can contain part of an event or several events, so retain incomplete lines between reads.
Should I use content() or chatResponse()?
Choose content() when the browser only needs text. Choose chatResponse() when the UI or application logic needs metadata in addition to text.
| Spring AI call | Emitted value | Use it when | Typical SSE payload |
|---|---|---|---|
.stream().content() |
Flux<String> |
You are rendering an answer incrementally and do not need provider metadata. | A token event containing a text fragment. |
.stream().chatResponse() |
Reactive ChatResponse objects |
You need generation information, usage data, finish details, or response-level metadata. | A JSON DTO with text, metadata, and a stable event schema. |
Do not expose provider-specific response objects directly as your long-term browser contract. Map them to a DTO that your frontend owns. That lets you change model providers or Spring AI versions without forcing a client rewrite.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →Example of a richer event contract
record LlmEvent(String type, String text, Object metadata) {}
@GetMapping(value = "/stream-with-metadata",
produces = MediaType.TEXT_EVENT_STREAM_VALUE)
Flux<ServerSentEvent<LlmEvent>> streamWithMetadata(@RequestParam String prompt) {
return chatClient.prompt()
.user(prompt)
.stream()
.chatResponse()
.map(response -> {
var event = new LlmEvent(
"delta",
response.getResult().getOutput().getText(),
response.getMetadata());
return ServerSentEvent.<LlmEvent>builder()
.event("response")
.data(event)
.build();
});
}The exact accessor methods on response objects can vary by Spring AI release and provider adapter. Verify them against the version in your build; the important design is to map each response to an application-owned event type.
How can I signal completion and errors?
A normal completion is represented by the publisher completing, which closes the HTTP response. Many applications also send an explicit terminal event so the browser can update state deterministically before the connection closes.
Flux<ServerSentEvent<String>> tokens = chatClient.prompt()
.user(prompt)
.stream()
.content()
.map(text -> ServerSentEvent.<String>builder()
.event("token")
.data(text)
.build());
Flux<ServerSentEvent<String>> completed = Flux.concat(
tokens,
Flux.just(ServerSentEvent.<String>builder()
.event("complete")
.data("{}").build()));Handle failures before the response commits with a suitable HTTP error where possible. Once token bytes have reached the client, an HTTP status change is generally no longer useful. Define an event such as error with a safe code and user-facing message, then terminate the stream.
Flux<ServerSentEvent<String>> safeStream(String prompt) {
return chatClient.prompt()
.user(prompt)
.stream()
.content()
.map(text -> ServerSentEvent.<String>builder()
.event("token")
.data(text)
.build())
.onErrorResume(error -> Flux.just(
ServerSentEvent.<String>builder()
.event("error")
.data("generation_failed")
.build()));
}Log the server-side exception with a correlation identifier, but do not send stack traces, provider credentials, or raw prompts to an untrusted browser.
How do I add heartbeats to a long-running stream?
Quiet connections can be closed by browsers, proxies, load balancers, or network equipment before the model emits another token. Periodic heartbeat traffic helps detect a disconnected client and keeps idle timeouts from firing when infrastructure permits it.
SSE supports comment lines, but an explicit no-op event is easier to observe and instrument. Merge a heartbeat publisher with the model stream and stop it when the model completes:
Flux<ServerSentEvent<String>> modelEvents = chatClient.prompt()
.user(prompt)
.stream()
.content()
.map(text -> ServerSentEvent.<String>builder()
.event("token")
.data(text)
.build());
Flux<ServerSentEvent<String>> heartbeats = Flux.interval(Duration.ofSeconds(15))
.map(tick -> ServerSentEvent.<String>builder()
.event("heartbeat")
.data("{}").build());
return modelEvents
.takeUntilOther(heartbeats.ignoreElements().then());The exact composition should ensure that the heartbeat publisher is cancelled as soon as generation finishes. A simpler production implementation can use a periodic comment or a framework-supported keep-alive operator; test the resulting lifecycle and verify that heartbeats do not become visible answer content.
How do WebFlux and Spring MVC compare for SSE?
WebFlux is the natural choice when the rest of the application already uses Reactor and you want cancellation and backpressure represented directly. MVC is often easier to introduce into an existing servlet application, where SseEmitter provides an asynchronous send-and-complete lifecycle.
| Concern | WebFlux | Spring MVC |
|---|---|---|
| Streaming abstraction | Returns Flux<ServerSentEvent<T>> or another reactive publisher. |
Returns SseEmitter and calls send as chunks arrive. |
| Backpressure and cancellation | Native Reactor signals can propagate demand and cancellation upstream where supported. | Servlet asynchronous processing requires explicit lifecycle handling; backpressure is less direct. |
| Existing application fit | Best when the application is already reactive or can adopt WebFlux. | Best when the application is servlet-based and replacing its web stack is undesirable. |
| Event serialization | Generic event data and media-type negotiation are handled through reactive codecs. | SseEmitter.event() gives direct control over event fields while you send. |
| Operational behavior | Still depends on proxy buffering and idle timeouts; a reactive stream does not override intermediary settings. | Still requires servlet async timeouts, executor sizing, proxy configuration, and disconnect handling. |
MVC example with SseEmitter
@RestController
@RequestMapping("/api/llm")
class MvcLlmController {
private final ChatClient chatClient;
MvcLlmController(ChatClient.Builder builder) {
this.chatClient = builder.build();
}
@GetMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
SseEmitter stream(@RequestParam String prompt) {
SseEmitter emitter = new SseEmitter();
chatClient.prompt()
.user(prompt)
.stream()
.content()
.subscribe(
text -> {
try {
emitter.send(SseEmitter.event()
.name("token")
.data(text));
} catch (Exception sendError) {
emitter.completeWithError(sendError);
}
},
emitter::completeWithError,
emitter::complete);
return emitter;
}
}In a real MVC service, retain the subscription or cancellation handle, register completion and timeout callbacks, and ensure that a client disconnect stops unnecessary model work. Also review which executor performs the subscription and sends; unbounded blocking work on servlet or shared reactive threads can exhaust capacity under concurrent streams.
How do I test an SSE LLM endpoint?
Start with a command-line request that disables client-side buffering and displays events as they arrive:
curl -N -G
-H 'Accept: text/event-stream'
--data-urlencode 'prompt=Tell me a short joke'
http://localhost:8080/api/llm/streamThe -N option asks curl not to buffer output. You should see event fields arriving over time, for example:
event: token
data: Why did the developer...
event: token
data: ...
event: complete
data: {}
Test the behavior, not only the status code
- Verify the response content type is
text/event-stream. - Confirm the first event arrives before the complete model answer is available.
- Check that multiple chunks are preserved in order.
- Close the client midway and confirm upstream work is cancelled or stopped when the provider supports cancellation.
- Force a provider error before the first event and after several events; verify both paths are understandable to clients.
- Leave a stream quiet long enough to exercise heartbeat and intermediary idle-timeout behavior.
- Test prompts containing Unicode, newlines, and characters that require URL encoding.
What production settings prevent broken or delayed streams?
Most streaming failures occur outside the controller. Treat the entire path—from model provider to browser—as a long-lived connection.
Disable unintended buffering
Reverse proxies and load balancers may buffer upstream output and release it in larger blocks. Configure them to pass text/event-stream incrementally, and verify the behavior through the same proxy path used by production clients. A correct WebFlux publisher cannot make a buffering intermediary deliver token-level latency.
Set compatible idle and request timeouts
Choose proxy, load-balancer, server, and client timeouts longer than the longest expected quiet interval, or emit heartbeats more frequently than the shortest idle timeout. Keep these values documented together; changing one layer can make an otherwise healthy stream appear to fail randomly.
Propagate cancellation
When a browser closes the connection, Reactor cancellation should travel through the chain. Confirm that the provider adapter honors cancellation and that any custom operators, queues, or blocking bridges do not continue consuming model output after the client is gone. Cancellation saves provider usage and worker capacity.
Protect the endpoint
- Authenticate the caller and authorize access to the requested conversation or tenant.
- Apply per-user and per-IP rate limits because one request occupies a connection for an extended period.
- Limit prompt size and reject obviously excessive requests before invoking the model.
- Set concurrency limits so a burst of streams cannot exhaust memory, connection pools, or provider quotas.
- Validate query parameters and avoid reflecting untrusted values into event fields.
Handle sensitive data carefully
Prompts and generated text may contain personal, confidential, or regulated information. Avoid indiscriminate logging, redact payloads in traces, and use correlation IDs instead of recording complete conversations in every network component. Ensure that metrics count events and durations without exposing their contents.
Free tools Windows power users keep installed
One-click scans. No signup required.
What common problems stop tokens from appearing?
The client receives one large response at the end
Check that the mapping produces text/event-stream, that the return type is a multi-value publisher, and that no operator collects the entire stream into a list or string. Then inspect proxy buffering and test the application directly with curl before testing through a gateway.
The browser reports a connection error after some tokens
Look for intermediary idle timeouts, server-side async timeouts, and provider exceptions. Add a terminal error event, emit heartbeats during quiet periods, and correlate the browser close time with server logs. A status code cannot reliably describe an error after bytes have already been sent.
EventSource reconnects unexpectedly
EventSource automatically attempts reconnection for many failures. Decide whether replay is safe for your application; otherwise close the source on an application-level error and show a retry control. If replay is required, design event IDs and resume semantics deliberately rather than rerunning the prompt blindly.
The controller does not compile
Check imports and release compatibility first. The documented .stream().content() pattern is associated with Spring AI 2.0.1 in the supplied reference, while older releases may expose different response accessors or configuration. Align Spring AI modules and provider starters to one release line.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteChunks contain awkward word breaks
Model providers do not promise word-level chunks; a fragment can end in the middle of a word, punctuation mark, or Unicode sequence. Append fragments exactly as received. If your UI needs markdown rendering, buffer only at the presentation layer and never change the server’s event ordering.
How should I design an SSE contract that can evolve?
Define event names and payloads before publishing the endpoint to frontend teams. A practical contract can include token, complete, error, and heartbeat. Keep each data payload machine-readable where possible, and reserve a field for a request or stream identifier.
| Event | Purpose | Client action |
|---|---|---|
token |
Incremental answer text. | Append the data to the current assistant message. |
complete |
Generation ended normally. | Mark the message final and close the client stream. |
error |
Generation or transport failure after streaming began. | Stop rendering, retain already received text, and show a retry option. |
heartbeat |
Connection liveness signal. | Ignore for answer rendering and update liveness metrics. |
If you add event IDs, make them monotonic within a stream and document whether a reconnect can resume from an ID. SSE reconnection alone does not provide model-generation replay; replay requires storing or deterministically regenerating the response.
Is provider portability preserved when streaming?
Yes, at the controller boundary. Spring AI presents a provider-portable streaming abstraction, so the endpoint can continue consuming a reactive stream while model-provider configuration changes. Portability is strongest when your code depends on content() or an application-owned DTO rather than provider-specific response fields.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Provider differences still matter for token timing, finish metadata, tool calls, usage reporting, error types, and cancellation behavior. Keep those differences behind a service or mapper and test each configured provider’s actual event sequence before switching production traffic.
Which implementation should I choose?
Choose WebFlux when you can model the application as reactive and want a direct Flux-to-SSE pipeline with cancellation semantics. Choose MVC with SseEmitter when the application is already servlet-based and replacing its web stack would create more risk than value.
For a first implementation, start with text-only content(), explicit token and complete events, direct curl testing, and proxy configuration. Add richer response metadata, authentication headers, resumable IDs, and advanced heartbeat behavior only when the client contract requires them.
Frequently Asked Questions
Can I stream a Spring AI response over a POST request?
Yes, but native browser EventSource is designed around GET. Use fetch with a streaming response reader for a POST body or authenticate through headers, or submit the prompt as a job and open a separate GET SSE subscription.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →How do I prevent duplicate text after an SSE reconnect?
Do not blindly rerun the prompt on reconnect. Add event IDs and a server-side replay strategy, or treat reconnect as a new generation and let the client discard the abandoned stream.
Can SSE send JSON instead of plain text?
Yes. Map each response to a DTO and pass it as the event data; the browser receives the serialized JSON string and can parse it. Keep the schema stable and separate metadata events from text-rendering events.
Does SSE work through HTTP/2?
SSE is an application-level event format and can be carried over HTTP/2, but intermediary buffering and stream limits still determine practical latency. Validate the complete deployment path rather than assuming the protocol version solves delivery delays.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
