Server-Sent Events

What is a Server-Sent Event?

A server-sent event (SSE for short) is a communication protocol that utilizes a persistent HTTP connection to enable a server to send data to a web page at any time by pushing events to the client in text/event-stream format. If there is a payload attached to the event, it is accessed via the “data” field (i.e., event.data). This differs from traditional HTTP polling in which the web page must send a request to the server in order to receive new data.

Text/Event Stream Format

text/event-stream format is the MIME type (a MIME type is a standardized format for a document, file, or assortment of bytes) used by SSEs. Each message in the data stream is a plain-text block comprised of one or more fields, separated by double newline characters (\n\n).

Key fields include:

• data: carries the actual payload

• event: an optional name for the event type

• id: the last-event ID, used to automatically reconnect if connection is dropped

• retry: indicates length to wait before reconnecting (in milliseconds)

• : A colon with nothing preceding it is often used to keep the connection alive. Any text after the colon is ignored by the client.

Example event stream:

: connection established

event: status:
data: {"connected": true}

data: Hello world

data: Hello world
data: but with two lines

id: 4
event: update 
data: {"temperature": 67, "humidity": 0.4}
retry: 3000

When the SSE is established, the client reads the byte stream continuously, buffering each incoming line until reaching the double newline, then it treats everything buffered to that point as one event. In essence, SSE is just structured plaintext over an indefinite HTTP response.

Server-Sent Events in the Real World

SSE vs. WebSockets

SSEs are similar to WebSockets as both use a continually open line of communication, but differ in that the connection is one-way, not bi-directional (only the server can send events to the client). This makes SSEs desirable in applications where only the server needs to send data and the client acts as a medium to display, store, or do something with it. Of particular interest in today's technology landscape is streaming AI API responses. Other popular use cases include social media feed or news feed updates, weather data, and stock tickers.

The EventSource API

The EventSource API offers a convenient interface for consuming and parsing server-sent events. The drawback is that EventSource only supports GET requests since it is designed for simple subscription-style streams, making it a poor choice for processing APIs that use POST requests. Below I will briefly introduce the EventSource API, then demonstrate two real-world examples of implementing SSE (one using EventSource, one without).

An EventSource instance takes in the url of the script that generates the events. If the event generator is hosted on another origin (commonly a decoupled API), the withCredentials option must be set to true

const evtSource = new EventSource(
        "//api.example.com/sse-demo.js”, 
        {withCredentials: true,}"
    );

Once a connection is established, you can listen for events in two ways:

• Using the onmessage method to listen for all events.

• Listen for named events by adding event listeners. Listener functions should take in an event, which can then be used to access any data attached.

// Option 1: 
evtSource.onmessage(e){
    console.log(`unnamed event received:${e.data}`);
    // Do something with the data
}

// Option 2: listen for named events by adding Event Listeners 
evtSource.addEventListener("new-message", (e) {
    console.log(`\({e.event} event received: \){e.data}`);
}

Simulating an AI-style response:

I created a GitHub repo which provides two sample implementations of server-sent events: one utilizing the EventSource API (eventsource), the other manually parsing the data stream with fetch (manual). What makes SSE format powerful is that the structure provides named event types, a delimiter between events, and an id field for resumeability.

When the server sends an event it is doing three things:

1. Setting Content-Type: text/event-stream

2. Keeping the connection open (handled by FastAPI's StreamingResponse)

3. Writing strings in a format that follow SSE naming and newline conventions (discussed above)

To achieve this, we can use a simple format_sse function that takes in an event name and a dictionary containing all data as key-value pairs, then return a string with the newline characters integrated to adhere to SSE guidelines:

def format_sse(event: str, data: dict) -> str:
    """Format a single SSE event with the given event type and JSON data."""
    payload = json.dumps(data)
    return f"event: {event}\ndata: {payload}\n\n"

Tradeoffs

When using EventSource, you are trading convenience and out-of-the-box functionality for flexibility. EventSource will automatically retry to connect to the server if something happens to the connection, whereas you'd have to detect it and rebuild the connection yourself with fetch. Additionally, EventSource has built-in resumeability, automatically sending a Last-Event-ID header to the server to pick up where it left off. Finally, EventSource offers standardized error handling, with clear readyState values (CONNECTING, OPEN, CLOSED) and an established lifecycle. However, with fetch you need to implement retry logic, backoff timing, and a way to resume the stream yourself.

Conversely, the manual approach offers some functionality that isn't possible with the EventSource API. EventSource only supports GET requests, but by using fetch and ReadableStream you can process a POST request, which would allow you to send a prompt in the body (important for AI APIs since sending it in the URL is not feasible or recommended for production). Additionally, you can set custom headers, such as auth tokens or custom content types (worth noting that while native browser EventSource does not support this, but can be included with polyfill). Finally, ReadableStream provides the ability to implement backpressure (a mechanism for dealing with a queue growing faster than the consumers can process events) at the application-level.

For example, if you have a stock ticker that pushes hundreds of updates per second via SSE to be processed by EventSource, the only backpressure mechanism it has is TCP-level flow control underneath, which eventually slows down the server's writes. The issue is that this is a blunt safety net, not something your application controls. Meanwhile you could keep the application data more fresh with a manual parser by reading everything that accumulated since your last read and discarding everything except the latest event, or aggregating them into a summary. With EventSource, each accumulated event becomes a separate callback queued on the event loop, and your handler runs them sequentially, regardless of how stale they are, meaning there is a gap between what is actually happening with the data and what the user sees.

Taken together, if you need a quick and reliable method for simply listening to SSEs, then use EventSource. If you require more control over headers or backpressure, or need to send data from the client in a POST request, opt for the fetch method.