Streaming

The Oxygen API provides the ability to stream responses back to a client in order to allow partial results for certain requests. To achieve this, we follow the Server-sent events standard.

OpenAI official Node and Python libraries handle Server-sent events for you. In Python, a streaming request looks like :

from openai import OpenAI

client = OpenAI(api_key="OXYGEN_API_KEY",base_url="https://app.oxyapi.uk/v1")

stream = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "Say this is a test"}],
    stream=True,
)
for chunk in stream:
    if chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end="")

Parsing Server-sent events

Parsing Server-sent events is non-trivial and should be done with caution. Simple strategies like splitting by a new line may result in parsing errors. We recommend using existing client libraries when possible.