What is a response?

The Missing Piece in LLM Compatibility Layers: Comprehensive Documentation

In the Python world there are two popular packages that offer universal APIs for LLMs and that are pure compatibility layers, not frameworks with additional functionality: LiteLLM and aisuite. I think it's great that these tools exist—after all, it’s incredibly frustrating to discover that the shiny new model solving all your problems can’t be used because it’s from a provider you didn’t initially target. Compatibility layers let you switch providers with minimal effort.
But … Neither of them documents the results of their most basic functions! Oh come on!

There are code examples like:

response = completion(model="command-nightly", messages=messages)
print(response)

for LiteLLM. You are supposed to print out the object and deduce what fields it has, try methods you maybe know from other LLM libraries and hope they work.

Alternatively, you can try to follow the source code. But good luck with that—starting from __init__.py (over a thousand lines long) quickly sends you down a rabbit hole of imports and indirection.

Or take aisuite. Their first example looks like this:

response = client.chat.completions.create(
    model=model,
    messages=messages, 
    temperature=0.75
)
print(response.choices[0].message.content)  

From this, you can gather that the response object has a choices attribute, which is a list of message objects, that have a content field. And that’s… all the documentation tells you.

Sure, this might be enough to throw together a quick script. But is that the goal of a compatibility layer? The main function of a compatibility layer is to declare what is universal—what you can reliably access across different providers—and what is provider-specific and needs to be used with care.

Type hints could be a good first step here - a quick way to find the definitions of Response and Messages classes is a must!

---