LLMEasyTools is an alternative to Instructor for agentic program flow

Why I wrote LLMEasyTools

The need to handle agentic program flow

Last year I had the idea to write a Question Answering engine using the Agentic RAG architecture. Initially I tried to use Instructor for the task of getting structured output from the LLM. It is a great library, with many advantages over the other existing solutions. However when doing a call to client.chat.completions.create it always creates the tools list with just one element and I wanted to the LLM to chose the next action from a list of tools. The author explained to me that he wants a consistent output types for that create call - so I don’t expect this to change. This limitation led me to develop LLMEasyTools (LET).

Libraries over frameworks

I like how Instructor stays a simple library despite all the alternatives evolving into frameworks that limit experimentation by enforcing rigid structure how your code is organized. I believe in Write libraries not frameworks. I admire how Instructor expands by adding usage examples into the Cookbook instead of incorporating these code samples into the library itself. I’ll try to follow this approach with LET.

Previous versions of Instructor relied on modifying the OpenAI client directly. I find this less elegant than using standard object-oriented techniques like inheritance or composition. But now it is hidden behind the Instructor API, so this is not why I decided to write my own library. Instead it was trying to use the Union type with Instructor for an agentic workflow.

Using Union type is not the panaceum

To overcome the limitation of just one element tools list the instructor's author suggested to use a Union type for handling multiple tools. But that generated more complicated schemas, often causing the LLM call to fail, and additionally it was breaking for no parameter functions. What could work for that case (or more generally for functions with the same lists of parameters) is making the LLM call just one function with additional parameter specifying the actual function that we want to dispatch to. But I believe that using the OpenAI API as it is designed is a better approach. Pass a tools list instead of doing gymnastics with just one tool packed as a Union type! This is why I created LET.

Example code

Imagine needing functions for sending emails and calculating discounts:

def send_email(recipient: str, subject: str, message: str):
    print(f"Email sent to {recipient} with subject '{subject}' and message '{message}'")

def calculate_discount(price: float, discount: float):
    discounted_price = price * (1 - discount / 100)
    print(f"Original price: ${price}, Discount: {discount}%, Discounted price: ${discounted_price}")

With LET, I can define a tools list with those functions and pass it seamlessly to a chat completions create call on an OpenAI (or compatible) client:

tools=[send_email, calculate_discount]

# Generate the tool definitions (schemas)
tool_defs = get_tool_defs(tools)

# Call the LLM
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=tool_defs,
    tool_choice="auto"
)

# Dispatch
results = process_response(response, tools)

This example involves two functions with different parameters, so it would not require using the function name in parameters as a workaround in Instructor with the Union type. However, the code would still be more complicated:

# Define models for each function's parameters

class EmailParams(BaseModel):
    recipient: str
    subject: str
    message: str

    def process(self):
        send_email(self.recipient, self.subject, self.message)
        

class DiscountParams(BaseModel):
    price: float
    discount: float

    def process(self):
        calculate_discount(self.price, self.discount)
    

# Union model to handle any of the function parameter sets
class FunctionParams(BaseModel):
    params: Union[EmailParams, Discountparams]

    def process(self):
        self.params.process()

# Patch the OpenAI client
client = instructor.from_openai(OpenAI())

# Call the LLM and create the parameters object
action = client.chat.completions.create(
        model="gpt-4o",
        response_model=FunctionParams,
        messages=messages
)

# Dispatch
action.process()