Separating Data from Instructions

Introducing Prompete - a LiteLLM wrapper for prompting with templates

I have been working on Prompete — my small wrapper around LiteLLM — for a few weeks now and I just discovered that Anthropic recommends separating data from instructions in their latest online course (hat tip to Ruben Hassid). I am excited — this is my main idea behind this library. It is a Claudette inspired API plus templates for LiteLLM.

I remember how templates were introduced to web programming in the first internet boom - we are now at a similar point. People are just starting to realise that keeping long text in code is not very practical. Such text disrupts the visual code structure, changes according to different patterns and timelines compared to the code itself, and should ideally be separated out. The simple text substitutions currently used for prompt templates are somewhat rudimentary, highlighting the need for more specialised templating solutions.

The analogy The analogy extends beyond the surface; it is also structural, it is about connecting traditional software systems that work with data structures to something closer to human language—whether HTML web interfaces or LLM instructions. I believe that integrating LLMs with traditional software systems is the most promising area for experimentation right now. We’ll see many more patterns for these hybrid systems than just one-pass RAG systems or agents that don’t work well because LLMs aren’t yet very good at planning. We can leverage traditional software where it excels—such as using well-established algorithms for planning and deduction-based reasoning—while employing LLMs where they shine, namely in handling higher-level concepts and translating them into more precise machine code.

Traditional software works best with data structures, while LLMs require text-based instructions. LLM can work with data structures as well, but only if they are accompanied by instructions. Perhaps in the future, we’ll have separate inputs for LLMs: one channel for data structures and another for semantic instructions—just as we can currently feed them text and images. For now, however, the best way to connect text and data structures is through the use of templates.

In Prompete prompts have two parts: the first is the data expressed as a simple dataclass:

@dataclass(frozen=True)
class TaskPrompt(Prompt):
    user_name: str
    language: str
    task: str

task_prompt = TaskPrompt(
    user_name="Alice",
    language="Python",
    task="write a function to calculate the factorial of a number"
)

The second is a template that is associated with the prompt data object by the class name. This mechanism of finding the template from the prompt data object is crucial to make it convenient. For now I implemented only Jinja2 templates - but it is easy to add different systems. It is early version - try it out and tell me how it works for you - maybe you have ideas about what to add to it - please comment. I use it in my experiments with iterative RAG question answering.