Ragged

What is this?

Ragged is a 0-dependency, lightweight, universal LLM client for JavaScript and Typescript. It makes it easy to access LLMs via a simple, easy to understand, and uncomplicated API.

The heart of Ragged is a simple abstraction that allows you to interact with LLMs in a consistent way, regardless of the provider or model you are using. This abstraction makes it easy to switch between different LLMs without having to change your code.

Ragged is not a framework.

Ragged's job is to be a low-level connector library. it is, explicitly, not a framework; but it is meant to be easy to use. You can build your own framework on top of Ragged, or use it as a standalone library in your existing projects.

Installation

Installing Ragged is very easy.

# either
npm install --save-exact ragged
# or
pnpm add --save-exact ragged
# or
yarn add --exact ragged

That's it. You're ready to go!

Ragged's Chat Completion Abstraction

Ragged's core Chat Completion abstraction is an easy-to-use Message interface.

import type { Message } from "ragged";

const history: Message[] = [
    { type: "user", text: "What is a rickroll?" },
    { type: "bot", text: "A rickroll is a prank..." }
]

Because this interface is standard, a lot of operations become very easy to perform. For example, you can access the last message in the history using the .at method.

console.log(history.at(-1)?.text); // A rickroll is a prank...

Or, you can simply modify history by pushing new messages to the array.

history.push({ type: "bot", text: "I'm a bot!" });

About 90% of Ragged is built around this simple interface. (The other 10% is for Embeddings, which has its own analog to the Message type). This standard interface is the same across all LLM providers, making it easy to switch between providers without changing your code.

In the following sections, we will show you how to use Ragged to perform many complex operations with ease, including chat completion, tool calling, multimodal input, agent creation, and more.

Simple chat

Ragged is very easy to use. Here is a complete application that shows chat completion.

import { Chat } from "ragged"

// create a new Chat instance with the OpenAI provider
const c = Chat.with({
    provider: 'openai',
    config: { apiKey: process.env.OPENAI_API_KEY }
});

// chat with the model
const {history} = await c.chat('What is a rickroll?');

// {history}.at(-1) is a native JS array method for the last element
console.log(history.at(-1)?.text); // A rickroll is a prank...

Nothing to it!

Message History

Tip

You can see an example of how to use the history functionality in the examples folder. Click here to see the example. This is a fully working example, so if you set up Ragged locally, you can execute the example using pnpm run:example history.ts.

By default, each instance of the Chat object records the history of the conversation. You can access the history of the conversation using the .history property.

console.log(c.history);

This array gets updated with each call to the chat method.

You can also set the history of the conversation by setting the .history property to an array of messages. This way, you can control the history of the conversation.

const history = c.history;

Accessing message history

You can access the history using the .history property.

console.log(c.history); // [ { text: 'What is a rickroll?' ... ]

You can also access the last message in the history using the .at method.

console.log(c.history.at(-1)?.text); // A rickroll is a prank...

Tip

The .at method is a native JavaScript array method that allows you to access elements in an array using negative indices. This is useful for accessing the last element in an array. The .at() method has been available in JavaScript since ES2022. See MDN documentation.

Setting message history

You can set the history by setting the .history property to an array of messages.

c.history = [
    { type: "user", text: "What is a rickroll?" },
    { type: "bot", text: "A rickroll is a prank..." }
];

You can clear the history by setting the .history property to an empty array.

c.history = [];

Warning

Never modify elements inside the history object directly. Always set the .history property to a new array. This prevents unexpected behavior and makes the code more predictable.

Freezing History

Tip

You can see an example of how to use the freezing functionality in the examples folder. Click here to see the example. This is a fully working example, so if you set up Ragged locally, you can execute the example using pnpm run:example frozen.ts.

You can turn recording on and off by passing a boolean to the .record method. To turn recording off, pass false. We call this "freezing" the conversation. When the conversation is frozen, the history will not be updated with each call.

// Recording is on by default. Here is how you can turn it off.
c.record(false);

This is useful when you want to create multiple responses to a single prompt. Then, you can prompt the model multiple times. Each time, the model will respond as if it were the first time, and the history will not be updated with each call.

// Turn off history
c.record(false);

// Chat with the model
const response1 = await c.chat('Remember that my name is "John."'); 
// Response: Okay! I will remember that your name is "John."

const response2 = await c.chat('What is my name?');
// Response: I do not know your name. Please tell me.

Image Input, a.k.a. Multimodal

Ragged supports multimodal input. This means that you can pass images to the LLM along with text. This is useful for creating more interactive and engaging chat experiences.

Currently we only support base64 encoded images, but will expand this support in the future.

// chat with the model
const { history } = await c.chat([
    {
        type: "user",
        text: "What do these images contain? Describe them.",
        attachments: [
            {
                type: "image",
                payload: {
                    data: "iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAApgAAAKYB3X3/OAAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAAANCSURBVEiJtZZPbBtFFMZ/M7ubXdtdb1xSFyeilBapySVU8h8OoFaooFSqiihIVIpQBKci6KEg9Q6H9kovIHoCIVQJJCKE1ENFjnAgcaSGC6rEnxBwA04Tx43t2FnvDAfjkNibxgHxnWb2e/u992bee7tCa00YFsffekFY+nUzFtjW0LrvjRXrCDIAaPLlW0nHL0SsZtVoaF98mLrx3pdhOqLtYPHChahZcYYO7KvPFxvRl5XPp1sN3adWiD1ZAqD6XYK1b/dvE5IWryTt2udLFedwc1+9kLp+vbbpoDh+6TklxBeAi9TL0taeWpdmZzQDry0AcO+jQ12RyohqqoYoo8RDwJrU+qXkjWtfi8Xxt58BdQuwQs9qC/afLwCw8tnQbqYAPsgxE1S6F3EAIXux2oQFKm0ihMsOF71dHYx+f3NND68ghCu1YIoePPQN1pGRABkJ6Bus96CutRZMydTl+TvuiRW1m3n0eDl0vRPcEysqdXn+jsQPsrHMquGeXEaY4Yk4wxWcY5V/9scqOMOVUFthatyTy8QyqwZ+kDURKoMWxNKr2EeqVKcTNOajqKoBgOE28U4tdQl5p5bwCw7BWquaZSzAPlwjlithJtp3pTImSqQRrb2Z8PHGigD4RZuNX6JYj6wj7O4TFLbCO/Mn/m8R+h6rYSUb3ekokRY6f/YukArN979jcW+V/S8g0eT/N3VN3kTqWbQ428m9/8k0P/1aIhF36PccEl6EhOcAUCrXKZXXWS3XKd2vc/TRBG9O5ELC17MmWubD2nKhUKZa26Ba2+D3P+4/MNCFwg59oWVeYhkzgN/JDR8deKBoD7Y+ljEjGZ0sosXVTvbc6RHirr2reNy1OXd6pJsQ+gqjk8VWFYmHrwBzW/n+uMPFiRwHB2I7ih8ciHFxIkd/3Omk5tCDV1t+2nNu5sxxpDFNx+huNhVT3/zMDz8usXC3ddaHBj1GHj/As08fwTS7Kt1HBTmyN29vdwAw+/wbwLVOJ3uAD1wi/dUH7Qei66PfyuRj4Ik9is+hglfbkbfR3cnZm7chlUWLdwmprtCohX4HUtlOcQjLYCu+fzGJH2QRKvP3UNz8bWk1qMxjGTOMThZ3kvgLI5AzFfo379UAAAAASUVORK5CYII=",
                    encoding: "base64_data_url",
                    mimeType: "image/png",
                },
            },
            {
                type: "image",
                payload: {
                    encoding: "base64_data_url",
                    data: "iVBORw0KGgoAAAANSUhEUgAAAQAAAAEACAIAAADTED8xAAADMElEQVR4nOzVwQnAIBQFQYXff81RUkQCOyDj1YOPnbXWPmeTRef+/3O/OyBjzh3CD95BfqICMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMK0CMO0TAAD//2Anhf4QtqobAAAAAElFTkSuQmCC",
                    mimeType: "image/png",
                },
            },
        ],
    },
], {
    model: "gpt-4o"
});

// log the messages
console.log(history.at(-1)?.text);

// Output:
// The first image is an emoji of a face with heart-shaped eyes, typically used to express love, adoration, or strong liking for something or someone.
// The second image is a gradient background that transitions from dark to light colors. The colors include black, brown, orange at the top, and transition through white and blue towards the bottom.

Tool Calling

Tip

You can see 3 examples of how to use the tool calling functionality in the examples folder. The Simple Example shows a very simple tool calling example that returns some mock data (although the LLM thinks it's coming from an actual website). The Fetch BBC News RSS Feed Example is a simple, single-tool example that shows how to give the LLM the ability to fetch the latest news from the BBC RSS feed. The List Files example is a more complex example that shows how to use multiple tools in order to let the LLM read your local file system. These are fully working examples, so if you set up Ragged locally, you can execute the example using pnpm run:example tools.ts.

Ragged allows you to further extend its functionality using tools. This gives you the power to integrate custom behavior or commands directly into your chat-based application.

Tools are very powerful! You can use tools to fetch data from external APIs, perform calculations, generate dynamic content, and much more. Tools can be used to extend the capabilities of the LLM and create more interactive and dynamic chat experiences.

To define a tool, first we import the Tool type from Ragged and then define a tool object.

import type { Tool } from "ragged";

const getHomepageTool: Tool = {
    id: "get-homepage-contents",
    description: "Gets the contents of my homepage.",
    handler: async () => {
        return Promise.resolve("Hello! My name is John! I'm a student at a community college!")
    }
}

In this example, we define a tool called getHomepageTool. This tool has the following properties:

id: A unique identifier for the tool. This identifier is how the LLM references the tool in the chat. The LLM will be aware of the tool's existence and will be able to use it in the chat.
description: A brief description of what the tool does. This description is used to help the LLM understand the purpose of the tool.
handler is a function that processes the input and returns the output. It handles the logic of the tool and manages errors gracefully.

Once you have defined a tool, you can use it in a chat interaction by passing it to the chat method.

const { history } = await c.chat("Get the contents of my homepage.", {
    // Pass the tool to the chat method.
    tools: [getHomepageTool],
    model: "gpt-3.5-turbo"
});

console.log(history.at(-1)?.text);
// RESPONSE: I retrieved the contents of your homepage. It says: "Hello! My name is John! I'm a student at a community college!"

Putting it all together, here's what it looks like:

import { Chat } from "ragged"
import type { Tool } from "ragged";

const getHomepageTool: Tool = {
    id: "get-homepage-contents",
    description: "Gets the contents of my homepage.",
    props: {
        type: "object",
        description: "The properties of the tool.",
        props: {}
    },
    handler: async () => {
        return Promise.resolve("Hello! My name is John! I'm a student at a community college!")
    }
}

const c = Chat.with({
    provider: 'openai',
    config: {
        apiKey: process.env.OPENAI_API_KEY 
        // You can also pass the organization ID here.
        organizationId: process.env.OPENAI_ORGANIZATION_ID
    }
});

const { history } = await c.chat("Get the contents of my homepage.", {
    // Pass the tool to the chat method.
    tools: [getHomepageTool],
    model: "gpt-3.5-turbo"
});

console.log(history.at(-1)?.text);

// RESPONSE: I retrieved the contents of your homepage. It says: "Hello! My name is John! I'm a student at a community college!"

Tool Props

The Tool object can also take an optional props object. This object allows the LLM to pass pass additional information to the tool handler. This can be useful if you want the LLM to pass configuration options, data, or other information to the tool handler.

Here is an example of how to use the props object:

import type { Tool } from "ragged";

const fetchTool: Tool = {
    id: "fetch",
    description: "Do a simple GET call and retrieve the contents of a URL.",
    // The props object describes the expected input for the tool.
    props: {
        type: "object",
        props: {
            url: {
                type: "string",
                description: "The URL to fetch.",
                required: true
            }
        }
    },
    // The handler function processes the input and returns the output.
    handler: async (props: string) => {
        // Props are passed to the handler function as a string.
        // This string needs to be parsed into an object before it can be used.
        // Several examples can be seen in the `/examples` folder.
    }
}

Here, we define a tool called fetchTool. This tool has a props object that describes the expected input for the tool. The props object contains a url property that is required and must be a string. The handler function processes the input and returns the output.

Warning

It's important to note that the LLM can hallucinate the props object. This means that the LLM can pass props which are not defined in the props object. This is dangerous, as it can lead to unexpected behavior. To prevent this, you should always validate the props object before using it in your tool handler.

Raw request and response objects

You can get the raw request and response objects from the tool handler by using the raw property. This property contains the raw vanillla Request and Response objects that were sent to and received from the tool handler.

const { history, raw } = await c.chat("What is a rickroll?");
console.log(history.at(-1)?.text); // A rickroll is a prank...
console.log(raw?.requests); // All API requests done during the chat (there can be more than 1 if you are doing tool calling)
console.log(raw?.responses); // All API responses done during the chat (there can be more than 1 if you are doing tool calling)

Autonomous Agents

Tip

You can see 2 examples of how to use the agent functionality in the examples folder. The Simple Example shows a very simple agent that increments a number automatically. The Multiple Agents Example uses multiple agents to generate tweets. These are fully working examples, so if you set up Ragged locally, you can execute the example using pnpm run:example agents-simple.ts or pnpm run:example agents-multiple.ts.

What is an agent?

Many LLM frameworks support autonomous agents. These are tools that can be used to perform tasks without user input. They can be used to automate repetitive tasks, provide real-time information, or interact with external services.

How agents work in Ragged

In many LLM frameworks, agents are complex and require a lot of setup. But in Ragged, agents are very simple to implement using normal, easy-to-understand code. Agents are just pieces of code that take input and return output in a recursive or repetive way.

The simplest agent has the following main components:

A starting state
A loop which calls an LLM to mutate the state recursively
A stop condition which determines when the agent should stop

Using these components, you can create agents that perform a wide variety of tasks.

Incrementing Agent Example

Here is an example of a simple agent that generates a conversation with an LLM. You can also access it here: examples/nodejs/agents-simple.ts. If you run this code, you will see the agent incrementing a number automatically.

/**
 * This example demonstrates how to build a simple agent that increments a number automatically.
 * This is a very simple example, but the principles can be applied to more complex agents.
 * 
 * EXPECTED OUTPUT: 
 * The current number is 1.
 * The current number is 2.
 * The current number is 3.
 * The current number is 4.
 * The current number is 5.
 * The current number is 6.
 * The current number is 7.
 * The current number is 8.
 * The current number is 9.
 * The current number is 10.
 * The agent has reached the stop condition.
 */

import { config } from 'dotenv';
config();
import { Chat } from "ragged"

// Define the main function
async function main() {
    const c = Chat.with({
        provider: 'openai',
        config: { apiKey: process.env.OPENAI_API_KEY }
    });
    c.record(false);

    // Start with the initial state
    let currentState = `The current number is 1.`;

    // Define the stop condition
    const stopCondition = () => currentState.includes("10");

    // Start iterating
    console.log(currentState);
    while (!stopCondition()) {
        currentState = await getNextNumber(c, currentState);
        console.log(currentState);
    }

    // Print the stop condition
    console.log("The agent has reached the stop condition.");
}

// Define the agent function

async function getNextNumber(c: Chat, input: string): Promise<string> {
    // Call the LLM with the input
    const { history } = await c.chat([
        {
            type: "system",
            text: `
                The user will state that "The current number is X". Output "The current number is X+1". Examples:

                If the user input is empty, malformed, or not a number, return "The current number is 1."

                EXAMPLES:

                User: The current number is 1.
                AI: The current number is 2.

                User: The current number is 2.
                AI: The current number is 3.

                User: The current number is 3.
                AI: The current number is 4.

                // If the user input is malformed
                User: The current number is 
                AI: The current number is 1.

                // If the user input is empty
                User: 
                AI: The current number is 1.

                // If the user input is not a number
                User: The current number is yellow.
                AI: The current number is 1.


            `
        },
        {
            type: "user",
            text: input
        }
    ]);

    // Get the last message from the response
    const lastMessage = history.at(-1)?.text;
    return lastMessage || "";
}

// run the code
await main();

Multiple Agents Example

Agents can get very complex, with multiple agents running at the same time. Here is an example of a simple chat application that uses multiple agents to generate a conversation: examples/nodejs/agents-multiple.ts.

Logging

Ragged has a built-in logging system that allows you to log messages to the console. This is useful for debugging and troubleshooting your code. You can use the Logger class to control the logging level and format of the log messages.

import { Logger } from "ragged";
Logger.setLogLevel('debug'); // make it verbose
Logger.setLogLevel('info'); // default
Logger.setLogLevel('warn'); // only log warnings
Logger.setLogLevel('error'); // only log errors
Logger.setLogLevel('none'); // turn off logging altogether

Hooks

Ragged provides a powerful and flexible hook system that allows you to customize and extend the behavior of the API client. Hooks are functions that are executed at specific points during the request/response lifecycle, allowing you to modify requests, handle responses, and perform additional actions based on the API interactions.

Types of Hooks

There are three types of hooks in Ragged:

BeforeRequestHook
AfterResponseHook
AfterResponseParsedHook

Hooks can be asynchronous

Hooks can be asynchronous, allowing you to perform asynchronous operations such as making additional API calls, reading/writing files, or interacting with databases. You can use async functions as your hooks to handle asynchronous operations.

Hook Contexts

Each hook type receives a specific context object that contains relevant information about the request or response. The context objects share a base structure and have additional properties specific to the hook type.

BaseHookContext

The BaseHookContext contains common properties available to all hooks. Each hook also may have additional properties specific to its type.

type BaseHookContext = {
    apiClient: ApiClient;
    requestParams: {
        method: string;
        url: string;
        headers?: RequestInit["headers"];
        body?: any;
    }
}

Hook Types

BeforeSerializeHook

The BeforeSerializeHook is executed before the request is serialized. This allows you to modify the request parameters before they are sent to the API. Its context includes the requestParams object.

Note that after the BeforeSerializeHook is executed, the requestParams are frozen and cannot be modified in subsequent hooks.

Example Usage:

c.chat(`Hello, World!`, {
    hooks: {
        beforeSerialize: ({ requestParams }) => {
            // You could log the request parameters before they are serialized
            console.log("Before serialize:", requestParams);
            
            // You could also modify the request parameters
            requestParams.url = "http://modified.com";
            requestParams.body.some = "modified-field";
            if (requestParams.method === "PUT" || requestParams.method === "PATCH") {
                requestParams.method = "POST";
            }
        }
    }
})

BeforeRequestHook

The BeforeRequestHook is executed before the request is sent. This allows you to modify the request or perform actions before the request is made. Its context includes the request object.

Example Usage:

c.chat(`Hello, World!`, {
    hooks: {
        beforeRequest: ({ request }) => {
            // You could log the request before it is sent
            console.log("Before request:", request);
            // You could also modify the request headers
            request.headers.append('X-Custom-Header', 'custom-value');
        }
    }
})

AfterResponseHook

The AfterResponseHook is executed after the response is received but before it is parsed. This allows you to handle raw responses or perform actions based on the response status. Its context includes both the request and response objects.

Be careful not to use the .json() or .text() methods on the response object in this hook! This will cause the response to be consumed and will prevent it from being parsed later, which will cause an error to be thrown by the chat method.

Example Usage:

c.chat(`Hello, World!`, {
    hooks: {
        afterResponse: ({ response }) => {
            // You could log the response JSON
            console.log("Logging the raw response:", response);
        }
    }
});

AfterResponseParsedHook

The AfterResponseParsedHook is executed after the response is parsed into JSON. This allows you to handle the parsed response or perform actions based on the response data.

Example Usage:

c.chat(`Hello, World!`, {
    hooks: {
        afterResponseParsed: async ({ json }) => {
            // You could log the response JSON
            console.log("JSON response:", json);
            // You could also modify it
            json.data = "Hello, World!";
        }
    }
});

Hooks can be async

All hooks can be asynchronous, allowing you to perform asynchronous operations such as making additional API calls, reading/writing files, or interacting with databases. You can use async functions as your hooks to handle asynchronous operations.

c.chat(`Hello, World!`, {
    hooks: {
        // notice the async keyword... hooks can be async!
        afterResponseParsed: async ({ json }) => {
            await sendToSlack(json);
        }
    }
});

Example with hooks

import { config } from "dotenv";
config();
import { Chat } from "ragged";

const c = Chat.with({
    provider: 'openai',
    config: {
        apiKey: process.env.OPENAI_API_KEY
    }
});

c.chat(`say hello world`, {
    hooks: {
        beforeSerialize: ({ requestParams }) => {
            // You could log the request parameters before they are serialized
            console.log("Before serialize:", requestParams);
            
            // You could also modify the request parameters
            requestParams.url = "http://modified.com";
            requestParams.body.some = "modified-field";
            if (requestParams.method === "PUT" || requestParams.method === "PATCH") {
                requestParams.method = "POST";
            }
        }
        beforeRequest: ({ request }) => {
            // Print the Content-Type header value, just to test the hook
            console.log("We will be sending the Content-Type header with value: ",
                request.headers.get('Content-Type'));
        },
        afterResponse: ({ response }) => {
            // Get the rate limit info from the response headers. This is very useful!
            console.log("Received rate limit info from OpenAI: ",
                Array.from(response.headers.entries())
                    .filter(([key]) => key.startsWith('x-ratelimit')));
        },
        afterResponseParsed: (context) => {
            // Finally, print the raw JSON response from OpenAI
            console.log("Raw OpenAI response JSON: ",
                context.json);
        }
    }
});

Official LLM Adapters

Ragged supports multiple LLM providers out of the box. You can use these providers to interact with the LLMs and generate responses to your prompts.