Tools

Using Tools

Tools allow you to extend the capabilities of models by providing them with functions they can choose to call.

LLMs only have knowledge of the world up to the date of their training cutoff, so they can’t know things like the current weather. Tools allow you to provide them with the ability to look up information or perform actions in the real world.

Give a model access to a tool by passing it to the open operation:

 
// define a tool
export const WEATHER_TOOL = {
  name: "get_current_weather",
  description: "Retrieves the current weather for a city",
  fn: async ({ location }: { location: string }) => {
    console.log("*** weather tool called with location: ", location);
    if (location.toLowerCase().includes("san francisco")) {
      return { temperature: 60, units: "F" };
    }
 
    return { error: "I don't know the weather in that city" };
  },
  parameters: {
    location: {
      param_type: "string",
      description:
        'City and State and Country to retrieve the weather for. ',
      required: true,
    },
  },
};
 
// give the seq access to the tool using the tools option
const seq = await socket.open("meta/llama3.1-8b-instruct-free", 
    { tools: [WEATHER_TOOL] });
 
await seq.append("What's the weather in San Francisco?", 
    { role: "user" });
 
const response = await seq.gen({ role: "assistant" }).text();
 
console.log(response);

Tool specification

Tools are defined as objects with the following required properties:

  • name (string) A unique identifier for the tool. This should be descriptive and follow naming conventions (e.g., snake_case or camelCase).

  • description (string) A clear description of what the tool does. This helps the model understand when and how to use the tool.

  • fn (function) The actual function that will be executed when the model calls the tool. This should be an async function that:

    • Takes a single parameter object with the tool’s input parameters
    • Returns a result that the model can understand
    • Handles errors gracefully

  • parameters (object) Defines the input parameters the tool accepts. Each parameter should specify:

    • param_type: The data type (e.g., “string”, “number”, “boolean”)
    • description: What the parameter represents
    • required: Whether the parameter is mandatory (boolean)

Example

const myTool = {
  name: "tool_name",
  description: "What this tool does",
  fn: async (params: { param1: string, param2?: number }) => {
    // Tool implementation
    return { result: "some value" };
  },
  parameters: {
    param1: {
      param_type: "string",
      description: "Description of param1",
      required: true,
    },
    param2: {
      param_type: "number", 
      description: "Description of param2",
      required: false,
    },
  },
};

The model will automatically parse your tool specification and call the appropriate function with the correct parameters when it determines the tool should be used.

BasicsForking