Quickstart

This guide will help you get started with Mixlayer and ModelSocket’s Typescript SDK (our protocol for statefully interacting with LLMs). If you prefer not to use ModelSocket, you can use Mixlayer’s OpenAI-compatible REST API instead.

To complete this quickstart, you’ll need to have a basic understanding of Javascript and know how to run commands in your terminal.

Create an account

If you haven’t already, create and sign-in to a Mixlayer account. You can sign up at console.mixlayer.com/sign-up.

Get an API Key

An API key will allow you to make requests to Mixlayer’s models. Once you have a key, set it to an environment variable so the library can access it.

env.sh

export MODELSOCKET_API_KEY=sk_********

Setup your project

Create a new directory for your project and initialize it with npm and the ModelSocket SDK.

init.sh

mkdir mixlayer-quickstart
cd mixlayer-quickstart
npm init -y
npm install modelsocket

Create a script

Create a new file called script.mjs.

Be sure to use the .mjs extension or else the script won’t be able to import other modules.

Use ModelSocket.connect to open a connection. Then use open, append and gen crate sequences and interact with them.

script.mjs

import { ModelSocket } from "modelsocket";
 
const socket = await ModelSocket.connect("wss://models.mixlayer.ai/ws");
 
try {
  // allocate sequence on a gpu for llama 8b
  const seq = await socket.open("meta/llama3.1-8b-instruct-free");
 
  // append a prompt to its context window
  await seq.append("Explain quantum computing.\n", { role: "user" });
 
  // generate a response
  const stream = seq.gen({ role: "assistant" }).textStream();
 
  // print the response as it generates
  for await (const chunk of stream) {
    process.stdout.write(chunk);
  }
} catch (error) {
  console.error(error);
} finally {
  // be sure to close the socket to release any resources
  socket.close();
}

Run your script

Run your script with node script.mjs.

run.sh

node script.mjs

You should see the output stream to your terminal:

> node script.mjs
**Introduction to Quantum Computing**

Quantum computing is a revolutionary technology that uses the principles of quantum mechanics to perform calculations and operations on data. Unlike classical computers, which use bits to represent information as 0s and 1s, quantum computers use quantum bits or...

Congratulations! You’ve just made your first request to Mixlayer.

Next Steps

Appending a system prompt at the beginning to customize the output and personality of the model to your liking
Deploy your code behind an API so you can use it in your own apps
Give the model access to tools to exhance its capabilities
Integrate additional models like a guard model to protect against unauthorized uses

Introduction Basics