Features

Embeddings

Embeddings represent text as numerical vectors. Text with similar meaning produces vectors that are close together, making embeddings useful when exact keyword matching is not enough.

Common use cases

  • Semantic search and RAG: retrieve relevant passages for a search query or model prompt.
  • Similarity and recommendations: find related documents, products, or messages.
  • Clustering and deduplication: group related content and identify near duplicates.
  • Classification: compare text with embedded labels or representative examples.

Generating embeddings

Basic request

Pass a single string in input to create one embedding:

$curl https://models.mixlayer.ai/v1/embeddings \
>  -H "Authorization: Bearer $MIXLAYER_API_KEY" \
>  -H "Content-Type: application/json" \
>  -d '{
>    "model": "qwen/qwen3-embedding-8b",
>    "input": "How does semantic search work?",
>    "input_type": "query"
>  }'

Batch inputs

Mixlayer accepts multiple inputs in one request. Pass an array of strings to embed up to 2,048 items together; the response contains one embedding per input in the same indexed order.

Batching usually improves throughput because it lets Mixlayer schedule inputs together and use the GPU more efficiently. Prefer one batched request over many single-input requests when you have multiple items ready to embed.

$curl https://models.mixlayer.ai/v1/embeddings \
>  -H "Authorization: Bearer $MIXLAYER_API_KEY" \
>  -H "Content-Type: application/json" \
>  -d '{
>    "model": "qwen/qwen3-embedding-8b",
>    "input": [
>      "Mixlayer provides OpenAI-compatible inference APIs.",
>      "Embeddings turn text into vectors."
>    ],
>    "input_type": "document"
>  }'

See the Embeddings API reference for all request parameters and response fields.

Available models

input_type tells a model how the embedding will be used. For retrieval, embed stored content with document and incoming searches with query. Omit it when you do not want a retrieval prompt applied.

Qwen3 Embedding 8B

  • Identifier: qwen/qwen3-embedding-8b
  • Context window: 32K tokens
  • Native dimensions: 4,096
  • Supported dimensions: 32–4,096
  • Supported input_type: query, document

query adds Qwen’s retrieval instruction. document does not add a prefix, matching the Qwen model card.

Nemotron 3 Embed 1B

  • Identifier: nvidia/Nemotron-3-Embed-1B-BF16
  • Context window: 32K tokens
  • Native dimensions: 2,048
  • Supported dimensions: 1–2,048
  • Supported input_type: query, document

query applies the model’s query: prompt. document applies its passage: prompt. See the NVIDIA model card.

Nemotron 3 Embed 8B

  • Identifier: nvidia/Nemotron-3-Embed-8B-BF16
  • Context window: 32K tokens
  • Native dimensions: 4,096
  • Supported dimensions: 1–4,096
  • Supported input_type: query, document

query applies the model’s query: prompt. document applies its passage: prompt. See the NVIDIA model card.

Use the same model and compatible input_type values for both sides of a retrieval index. Embeddings from different models or dimensions cannot be compared directly.

Best practices

  • Cache embeddings. Embedding generation is deterministic for the same model, input, and settings. Cache results using the model, input, input_type, dimensions, and normalization setting as the key; regenerate them when the source text or any of those values changes.
  • Normalize before comparing. Set normalize: true or normalize vectors in your application before using cosine similarity or dot product. Apply the same normalization strategy to every vector in an index.
  • Choose a model for your languages. Qwen supports more than 100 languages; Nemotron was evaluated across 34. For multilingual or cross-language retrieval, confirm that both the query and document languages are covered and test quality on representative content.
  • Respect the context window. The current models accept up to 32K tokens per input. Count tokens with the model’s tokenizer and split longer content instead of relying on truncation.
  • Batch when possible. Send multiple ready inputs together to improve throughput and let Mixlayer schedule GPU work more efficiently.
  • Chunk by meaning. Split long documents at natural boundaries such as sections or paragraphs, with a small overlap where context crosses a boundary. Store source metadata with each chunk so retrieved results can be traced back to the original document.