Embedding Batch Planner

Overview

Plan the requests, do not send them. You give us the items you intend to embed — each with its text or with a token count you already have — plus your provider's limits, and we hand back which item goes in which request, how many tokens each request carries, and which items are too large for any request at all. Nothing is embedded, no provider is called and no vector comes back; the answer is a grouping of your identifiers. Every item lands in exactly one batch or in the oversized list, so a plan never quietly loses work. Two packing modes are available: fill requests in order, or pack largest-first to need fewer requests at the cost of the original order. Reach for context.pack_chunks instead when you want to select a subset for one context window rather than partition a whole corpus across many requests.

Embedding providers cap both the tokens and the items in one request, and discovering that mid-run means a half-embedded corpus and a retry you cannot resume cleanly. Planning the requests first turns that into an answer you can inspect, store and replay.

When to use it

Scope

Non-goals

• Does not compute, request or store an embedding vector
• Does not call an embedding provider or any other endpoint
• Does not select a subset — every item is planned or reported as oversized
• Does not truncate an oversized item to make it fit
• Does not return the text you sent, only your ids

Properties

Capabilities

Cost & billing

Flat rate per invocation

File handling

How we handle your data

Input schema

Output schema

API contract

curl -sS -X POST "$BASE/v1/execute" -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"skillKey":"knowledge.generate_embeddings_batch_plan","input":{"items":[{"id":"c1","tokenCount":420},{"id":"c2","text":"Second chunk."}],"modelLimits":{"maxTokensPerRequest":8191,"maxItemsPerRequest":2048},"packing":"SEQUENTIAL"}}'

import { Squillify } from "@squillify/sdk";

const sq = new Squillify({ apiKey: process.env.SQUILLIFY_API_KEY });

const { output } = await sq.execute("knowledge.generate_embeddings_batch_plan", {"items":[{"id":"chunk-1","text":"We refund an unused subscription within 30 days."},{"id":"chunk-2","text":"A refund needs the original order id."},{"tokenCount":40,"id":"chunk-3"}],"modelLimits":{"maxTokensPerItem":32,"maxTokensPerRequest":32,"maxItemsPerRequest":2}});

Errors

Other limits

Limitations

• Does not compute, request or store an embedding vector
• Does not call an embedding provider or any other endpoint
• Does not select a subset — every item is planned or reported as oversized
• Does not truncate an oversized item to make it fit
• Does not return the text you sent, only your ids
• Getting embeddings — nothing here calls a model
• Selecting the best chunks for one context window, which context.pack_chunks does
• Planning a corpus you have not sized: an item needs text or a token count

How to call

POST /v1/execute (requires authentication)

{
  "skillKey": "knowledge.generate_embeddings_batch_plan",
  "input": {
    "items": [
      {
        "id": "chunk-1",
        "text": "We refund an unused subscription within 30 days."
      },
      {
        "id": "chunk-2",
        "text": "A refund needs the original order id."
      },
      {
        "tokenCount": 40,
        "id": "chunk-3"
      }
    ],
    "modelLimits": {
      "maxTokensPerItem": 32,
      "maxTokensPerRequest": 32,
      "maxItemsPerRequest": 2
    }
  }
}

Examples

{
  "items": [
    {
      "id": "chunk-1",
      "text": "We refund an unused subscription within 30 days."
    },
    {
      "id": "chunk-2",
      "text": "A refund needs the original order id."
    },
    {
      "tokenCount": 40,
      "id": "chunk-3"
    }
  ],
  "modelLimits": {
    "maxTokensPerItem": 32,
    "maxTokensPerRequest": 32,
    "maxItemsPerRequest": 2
  }
}

{
  "batches": [
    {
      "index": 0,
      "itemIds": [
        "chunk-1",
        "chunk-2"
      ],
      "itemCount": 2,
      "tokenCount": 18
    }
  ],
  "summary": {
    "batchCount": 1,
    "itemCount": 3,
    "totalTokens": 58,
    "maxBatchTokens": 18,
    "packing": "SEQUENTIAL",
    "ordering": "INPUT_ORDER",
    "tokenizer": "cl100k_base",
    "tokenizer_version": "jtokkit-1.1.0"
  },
  "oversized": [
    {
      "id": "chunk-3",
      "tokenCount": 40,
      "reasonCode": "EXCEEDS_MAX_TOKENS_PER_ITEM"
    }
  ]
}

{
  "modelLimits": {
    "maxTokensPerRequest": 100
  },
  "items": [
    {
      "tokenCount": 60,
      "id": "a"
    },
    {
      "tokenCount": 60,
      "id": "b"
    },
    {
      "tokenCount": 40,
      "id": "c"
    },
    {
      "tokenCount": 40,
      "id": "d"
    }
  ],
  "packing": "FIRST_FIT_DECREASING"
}

{
  "batches": [
    {
      "index": 0,
      "itemIds": [
        "a",
        "c"
      ],
      "itemCount": 2,
      "tokenCount": 100
    },
    {
      "index": 1,
      "itemIds": [
        "b",
        "d"
      ],
      "itemCount": 2,
      "tokenCount": 100
    }
  ],
  "summary": {
    "batchCount": 2,
    "itemCount": 4,
    "totalTokens": 200,
    "maxBatchTokens": 100,
    "packing": "FIRST_FIT_DECREASING",
    "ordering": "INPUT_ORDER",
    "tokenizer": "cl100k_base",
    "tokenizer_version": "jtokkit-1.1.0"
  },
  "oversized": []
}

Recommended use cases

• Turn the chunks of a document set into the exact embedding requests to send
• Check whether a corpus fits a provider's request quota before starting a run
• Find the items that need splitting before an embedding job, not during it

Avoid for

• Getting embeddings — nothing here calls a model
• Selecting the best chunks for one context window, which context.pack_chunks does
• Planning a corpus you have not sized: an item needs text or a token count

Related skills

Composable with

Safety & privacy

• Runs without calling any external service.
• Your payloads are never stored permanently.
• Telemetry captures metadata only — never your payload contents.
• We never train models on your payloads.
• No human reviews your payloads.

Tags