Context Chunk Packer

Overview

When you have ranked context chunks and a token budget, we walk them in order and keep each one only if the full assembled rendering (prefix + selected chunks + separator + suffix) still fits after counting with your tokenizer (cl100k_base or o200k_base). Skipped chunks do not block later smaller ones. We return selected and skipped ids plus the packed token count — not the concatenated text. Do not use this to reorder, split, or truncate chunks.

Byte-pair token counts are not additive across joins. We admit a chunk only when the full rendering still fits, so you fill the window safely in priority order.

When to use it

Scope

Non-goals

• Does not reorder or truncate chunks
• Does not sum per-chunk token counts
• Does not model provider message framing

Properties

Capabilities

Cost & billing

Flat rate per invocation

File handling

How we handle your data

Input schema

Output schema

API contract

curl -sS -X POST "$BASE/v1/execute" -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"skillKey":"context.pack_chunks","input":{"chunks":[{"id":"c1","text":"First chunk."},{"id":"c2","text":"Second."}],"windowTokens":8000,"reservedOutputTokens":1000,"separator":"\n\n"}}'

import { Squillify } from "@squillify/sdk";

const sq = new Squillify({ apiKey: process.env.SQUILLIFY_API_KEY });

const { output } = await sq.execute("context.pack_chunks", {"chunks":[{"id":"intro","text":"Hello world"}],"windowTokens":1000});

Errors

Other limits

Limitations

• Does not reorder or truncate chunks
• Does not sum per-chunk token counts
• Does not model provider message framing
• Reordering or summarizing chunks to fit
• Splitting an oversized chunk without truncating it first

How to call

POST /v1/execute (requires authentication)

{
  "skillKey": "context.pack_chunks",
  "input": {
    "chunks": [
      {
        "id": "intro",
        "text": "Hello world"
      }
    ],
    "windowTokens": 1000
  }
}

Examples

{
  "chunks": [
    {
      "id": "intro",
      "text": "Hello world"
    }
  ],
  "windowTokens": 1000
}

{
  "reservedOutputTokens": 0,
  "selectedCount": 1,
  "skill": "context.pack_chunks",
  "totalChunks": 1,
  "availableInputTokens": 1000,
  "skippedIds": [],
  "warnings": [],
  "packedTokenCount": 2,
  "windowTokens": 1000,
  "selectedIds": [
    "intro"
  ],
  "skippedCount": 0,
  "tokenizer_version": "jtokkit-1.1.0",
  "tokenizer": "cl100k_base"
}

{
  "chunks": [
    {
      "id": "big",
      "text": "The quick brown fox jumps"
    },
    {
      "id": "small",
      "text": "hi"
    }
  ],
  "windowTokens": 1
}

{
  "reservedOutputTokens": 0,
  "selectedCount": 1,
  "skill": "context.pack_chunks",
  "totalChunks": 2,
  "availableInputTokens": 1,
  "skippedIds": [
    "big"
  ],
  "warnings": [],
  "packedTokenCount": 1,
  "windowTokens": 1,
  "selectedIds": [
    "small"
  ],
  "skippedCount": 1,
  "tokenizer_version": "jtokkit-1.1.0",
  "tokenizer": "cl100k_base"
}

Recommended use cases

• Pack ranked RAG results into a context window
• Fill a prompt in priority order while reserving output tokens
• Decide which snippets to include before a model call

Avoid for

• Reordering or summarizing chunks to fit
• Splitting an oversized chunk without truncating it first

Related skills

Composable with

Safety & privacy

• Runs without calling any external service.
• Your payloads are never stored permanently.
• Telemetry captures metadata only — never your payload contents.
• We never train models on your payloads.
• No human reviews your payloads.

Tags