Skip to content

Budgets & caching

Workload rate limits and budgets, organisation caps and the exact-match response cache, with how each limit behaves when it is reached.

Budgets & limits

Each workload owns rate limits (requests and tokens per minute) and a spend budget shared across its credentials. An organisation-wide aggregate cap applies across all workloads.

Choose a budget period: total, daily or monthly. Budget action block stops over-budget calls, and notify lets them continue.

How limits are enforced

  • Past the workload's rate limit the call returns 429 with a Retry-After header: the whole seconds until the next per-minute window (1 to 60). Past a blocking budget it returns 402. Either way the request never reaches a provider.
  • Before dispatch the gateway checks settled spend and reserves an estimated amount against capped budgets. Actual usage is priced from the live price book and debited when the response is metered.
  • If the counters cannot be reached, a blocking budget is reconciled from the immutable audit ledger and fails closed when it cannot be verified. Rate windows fail open when Redis is unavailable, and notify-mode budgets never block.
  • Workspace Knowledge uploads are metered the same way. The embeddings a document needs are priced like a /v1/embeddings call and charged to the organisation, its prepaid credit and the uploading member's budget. An upload is refused with 402 when the organisation budget or credit is exhausted.
Usage: review consumption and spending.
Usage: review consumption and spending.English interface · illustrative demo data. Open the image for full size.
# past the workload's rate limit: the request never reaches a provider
# response header: Retry-After: 23 (whole seconds until the next per-minute window)
{
  "error": {
    "message": "rate limit exceeded",
    "type": "rate_limit_error",
    "param": null,
    "code": "rate_limit_exceeded"
  }
}

Caching

An opt-in exact response cache matches normalized requests. Enable it in the Console's Cache view.

  • It is strictly isolated per organisation and encrypted at rest.
  • It requires content retention.
Cache: configure exact response caching.
Cache: configure exact response caching.English interface · illustrative demo data. Open the image for full size.

New audit rows record how the call was served: none | exact.

# with retention and exact caching on, a repeat of a cacheable request can come from the cache
curl https://api.sluis.ai/v1/chat/completions \
  -H "Authorization: Bearer $SLUIS_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "model": "mistral/mistral-large-latest", "messages": [{ "role": "user", "content": "Define GDPR" }] }'

# audit row of a cache hit: cache_hit: exact, no provider dispatch, no token cost