Decide per question whether to call one model, a panel, or a person. Routing code in Python, LangGraph and JavaScript, including a support and ops queue.
A deliberation seats several models on one question, and a judge reads their answers, except on the express lane, where one seat answers alone. The call returns one answer, sometimes the best seat's as written and sometimes a synthesis, plus where the seats split. It costs more and takes longer than one model call. An agent that calls it on every turn pays that on questions a single model answers fine.
A router makes the choice per question. It has three outcomes: one model answers, a panel answers, or a person decides.
POST /v1/estimate classifies a prompt and quotes the whole call for it without running it. It is always free: billed_usd is 0 on every successful response, and an error response omits it.
curl https://www.quorumllm.com/v1/estimate \
-H "Authorization: Bearer $QUORUM_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "QRUM:STAN",
"messages": [{"role":"user","content":"Should we migrate off the monolith this quarter?"}]
}'{
"request_id": "0f5c…",
"model": "QRUM:STAN",
"mode": "QRUM:STAN",
"depth": "medium",
"difficulty_score": 0.78,
"task_type": "strategy",
"estimated_price_usd": 0.50,
"billed_usd": 0,
"deliberation_value": {
"verdict": "likely_helps",
"reason": "multi_step_conclusion",
"basis": "hypothesis",
"evidence": "…"
}
}The values in the sample are illustrative. A real response also carries a timing_ms object.
Four fields do the routing.
| Field | Use |
|---|---|
deliberation_value.verdict | Whether a panel is expected to help. Branch on this first. Values: likely_helps, likely_hurts, unknown. |
difficulty_score | 0 to 1. The main dial for your own threshold. |
task_type | One of general, coding, analysis, creative, explanation, strategy, math. Use it to exempt whole categories. |
estimate.total_usd | The quote for the whole call: the engines, the platform fee and the Mode's fee. Compare it to your budget. estimated_price_usd is the Mode's fee alone. |
| Verdict | When | Do |
|---|---|---|
likely_helps | The answer is a conclusion reached by multi-step reasoning: a task_type of math, analysis or strategy, or a difficulty_score of 0.6 or higher. | Escalate. Independent seats catch each other's errors. |
likely_hurts | The answer is a passage to reproduce or lightly edit: proofreading, "output this exact text", fixing typos in a supplied block. | Use one model. Synthesis rewrites by design, and a rewrite damages an exact answer. |
unknown | Neither pattern is present. | Apply your own thresholds. |
The verdict carries basis: "hypothesis". It is routing advice with a stated basis, so treat it as a strong default and keep your own override.
On the API, the Mode you call sets how the question is answered. Your levers are which Mode you call, the optional tier, and quorum.max_cost_usd, a ceiling on one call's bill. The ceiling applies after the fact: the deliberation runs, the answer comes back whole, and the response carries quorum.capped: true if the bill stopped at the ceiling.
So the escalation decision is a branch in your code: for this turn, call a single model or call a panel. The threshold belongs in your codebase and you will move it as you learn.
In Chat, Panel and Studio, the person asking picks a tier (Express, Foundation, Frontier) next to the composer. That control is theirs.
A panel returns an answer and a record of how it got there. When the seats did not settle, the third outcome applies. Read these on the response:
| Field | Meaning |
|---|---|
quorum.converged | false when the panel did not settle on one position. |
quorum.remaining_friction | Usually factual_conflict, when the judge flagged a factual conflict between seats, or null. It can carry a short sentence from the judge naming the disagreement. |
quorum.contested_passage | An object { seat, quote, why }, or null when no single passage carried the split. |
Send the person contested_passage.quote and the receipt (GET /v1/receipts/{request_id}), with the answer. The receipt shows each seat, so they can check whether the disagreement is real.
A repeat of a call that already succeeded returns a replay with quorum.replayed: true. It holds the answer but none of converged, remaining_friction, contested_passage or billed_usd. The Python below checks replayed first and sends a replay to a person, because its split state was not stored.
import os, requests
BASE = "https://www.quorumllm.com/v1"
H = {"Authorization": f"Bearer {os.environ['QUORUM_API_KEY']}"}
MODE = "QRUM:STAN"
def estimate(msgs):
try:
r = requests.post(f"{BASE}/estimate", headers=H, timeout=15,
json={"model": MODE, "messages": msgs})
return r.json() if r.ok else None
except requests.RequestException:
return None
# max_usd and threshold are examples to tune. estimate.total_usd quotes the
# whole call: the engines, the platform fee and the Mode's fee.
def route(msgs, max_usd=1.00, threshold=0.35):
est = estimate(msgs)
if est is None: # classification down: answer with one model
return "single", None
verdict = (est.get("deliberation_value") or {}).get("verdict")
if verdict == "likely_hurts":
return "single", est
if ((est.get("estimate") or {}).get("total_usd") or 0) > max_usd:
return "single", est
if verdict != "likely_helps" and (est.get("difficulty_score") or 0) < threshold:
return "single", est
return "panel", est
def deliberate(msgs):
try:
resp = requests.post(f"{BASE}/chat/completions", headers=H,
timeout=300,
json={"model": MODE, "messages": msgs})
except requests.RequestException as e:
return {"lane": "error", "error": type(e).__name__}
try:
body = resp.json()
except ValueError:
body = {} # a gateway error can return HTML instead of JSON
if not resp.ok or "quorum" not in body:
return {"lane": "error", "error": (body.get("error") or {}).get("code") or resp.status_code}
q = body["quorum"]
replayed = bool(q.get("replayed"))
split = q.get("converged") is False or q.get("remaining_friction") is not None
return {
"lane": "human" if (split or replayed) else "panel",
"answer": body["choices"][0]["message"]["content"],
"contested": q.get("contested_passage"),
"receipt": q["request_id"],
"replayed": replayed,
}
def decide(msgs):
lane, _ = route(msgs)
if lane == "panel":
out = deliberate(msgs)
if out["lane"] != "error":
return out
return {"lane": "single", "answer": call_single_model(msgs)}The sample checks the HTTP status and error.code before it reads choices, because an error body carries no choices. A failed panel call takes the single-model path.
call_single_model is your existing model call. The budget and threshold values are examples. Pick yours from the distribution of difficulty_score on your own traffic, which the free call lets you measure.
The same router as a graph. The human node pauses the run until a person resumes it, so the graph needs a checkpointer.
from typing import TypedDict, Optional
from langgraph.graph import StateGraph, START, END
from langgraph.checkpoint.memory import MemorySaver
from langgraph.types import interrupt, Command
class S(TypedDict):
msgs: list
lane: Optional[str]
result: Optional[dict]
def classify(s: S):
lane, _ = route(s["msgs"])
return {"lane": lane}
def single(s: S):
return {"result": {"lane": "single", "answer": call_single_model(s["msgs"])}}
def panel(s: S):
out = deliberate(s["msgs"])
return {"lane": out["lane"], "result": out}
def human(s: S):
r = s["result"]
ruling = interrupt({"contested": r["contested"], "receipt": r["receipt"],
"answer": r["answer"]})
return {"result": {**r, "ruling": ruling}}
g = StateGraph(S)
g.add_node("classify", classify)
g.add_node("single", single)
g.add_node("panel", panel)
g.add_node("human", human)
g.add_edge(START, "classify")
g.add_conditional_edges("classify", lambda s: s["lane"], {"single": "single", "panel": "panel"})
g.add_conditional_edges("panel", lambda s: s["lane"], {"panel": END, "human": "human", "error": "single"})
g.add_edge("single", END)
g.add_edge("human", END)
app = g.compile(checkpointer=MemorySaver())
cfg = {"configurable": {"thread_id": "ticket-4812"}}
app.invoke({"msgs": [{"role": "user", "content": "…"}]}, cfg)
# when it pauses at the human node:
app.invoke(Command(resume="Approved with the amended terms"), cfg)A support bot or an on-call assistant has the same shape. Many tickets are routine. A few carry a cost if the answer is wrong: a refund above a limit, a policy exception, an account closure, a production change. Route those by your own rule first, then use the panel to decide whether a person is needed.
HIGH_STAKES = ("refund", "chargeback", "close my account", "delete", "outage", "rollback")
def triage(ticket_text: str, amount_usd: float = 0):
msgs = [{"role": "user", "content": ticket_text}]
if amount_usd > 500 or any(w in ticket_text.lower() for w in HIGH_STAKES):
out = deliberate(msgs) # always a panel on these
return ("auto_reply" if out["lane"] == "panel" else "agent_review"), out
lane, _ = route(msgs)
if lane == "panel":
out = deliberate(msgs)
if out["lane"] != "error":
return ("agent_review" if out["lane"] == "human" else "auto_reply"), out
return "auto_reply", {"answer": call_single_model(msgs)}A split goes to the agent_review queue with contested and receipt attached, so the person starts from the point of disagreement. Agreement sends the reply. Routine tickets cost one model call and one free estimate.
The dollar limit and the keyword list are examples. Both are yours to set.
const BASE = 'https://www.quorumllm.com/v1';
const headers = {
Authorization: `Bearer ${process.env.QUORUM_API_KEY}`,
'Content-Type': 'application/json',
};
async function estimate(messages, model = 'QRUM:STAN') {
try {
const r = await fetch(`${BASE}/estimate`, {
method: 'POST', headers, body: JSON.stringify({ model, messages }),
});
return r.ok ? r.json() : null;
} catch { return null; }
}
// maxUsd and threshold are examples to tune.
async function route(messages, { maxUsd = 1.0, threshold = 0.35 } = {}) {
const est = await estimate(messages);
if (!est) return 'single';
const verdict = est.deliberation_value?.verdict;
if (verdict === 'likely_hurts') return 'single';
if ((est.estimate?.total_usd ?? 0) > maxUsd) return 'single';
if (verdict !== 'likely_helps' && est.difficulty_score < threshold) return 'single';
return 'panel';
}Fail open. If the estimate call fails, answer with your single model so your product keeps answering.
Check the verdict first. The cost and difficulty gates ask whether the question is hard enough and cheap enough. likely_hurts is about outcome: a hard, expensive question can still be one where a panel makes the answer worse.
Limits of the classifier. It is a model reading a prompt, and it will sometimes call a question easy that deserved a panel. Where a wrong answer is expensive, escalate on task_type or your own signal instead of on the score alone.
Latency. The estimate is one extra round trip, and its timing_ms field reports the cost. On a chat UI that renders every token as it arrives, decide whether the extra hop is acceptable.
request_id. GET /v1/receipts/{request_id} returns which engine sat in each seat, what the judge scored each one and what the call cost.converged: true with every seat scoring alike, raise the threshold. That traffic was paying for agreement.This guide is also a paper: The escalation router (PDF).
More papers are on the whitepaper shelf.