Jev Fan-Out: Ask Many Questions in One Call
Share one state across independent and speculative questions.
On this page
What fan-out isShared state and speculative questionsCost implicationsWhen to split callsPython and TypeScript examplesBudget and failure handlingWhat to measureWhat fan-out is
One request carries state plus a map of questions. Jev evaluates the questions independently against the same context. Your code can inspect all answers and choose which matter. This pattern is useful when network latency is meaningful and the likely branches can be evaluated without depending on one another.
Shared state and speculative questions
A support system can ask about department, urgency, refund requests, and cancellation threats together. It may only use churn information when a ticket enters retention review. That speculative question has an input cost even when unused; measure whether avoiding a later round trip is worth it.
Cost implications
If state is 700 tokens and four questions are 120 tokens each, a rough shared-state estimate is 1,180 tokens. Four separate calls would repeat state. Actual usage depends on provider accounting; compare usage.input_tokens, not just the estimate.
When to split calls
A question cannot consume another answer in the same request. If you need the department before retrieving the correct policy, use two stages. Also split workloads that have different privacy requirements, deadlines, or failure policies.
Python and TypeScript examples
The complete Python support example and JavaScript support example ask three independent questions in one call. Add cancellation detection as a fourth Noul. Always validate the expected answer keys before the decision function.
Budget and failure handling
Both the total request limit and the state-plus-longest-question limit apply. The shared input can grow as you add questions with long criteria. A failed batch needs a retry or review policy; do not treat a missing result as false.
What to measure
Compare end-to-end latency, token usage, failure rates, and decision agreement against separate calls. Vendor cookbook performance numbers describe a specific workload and are not a guarantee for yours.