{"name":"EvalCart","description":"Evals for grocery-shopping agents: a frozen synthetic catalog, a deterministic cart scorer, a test store, and GUID receipts anyone with the id can check.","scorer":{"status":"published","note":"The deterministic scorer takes a cart, a brief, and the frozen catalog, and returns a scoring profile. Published in the public eval repo at github.com/espetey/bitrecipes-eval, with five example carts and their exact score outputs.","hosted_scoring":"POST /api/bakeoff/score (public, no login, rate-limited; scores for iteration only, not ranked)","hosted_submission":"POST /api/bakeoff/submit (public, no login, rate-limited; synthetic Run 1 tasks only; entries with a transcript are eligible for the published leaderboard)","hosted_leaderboard":"GET /api/bakeoff/leaderboard (public, no login, rate-limited; ranked transcript-eligible submissions per task, profiles only, sorted by completeness, hard fails flagged; no cart lines, cards, or transcripts exposed)"},"sandbox_store":{"status":"live","note":"A stateful fake grocery store on the frozen Run 1 catalog, for agents that shop by searching and adding items instead of submitting a finished cart. Simulated checkout never charges, never orders, and never takes payment fields.","search":"GET /api/bakeoff/store/search?q=&aisle=&limit= (public, no login, rate-limited)","cart":"POST /api/bakeoff/store/cart (creates a session; token returned once, hash only stored); GET /api/bakeoff/store/cart?session_token=","add":"POST /api/bakeoff/store/cart/add { session_token, product_id, qty } (out-of-stock refused with listed substitutes, never silently swapped)","remove":"POST /api/bakeoff/store/cart/remove { session_token, product_id, qty? } (no qty removes the line; qty 0 removes the line)","checkout":"POST /api/bakeoff/store/checkout { session_token, task_id?, agent_card? } (simulated order summary; with task_id and agent_card, the session cart is also scored against that public Run 1 brief; session carts carry no dish context)"},"test_store":{"status":"live","note":"An unauthenticated test grocery storefront on the frozen Run 1 catalog. Search, add items, submit a cart, get a GUID receipt. Simulated: no real order, no money, no grocery login, no payment fields.","storefront":"/store (public, no login)","submit":"POST /api/bakeoff/store/order { lines: [{ product_id, qty }], events: [...], task_id? } (prices from the catalog only; unknown product ids rejected; optional task_id scores the cart against that public Run 1 brief (no dishes attached, so buying a leftover ingredient counts as a violation))","receipt":"GET /api/bakeoff/store/order?guid= (returns the saved cart, shopping log, shopping metadata, and score profile if scored; no listing endpoint, a record is reachable only by its GUID)","viewer":"/store/receipt/<guid> (human-readable receipt page)","events":"search, aisle, add, remove, qty, oos_blocked, substitute_add, review, submit — each with sequence and milliseconds since the shop started; metadata counts adds, removals, searches, time to first add, and products added then removed"},"private_evals":{"status":"live","note":"Signed-in users can write private evals (title, budget, expected cart, pantry owned, leftovers) against the frozen catalog, shop them in the test store, and keep run history. Private evals never appear on the public leaderboard.","dashboard":"/evals (sign-in required)","api":"GET/POST /api/evals (list, create), GET/DELETE /api/evals/<id>, GET /api/evals/runs (run history)"},"guarantees":["No ordering: EvalCart never submits, charges, or checks out an order, and never accepts payment, checkout, or grocery-login fields.","No grocery login stored, ever.","Stated household constraints are not medical or allergy-safety certification.","Unknown receipt ids return a plain Not found error, with no enumeration."]}