Token Tango
中文 Request samples

Every task runs.Every trajectory is audited.

We build coding and agent training data for AI labs: reproducible environments, tasks with reference solutions and grading tests, and full trajectories of frontier models solving them. Task types match the public benchmarks. We package them for RL or SFT and deliver in whatever format your pipeline expects.

Request samples See inside a task

Or write to contact@token-tango.com

  • Solvable: a frontier model must pass at least once in 5 independent runs
  • Reproducible: coding tasks are captured and graded offline, in pinned images
  • Coverage: software engineering, science, professional work, finance

  1. Model: Reads instruction.md

    Environment: Repo at base commit, deps installed, network off

  2. Model: grep -rn "next_cursor" src/

    Environment: 11 matches across 6 files

  3. Model: Writes a repro script, runs it

    Environment: Cursor overruns on an empty page: reproduced

  4. Model: Edits the pager, cache and serializer modules

    Environment: Patch applied: 9 files, +412 −87

  5. Model: Runs the existing test suite

    Environment: 1,204 passed

Grading: hidden new tests + existing suite Fail-to-Pass 38/38Pass-to-Pass 1,204/1,204 reward = 1
Illustration: one coding trajectory in an offline grading environment. The model takes a step, the environment answers. Grading tests decide the result: new tests the model never sees, plus the existing suite, which must keep passing. See the full trajectory

This illustrative task lives in a repository of 323 files. The reference solution touches 9 of them.

What's inside a task

Coding tasks are built on open-source repositories whose licenses permit this use. Each has newly written requirements, a reference solution and grading tests, and runs end to end in its own environment, from requirement to graded result.

Environment
A Docker image of the target repository. Dependencies preinstalled, runs fully offline, and pinned by digest so every rerun gets the exact same environment. One environment can yield 10 to 20 tasks.
Task
Instructions, a reference solution, and grading tests: Fail-to-Pass tests that only pass once the fix is in, and Pass-to-Pass tests that must keep passing. Requirements in the instructions and changes in the reference solution match one to one, and a frontier model must pass at least once in 5 independent runs for the task to count as solvable.
Trajectory
The full solving process: every tool call and every environment response. Each one is checked for cheating, and for wasted steps caused by a broken environment.

What to buy for RL vs. SFT

Reinforcement learning
Environment + taskCoding tasks: grading tests return a 0 or 1 reward. Science, professional and finance tasks: a model grades against a per-task rubric and returns a score between 0 and 1. Either plugs into your RL loop.
Supervised fine-tuning
Environment + task + trajectoryWhether a trajectory includes reasoning, and whether it is raw or summarized, depends on the teacher model and is stated in each batch's data notes. Agent scaffold, language, difficulty and patch size can be specified.

The delivery folder for one coding task

  • <task_id>/
  • ├── task.tomlResources, time limit, network policy, image, base commit
  • ├── instruction.mdInstructions
  • ├── environment/DockerfileSolving environment
  • ├── solution/Reference patch
  • ├── tests/Grading tests and grader, hidden from the solving model
  • ├── quality/validation.jsonQuality checks on the task itself
  • └── trajectory/trajectory.jsonSolving trajectory
Harbor task layout by default (an open-source framework for running and grading agent tasks). Trajectories are also available as ATIF v1.7, or as JSONL with task, trajectory and grading in one line. Instructions can follow DeepSWE or SWE-bench Pro style. Other formats are agreed during the pilot.

The model takes a step, the environment answers. The 38 new tests that decide the grade are mounted only at grading time.

Watch a full trajectory

The 20 steps of the illustrative solve, including a wrong turn the model took and reverted, and what the trajectory audit concluded.

Illustrative sample pagerkit-empty-page-0412 fictional repository

20 steps. Press play, or use ← → to go step by step. Read-only steps can be folded.

  1. Read: Reads the instructions: three requirements cat instruction.md
  2. Search: 11 matches across 6 files grep -rn "next_cursor" src/
  3. Read: Reads the paginator sed -n 100,160p src/pagerkit/core/pager.py
  4. Reproduced Run: Reproduced: the cursor overruns on an empty page python repro_empty_page.py ✗ fail
  5. Read: Reads the Page data structure sed -n 30,90p src/pagerkit/core/page.py
  6. Edit: Edits the paginator: empty pages return None apply_patch src/pagerkit/core/pager.py
  7. Run: Repro passes; 212 core tests pass python repro_empty_page.py && pytest tests/unit/core -q ✓ pass
  8. First fail Run: Second issue: the cursor changes after a redis round trip python repro_roundtrip.py --backend memory,redis ✗ fail
  9. Detour Edit: Edits the memory cache backend apply_patch src/pagerkit/cache/memory.py
  10. Run: Still fails python repro_roundtrip.py --backend memory,redis ✗ failDetour: wrong layer. The bug is in how cache keys are encoded, not in the memory backend. Audit verdict: a reasoning mistake by the model, not caused by a broken environment.
  11. Edit: Reverts the previous edit git checkout -- src/pagerkit/cache/memory.py
  12. Read: Reads the cache keys: json.dumps without fixed separators sed -n 1,80p src/pagerkit/cache/keys.py
  13. Edit: Edits cache keys and the cursor codec: canonicalize before encoding apply_patch src/pagerkit/cache/keys.py src/pagerkit/cursors/codec.py
  14. Run: Both backends pass python repro_roundtrip.py --backend memory,redis ✓ pass
  15. Edit: Edits the serializers: empty pages round-trip too apply_patch src/pagerkit/serializers/json.py src/pagerkit/serializers/base.py
  16. Edit: Wires the canonical cursor into windows and Page apply_patch src/pagerkit/cursors/opaque.py src/pagerkit/core/window.py src/pagerkit/core/page.py
  17. Edit: Updates the changelog apply_patch CHANGELOG.md
  18. All pass Run: All 1,204 existing tests pass pytest -q ✓ pass
  19. Run: Removes the repro scripts; the patch touches 9 files rm repro_empty_page.py repro_roundtrip.py && git status --short
  20. Submit: Submits submit
  21. Grading: hidden new tests + existing suite Fail-to-Pass 38/38 ✓Pass-to-Pass 1,204/1,204 ✓ reward = 1
Patch
+0 −0
Steps
0 / 20
Repro
not yetreproduced
Hidden tests
mounted at grading✓ 38/38
Wall time
23m41s

Task types aligned with public benchmarks

Task types follow the public benchmarks labs already use, so you can grade with the same harness. The tasks themselves are new; the last column shows how our samples differ structurally from the originals.

AreaAligned task typesCompared with the original
Software engineering Repository-level tasks in the style of DeepSWE and SWE-bench Pro; terminal tasks in the style of Terminal-Bench

Reference solutions for repository-level tasks span many files, with changes substantially larger on average than the official DeepSWE tasks. Mostly Go, plus Java, C++, TypeScript, Python, Rust and JavaScript.
Science End-to-end research tasks in the style of ResearchClawBench: data audit, method implementation, figures, written report

The model gets only the unprocessed raw data; the authors' processed results, ground-truth labels and original figures are hidden from it. Every grader is calibrated: the reference solution must score 1, and counterexamples with a single injected flaw must score 0.
Professional work APEX-Agents style: full investment banking, consulting and legal projects with their document sets; GDPval style: finished deliverables across occupations

In a 20-task GDPval-style sample, the solver produced 2.5 deliverable files per task on average, against 1.6 on matched originals. Nearly every APEX-style task adds its own attachments on top of the shared documents, and more of them ask for a complete memo or deck.
Finance Finance Agent Benchmark style: research answered from primary SEC filings, with calculations and verifiable citations; offline RL environments for financial statement audits

Our 5 sample tasks average 12.2 rubric items (official public questions: 8.9), and sample trajectories average 28.6 tool calls (official: 16.6). Every company in the official public questions is excluded. Audit environments run fully offline.

Benchmark names refer to the public benchmarks our task types align with. They do not imply any partnership with their authors.

Eight checks every coding task passes before it ships

Problems get more expensive the later they are found, so we check as far upstream as we can. Before delivery, rules scan everything and anything suspicious goes to model review.

  1. Sourcing

    Open-source repositories and real PRs whose licenses permit this use.

  2. Environment build

    Dependencies preinstalled, runs offline, image pinned.

    Adds task.toml environment/Dockerfile

  3. Two-way check

    The reference solution must pass; the untouched code must fail.

    Adds solution/ tests/

    Tasks whose reference solution fails, or whose untouched code already passes, are dropped here.

  4. Test calibration

    Split tests that pass only after the fix from tests that must always pass; new tests stay hidden from the solving model.

  5. Instructions

    Requirements in the instructions and changes in the reference solution match one to one, nothing more, nothing less.

    Adds instruction.md

  6. Trajectory capture

    Frontier models solve offline in independent runs, every step recorded. A task counts as solvable only with at least 1 pass in 5.

    Adds trajectory/trajectory.json

    Tasks a frontier model fails in all 5 independent runs are dropped here.

  7. Review

    Rules scan the whole batch first. Anything suspicious goes to model review, which must point to the exact step and output in the trajectory; flags without that evidence are discarded.

    Adds quality/validation.json

    Trajectories caught cheating in review are dropped here.

  8. Delivery

    Shipped in batches, with a validation record for every task.

From samples to steady supply

Small batches first. You validate each stage with your own harness before committing to the next.

  1. Samples and data notes

    A sample batch with data notes: task statistics, difficulty distribution, grading method.

  2. Pilot batch

    A small batch in your format and to your acceptance criteria, checked with your harness.

  3. Batched delivery

    Batches on an agreed schedule, with a validation record for every task. Every issue you report becomes an automated check that later batches must pass.

  4. Steady supply

    Scheduled against each task type's daily capacity. Task type, language, difficulty and teacher model can change between batches.

Capacity is per task type: 250 DeepSWE-style tasks a day, 300 SWE-bench Pro-style tasks a day. Priced per task, with volume and schedule in the contract.

Send a brief, get samples and a schedule back

Tell us the domain and task type, volume, timeline, RL or SFT, and delivery format, and we'll prepare samples and a schedule around them.

Email us contact@token-tango.com

Just want to look first? Request samples