FOR AI LABS · TRAINING DATA

Real-world agent trajectories.

Train better computer-use agents with real interaction data. The Trajectories program turns eligible Boxline sessions into datasets of the steps agents and people take in a browser, a shell and a file system, and how each task turned out. Personal data and secrets are removed before anything is used.

Talk to us about training dataSee what's included
For AI labs and teams building computer-use models, agentic systems, evaluators and multimodal models.
trj_8f2c41 · 7 stepsoutcome: success
TASK
Book the cheapest nonstop JFK → SFO on Oct 14 and save the receipt data to trip.json
  1. browsergoto skyfare.example/flightspage loaded · screenshotOBSERVE
  2. browsertype #to "San Francisco (SFO)"keyboard · 19 keysACTION
  3. browserclick button "Search"x 612 · y 188 · navigationACTION
  4. browserclick "Select" · fare $2741 stop, task asks for nonstopFAIL
  5. browserback · click "Select" · $318recovered: nonstop fareRECOVER
  6. shellexec python parse_receipt.pyexit 0 · wrote trip.jsonACTION
  7. diskread /workspace/trip.jsonbooking BX7Q2L savedOUTCOME
WHAT THE DATA CAN INCLUDE

Every observation, action and outcome, step by step.

Open a field to see what it captures. The record beside it highlights where it lives.

What the page looked like at each step, with its address and the viewport size.

  • screenshot
  • url
  • viewport

trajectory.json
{  "trajectory": "trj_8f2c41",  "task": "Book the cheapest nonstop JFK → SFO",  "environment": { "browser": "chrome", "shell": true },  "steps": [    { "t": 3,      "observation": {        "screenshot": "frames/003.jpg",        "url": "/flights", "viewport": [1280, 720] },      "action": { "type": "click", "x": 612, "y": 188,                  "target": "button[name=Search]" },      "events": ["navigation", "network_idle"] },    { "t": 4, "result": "fail", "reason": "not nonstop" },    { "t": 5, "retry_of": 4, "action": { "type": "goto" } },    { "t": 6, "action": { "type": "exec",                          "cmd": "python parse_receipt.py" },      "exit_code": 0, "files_written": ["trip.json"] }  ],  "outcome": { "status": "success" }}
An illustrative record, simplified. It is not the final delivery format.
PRIVACY BY DESIGN

Only eligible sessions. Personal data and secrets removed.

Every session records whether it is eligible when it starts, and that never changes. Before any session data is used, it goes through these steps; the Privacy Policy has the full list.

  1. 01 · SESSION

    Eligible sessions only

    Started while the program was on, after the project saw the notice. Sessions that use browser profiles are left out.

  2. 02 · SECRETS

    Secrets removed

    Passwords, API keys, tokens, cookies, session identifiers and the contents of browser profiles.

  3. 03 · PERSONAL DATA

    Removed or masked

    Names, contact details, account, card and ID numbers, IP addresses, precise locations and faces, in text, pages, shell output, files and screenshots.

  4. 04 · REVIEW

    Checked before delivery

    Every dataset is reviewed. Data that can't be cleaned with confidence is discarded.

FOR BOXLINE CUSTOMERS

On by default. Off whenever you like.

The program is a setting on your project. The first time you use the console, it explains the program and asks you to keep it on or turn it off, and you can change your mind at any time.

  • A session is eligible only if it started while the program was on, after your project saw the notice.
  • Turn it off in the console's Settings or with one API call. We stop collecting and delete your eligible data that hasn't been delivered yet.
  • Datasets go only to developers of AI models, under written agreements: no re-identifying anyone, the data kept secure, and used only to build and evaluate models.
  • Handling health, financial or children's data, or anything confidential? Turn the program off for that project.

Taking part is unpaid, and you can turn it off at any time. Data already delivered can't be recalled, but it was cleaned first. The full terms are in the Terms of Service.

Turn it off for a projectPUT /v1/project/settings
curl -X PUT https://api.boxline.dev/v1/project/settings \
  -H "x-api-key: $BOXLINE_API_KEY" \
  -H "content-type: application/json" \
  -d '{"trajectories": {"enabled": false}}'
The Trajectories setting in the docs
BUILT FOR MODEL DEVELOPMENT

Use Boxline trajectories for

  • 01

    Computer-use agent training

    What the agent saw, what it did and what happened next, from real tasks.

  • 02

    Supervised fine-tuning

    Plain-English tasks paired with the steps taken to complete them.

  • 03

    Agent evaluation

    Real tasks with their outcome, to measure how agents do.

  • 04

    Outcome and reward models

    Finished and unfinished tasks, to learn what “done” looks like.

  • 05

    Long-horizon task learning

    Multi-step work that crosses the browser, the shell and the disk.

  • 06

    Failure and recovery

    Steps that went wrong, and what the agent or a person did next.

CUSTOM DATASETS

Need a custom dataset?

We can support custom trajectory collection, domain-specific workflows, filtered datasets and enterprise licensing.

Talk to us about AI training data
  • Custom collectionTasks and environments you define
  • Domain workflowsIndustry-specific sites and tools
  • Filtered datasetsBy outcome, surface or task type
  • Enterprise licensingTerms that fit your lab