Robot-team eval

Robot-team test interface

Submit a policy endpoint, container, action trace, skill trace, teleop demo, or sim controller as artifact references against one capture-backed site package.

Open submission Request review

Proof boundary

Blueprint can evaluate submitted references against real-site packages when source evidence exists. A submission is advisory: it does not prove deployment readiness, safety validation, real robot execution, simulator run completion, rights clearance, or guaranteed thresholds.

Artifact refs firstMissing proof trackedHosted-session policyPipeline schema aligned

Site package

Choose the real-site package and task context

The submission is attached to a siteWorldId, taskId, scenarioId, startStateId, robotProfileId, and requested output list.

Package

Selected robot

Mobile manipulator

Bounded robot action vector for hosted rollout execution.

Modality 1

Policy API endpoint

Reference a callable policy service without pasting secrets into Blueprint.

Endpoint URLRequiredHTTPS endpoint or internal gateway URL.Auth handling / redacted secret refRequiredName the auth method and where the secret reference lives.Observation schema URI or JSON refRequiredArtifact reference for observation payload shape.Action schema URI or JSON refRequiredArtifact reference for action payload shape.Rate-limit / runtime constraintsRequiredTimeout, rate limit, batch, and cold-start constraints.Callback / log URIRequiredWhere Blueprint can send or retrieve run logs.Owner contactRequiredTechnical owner for schema or runtime questions.

missing required refsneeds_policy_api_endpoint_ref

Modality 2

Docker container

Reference a reproducible policy container and its runtime contract.

Image / artifact URIRequiredRegistry ref or artifact pointer.Digest / checksumRequiredImmutable digest or checksum.EntrypointRequiredCommand Blueprint should call inside the container.Environment contractRequiredRequired env vars, files, mount points, and secret refs.Hardware / GPU needsRequiredCPU, memory, GPU, accelerator, and driver expectations.Expected input / output schemaRequiredArtifact reference for full IO contract.Runtime notesRequiredStartup, shutdown, determinism, and unsupported modes.

not selected

Modality 3

Recorded action traces

Reference offline action traces aligned to Blueprint tasks and observations.

Trace manifest URIRequiredManifest containing trace files and run metadata.FormatRequiredTrace format, codec, and serialization.Task / scenario mappingRequiredHow traces map to Blueprint task_id and scenario_id.Timestamp alignmentRequiredClock source and alignment method.Observation / action alignmentRequiredHow observations and actions are paired.Success / failure labelsRequiredLabel source and label vocabulary.ChecksumRequiredManifest checksum or trace package digest.

not selected

Modality 4

High-level skill traces

Reference ordered skill-level plans with failure and coverage labels.

Skill taxonomy / versionRequiredTaxonomy name and version used by the trace.Ordered skill sequenceRequiredArtifact URI or inline ordered skill summary.Preconditions / postconditionsRequiredState assertions before and after each skill.Failure labelsRequiredFailure taxonomy or label mapping.Source typeRequiredPlanner, human-labeled demo, robot log, or policy output.Confidence / coverage noteRequiredCoverage gaps, confidence, and reviewed scope.

not selected

Modality 5

Teleop demos

Reference operator demonstrations and controls with rights/privacy scope.

Demo media or trajectory URIRequiredArtifact reference for media, trajectory, or manifest.Operator / deviceRequiredOperator role and input device.Control mappingRequiredControl dimensions and device mapping.Time syncRequiredClock sync with observation and action records.Task / scenario mappingRequiredHow demos map to Blueprint task and scenario IDs.Rights / privacy attestationRequiredArtifact reference or statement for demo use boundaries.LabelsRequiredSuccess, failure, intervention, and safety labels.

not selected

Modality 6

Sim controller plugin

Reference a simulator control plugin without claiming the sim has run.

Simulator / frameworkRequiredSimulator family and integration surface.Plugin URI / refRequiredArtifact reference for plugin package or source snapshot.Supported control modesRequiredPosition, velocity, torque, high-level skill, or custom mode.Observation / action spacesRequiredObservation and action schema references or summaries.Replay / export pathRequiredWhere replay outputs and logs are written.Version / compatibility notesRequiredVersion pinning, unsupported modes, and adapter notes.

not selected