Robot Buyers Should Measure Recovery, Not Just Successful Demonstrations

A successful robot demonstration should begin a buying decision, not finish it. Our analysis is that procurement teams should evaluate failure alongside success. Google describes semantic, physical and operational robotics safeguards. We recommend evaluating those different responsibilities rather than treating a model demonstration as a substitute for testing the proposed installation.

Separate a learned skill from an operating promise

Google DeepMind's Gemini Robotics 1.5 announcement describes an embodied reasoning model working with a vision-language-action model. Google reports transferring motions across different robot embodiments. Physical Intelligence's pi0.5 research announcement, published in April 2025, describes generalisation to previously unseen homes.

Those are attributed research results. Our procurement recommendation is to ask which objects, layouts and task sequences were evaluated, then test the intended application separately. A broader skill repertoire is useful evidence, but a buyer should not infer guaranteed uptime or savings from it. The distinction is between demonstrated capability and a contractually defined operating requirement.

Treat integration as part of the product

The ros2_control documentation covers robot control, while MoveIt documentation covers motion planning and manipulation. NVIDIA's Isaac Sim documentation describes simulation, sensors, synthetic data and robot-stack validation. These sources illustrate different technical responsibilities rather than interchangeable products.

Our inference is that buyers should assess the complete proposed system, not only its learning model. Ask who maintains hardware interfaces, who investigates a failed movement and which software changes require retesting. Request a clear responsibility map before agreeing to deployment. This is a recommended evaluation method, not a claim that every vendor has the same integration burden.

Make recovery visible in acceptance testing

Our recommendation is to include misplaced objects, interrupted sequences and failed grasps in a site-specific test plan. Record whether the robot resumes safely, requests help or stops. Also record the human work needed to restart the process. These are proposed measurements, not published performance figures.

LeRobot documentation provides tools for robotics learning, and Physical Intelligence's openpi repository provides robotics model code. We would treat access to such tools as a starting point for inspection and experimentation, not evidence of a finished commercial service. Ask suppliers to show how a changed model affects the same acceptance tests.

Keep model reasoning separate from workplace safety

Google's robotics safety framework describes semantic, physical and operational safeguards. In the United States, OSHA's robotics standards page identifies applicable standards and says there are no specific OSHA standards for the robotics industry. Its industrial robot systems technical manual discusses robot-system hazards and protective measures.

Our recommendation is to evaluate model behaviour alongside the installation's safety arrangements, with qualified assessment of the actual work. Do not treat a reasoning demonstration as proof that the workplace has met every relevant requirement. The OSHA material concerns the US; buyers elsewhere should identify their own applicable requirements rather than assume identical rules.

What This Means for Practitioners

Google's framework separates semantic, physical and operational safeguards. We would structure procurement around a defined task, an agreed operating environment and documented recovery procedures. Ask suppliers to measure completed work, intervention frequency and restart effort under the same test conditions. Request evidence for any proposed financial benefit instead of borrowing a return estimate from a different installation.

For developers, our assessment is that explaining failure behaviour could make a proposal easier to evaluate. For buyers, the practical question is not whether a robot appears intelligent, but whether the complete system meets a clearly stated operational requirement.

Related reading: MIT Robot-Learning Platform Tackles Tasks Beyond Literal Commands; ToyBlocks Opens a Controlled Testbed for Robot Learning; Humanoid Robots vs Industrial Robots Across Cost, Capability and Deployment; NVIDIA Isaac Sim Needs Real-World Acceptance Tests; AVEVA Outlines Guardrails for Autonomous Industrial AI.