A robot demo is not an acceptance test

A demonstration and an acceptance test answer different questions

A robot demonstration can show that a capability is plausible and worth investigating. It does not by itself establish that the configured solution will meet the buyer's requirement at the intended site, with representative objects, people, interfaces, operating conditions and failure modes.

Acceptance evidence needs a requirement, a frozen test object, a repeatable procedure, a pre-agreed pass rule and an attributable result. Without those elements, a smooth run remains useful discovery evidence rather than a procurement or release decision.

Write the decision before writing the test

The procedure should trace each case back to the approved requirement brief. A test that cannot identify the requirement it supports may still reveal something useful, but it should not silently become an acceptance gate after the result is known.

• Name the exact work and output being accepted, including units, tolerances, throughput, quality and allowed interventions.
• Separate hard constraints from preferences and unknowns; a failed hard constraint cannot be offset by an impressive result elsewhere.
• Define the representative objects, variants, task states, shifts, layout, environment, people and upstream or downstream dependencies.
• State who prepares, witnesses, measures, approves and owns each unresolved item, and who may change the test procedure.
• Agree the possible decisions—pass, fail, conditional pass or retest—and what evidence or corrective action each one requires.

Separate factory, integration and site evidence

IEC 62381:2024 defines requirements and checklists for factory acceptance, factory integration, site acceptance and site integration testing of automation systems in the process industry. Its public scope says the parties establish the activities and responsibilities and adapt test plans to the applicable process, plant or equipment specification.

That standard is not a universal robot-acceptance rule. It is a useful example of why one event should not be expected to answer every question. A project can use its own agreed stage names, but should say which configuration, interfaces, site conditions and responsibilities each stage actually covers.

• Before shipment, verify what can be established on the supplier's frozen build and record what the factory setup cannot represent.
• Test the proposed control, data, safety-related and equipment interfaces together rather than inferring integration from separate component demonstrations.
• At the site, repeat affected cases with the installed utilities, network, layout, fixtures, traffic, materials and operating team.
• Do not release production merely because an earlier stage passed; carry its evidence forward with stated limits and close the site-specific gates.

Freeze the exact test object and configuration

ASTM F3218-25 is an active practice for documenting environmental conditions used with test methods for robot arms, automatic machines and uncrewed ground vehicles. ASTM F3713-25 separately covers workpiece configuration in mobile-manipulator performance evaluation. ASTM says these records support context, comparison and replication. Their scopes are specific; the checklist below is RobotAtom's cross-robot synthesis, not a claim that either practice mandates it.

• Identify the robot, controller, hardware options, software and firmware, licences and enabled operating modes.
• Record the tool, adapters, sensors, fixtures, workpiece properties, payload states, coordinate frames, calibration and consumables.
• Preserve the program, parameters, maps, models, interface versions, network settings and safety configuration used in the test.
• Capture the test apparatus, layout, surface, lighting, temperature and other conditions that can affect the result.
• Treat a substitution or update as a configuration change to review, not as an invisible continuation of the same test.

Make the procedure repeatable and representative

NIST's emergency-response robotics programme uses test apparatuses, procedures and performance metrics to isolate capabilities, control variables and repeat tests. NIST also says user communities set the performance thresholds for their mission. This work is scoped to emergency-response robots and operator proficiency; it does not prescribe acceptance for every commercial robot application.

The practical lesson is to start with controlled cases that reveal a baseline, then introduce the material variability the real task contains. Preserve the order, repetitions, randomisation where relevant, measurement method, raw results, failures and interventions so another qualified team can understand what happened.

• Include normal object and process variation rather than only the easiest prepared sample.
• Exercise starts, stops, pauses, rejected inputs, blocked paths, communication loss and agreed recovery tasks.
• Measure the complete task outcome and productive time, not only a favourable robot motion or sub-step.
• Use the planned operators and maintainers where human proficiency or intervention affects the outcome.
• Record exclusions explicitly and assign follow-up evidence instead of treating an untested condition as a pass.

Choose repetitions and pass rules before seeing the result

ASTM F3499-21 provides a concrete example for automated guided vehicle docking. Its public text requires the test requestor to define the desired probability of a successful repetition and the confidence in that probability, which determines the sequential successes required, and it records environmental conditions. This is a docking test method, not a universal sample-size formula for robotics.

For each project measure, a qualified owner should choose repetitions, treatment of invalid runs, uncertainty, pass margin and review method that fit the consequence of error and the expected variability. A supplier-selected highlight run, mean value or best result should not be substituted for that agreed rule.

Preserve deviations, ownership and change control

• Record every deviation from the approved procedure, who authorized it, why it occurred and which conclusion it may affect.
• Keep raw measurements, logs, media, software and configuration identifiers, calibrated-instrument details, witness names and the signed decision together.
• Turn a conditional pass into named open actions with owners, due dates, evidence and a rule for closing the condition.
• Define which changes to hardware, software, tool, object, layout, interface, environment or operating method reopen affected tests.
• Continue operational monitoring where a bounded acceptance event cannot establish longer-term reliability, availability or process capability.

Keep every conclusion inside the tested boundary

IEC, NIST and ASTM are authoritative for the public scopes described here, but none of the cited pages verifies a named robot, supplier or deployment. A supplier-run demonstration or report remains supplier-provided evidence even when reviewed. Label a result buyer-witnessed or independently measured only when the observer, method, raw results and evidence custody support that status.

RobotAtom records the requirement, evidence stage, exact configuration, test conditions, result, deviations, owners and unknowns. It does not perform or certify acceptance testing, robot safety, process capability or regulatory compliance. Qualified engineering, safety, quality and operational owners must decide what applies and whether the evidence is sufficient for the actual application.