A robot vision detection rate is not a picking capability

Detection answers a narrower question

A vision system can report that it detected an object without establishing that the pose estimate is accurate enough, a valid grasp exists, the robot can reach it without collision, the object can be removed or the required placement can be completed.

The buyer's requirement should therefore describe the complete picking outcome and preserve the evidence between stages. A detection percentage is one perception measure—not proof that the configured robot, camera, software, tool, cell and recovery process can sustain the required work.

Put the denominator and conditions beside the rate

ASTM WK78941 is a proposed test method under development, not a published standard. Its public scope separates pose uncertainty, pose precision and detection reliability under complications including nesting, partial occlusion, symmetry, transparency and reflectiveness. Even that draft structure does not reduce vision performance to one context-free percentage or establish complete picking capability.

• Define what counts as an opportunity, detection, false detection, duplicate, missed object and invalid scene.
• Identify the object population, revisions, materials, colours, surfaces, flexibility and permitted damage or contamination.
• Describe count, orientation, pile or layer state, nesting, overlap, occlusion, depletion and foreign objects.
• Record camera pose, field of view, lighting, background, bin or conveyor geometry and allowed environmental variation.
• Separate development, tuning and acceptance scene sets and retain exclusions made after results were observed.

Measure pose error, not only detected or missed

NIST's 2025 work on bin-picking pose error treats pose as position and orientation compared with a ground-truth system registered to the same coordinate frame. NIST also explains that common orientation-error measures can be misleading for symmetric parts because their orientation may not be unique.

The project should define the position and orientation result the downstream grasp and placement need, the ground-truth method, coordinate frames, uncertainty treatment and symmetry rules. A correct class label or segmentation mask does not answer that metrology question.

Follow each candidate through the complete pick

NIST's current perception programme describes bin-picking vision as estimating part poses and supplying a preferred part and path, while its 2025 throughput research treats full bin-picking performance as a separate measurement problem. That independent research supports an end-to-end evidence chain rather than an inference from a perception headline; it does not prescribe a universal pass threshold or test size.

• Valid representative scene presented.
• Required object detected or segmented and a usable pose estimated.
• Candidate accepted as pickable with an appropriate tool-relative pick point.
• Approach, grasp, removal and retreat reachable and collision-free under the complete motion check.
• Grasp acquired and confirmed using the agreed signal without prohibited contact, loss or damage.
• Object placed in the required location and orientation, then the next cycle or agreed recovery completed.
• End-to-end elapsed time, sustained output, interventions and remaining objects recorded.

Classify failures by the decision they affect

Do not silently change denominators between stages or call every non-pick a vision miss. The responsible buyer and integrator should set consequence-based pass rules, uncertainty treatment, repetition plan and escalation path before seeing acceptance results.

• Missed, false, wrong-class or duplicate detection.
• Position error, orientation error or unresolved symmetry ambiguity.
• Detected object with no valid pick point or below the agreed confidence rule.
• Unreachable, colliding or otherwise path-infeasible candidate.
• Grasp not acquired, incorrectly confirmed, dropped or damaging the object.
• Object removed but placed incorrectly or left in an unknown state.
• Empty-bin, end-of-bin or foreign-object condition misclassified.
• Retake, alternate view, shuffle, recalibration, restart or operator intervention required.

Keep manufacturer guidance inside its boundary

Pickit 4.1 documentation distinguishes detected objects from objects that are pickable and adds tool-relative pick points, workspace constraints and collision checks. It also warns that its collision check covers the pick point rather than the robot trajectory leading to it, and its robot template separates no objects found, no reachable objects and no image captured. These are useful first-party product boundaries, not independent proof that a configured Pickit cell or another system completes a buyer's task.

Zivid's current guidance says transparent or semi-transparent objects, reflections, background, lighting and capture angle can affect surface coverage and pose estimation. It separately discusses point-cloud quality, calibration, coordinate transforms, gripper compliance, motion planning and collision avoidance. These are manufacturer instructions for Zivid workflows, not independently verified picking results.

Freeze the configuration and validate representative work

• Record the camera, lens, mounting, lighting, processor, robot, controller, tool, TCP, fixtures, bin and destination.
• Preserve sensor, algorithm, model, dataset and software versions; calibration; point-cloud and detection settings; pick strategy; collision model; paths; grasp confirmation; interfaces and recovery logic.
• Exercise representative variation, difficult and depleted-bin states, rejected candidates, communication faults and agreed recovery cases.
• Retain raw scenes, outputs, timestamps, failure classifications, interventions, calculations, deviations, reviewer and decision.
• Reopen affected evidence after any material change and preserve the previous result for the old configuration.

Keep every claim inside its evidence boundary

NIST is authoritative for its published research and project scope. ASTM is authoritative for the status and public scope of WK78941, which remains a work item rather than a published standard. Pickit and Zivid are authoritative for their own current documentation; their statements remain manufacturer evidence until the exact workflow is tested under the buyer's representative conditions.

RobotAtom records task populations, configurations, source dates, evidence stages, denominators, failures, owners, unknowns and next validation actions. It does not certify vision accuracy, picking capability, safety, integration quality or regulatory compliance. Qualified perception, robotics, integration, quality and safety owners must decide what applies and whether the evidence is sufficient for the actual application.