Choose production samples before judging a robotic picking trial
Agree the part mix, presentation conditions and downstream checks that make a picking trial relevant to your actual work.
Start with the work that arrives
Before a picking trial, ask who chose the parts on the table. A carefully arranged sample can be useful for initial setup without representing the work that arrives during a normal shift. The next decision is not whether the demonstration looked convincing. It is whether the agreed sample set covers the production conditions your team needs the solution to handle.
RobotAtom recommends agreeing a sample register before interpreting trial results. Name the part families, relevant conditions, planned sample counts, evidence references and responsible roles. Keep exclusions explicit. This is a practical scoping method, not a statistical sampling standard or a safety assessment. The responsible engineering and quality teams still need to determine an appropriate test design.
Select conditions, not just a catalogue of parts
Zivid's piece-picking application guidance distinguishes object shape and surface, gripper compliance and motion planning. It also describes the difficulty of transparent objects and the importance of lighting, background and capture angle. These are manufacturer observations about vision requirements; they do not establish the performance of a particular customer installation.
Use that distinction to organise the samples you bring. Record the normal presentation as well as plausible variations: packaging revisions, surface finish, overlapping items, bin fill and the conditions at the actual capture position. Ask the operator which variations occur often and which create difficult exceptions. Do not introduce an artificial worst case and quietly describe it as typical production, or exclude a common difficult case because it makes the trial less impressive.
Say how the trial loads were chosen
A test drawn from normal incoming work answers a different question from a hand-picked set of hard cases. Define the part population and source period first: revisions, packaging, containers, replenishment pattern and the shifts that will use the cell. Agree who selects the loads and when. If the supplier arranges every part for the camera, describe that as a setup test, not as an as-received production sample.
NIST's general process-sampling guidance asks teams to specify what will be measured, the sampling scheme and responsibilities. It also warns that biased samples may miss process variation. Applying that discipline to a robotic picking trial is RobotAtom's recommendation, not a NIST robot-acceptance standard. Keep the source load or capture reference with each run so an awkward result does not disappear into a summary count.
Use a separate lane to challenge the system
Deliberately selected challenge cases are worth testing: overlapping parts, wall and corner positions, low and high fill, changed lighting, and reflective or deformable surfaces where those conditions actually occur. NIST identifies lighting, flexible parts and bin-picking perception as measurement challenges. Pickit's collision guidance illustrates why a detected part can still lack a safe pick path. These sources describe mechanisms, not performance at your site.
Label the challenge set separately from the as-received set. A failure there can identify a boundary to engineer around, but the deliberately selected set cannot estimate how often that failure will occur in production. Conversely, an apparently clean as-received run does not close a rare but consequential condition that was never tested. Record both without combining them into one success percentage.
Agree what each sample needs to prove
A useful row states both the condition and the downstream task. For example, testing a packaged component should say whether it merely needs to leave the bin or must reach a particular fixture in an acceptable pose. Record which configuration was used and where the results are stored. Keep the actual acceptance criterion with those results rather than assigning a green label because a sample was tested.
Robotiq's re-grasp guidance illustrates why downstream handling matters: variation in the initial grasp can affect placement, while an added re-grasp step can consume cycle time. Its guidance recommends checking multiple samples. Whether a jig or another approach is suitable requires application-specific engineering; this article does not recommend a particular fixture or guarantee placement accuracy.
Keep coverage separate from success
Ten tested samples do not necessarily mean ten successful picks. Preserve failed detections, failed grasps, manual interventions, damaged items and unacceptable placements in the evidence record. Track the number tested separately from the outcomes. Otherwise a coverage checklist can accidentally become a performance claim that the test never supported.
The free register linked below flags missing planned counts, untested samples, missing responsible roles and absent evidence references. An agreed exclusion needs a reason. Completing the register means that the entries account for the selected groups; it does not mean the robot is reliable or ready for deployment. There is deliberately no overall readiness percentage and no default sample size.
Close each gap with a decision, not a green cell
For each run, link the selected load to the observed sequence: detection, reachable pick, grasp, placement, recovery or manual intervention, damage and elapsed time where relevant. A tested-count field only says that an observation occurred. It does not say which step passed, whether the downstream task succeeded or whether the test represented the incoming mix.
Before accepting the claim, give every failed or untested condition a disposition: retest after an agreed change, exclude it with a named alternate route, or leave that part of the claim open. The buyer's engineering and quality owners should set the acceptance criterion and any statistical precision requirement for the actual risk. NIST's sample-size guidance does not provide a universal number of parts or picks for this application.
Make exceptions part of the agreement
When a part or condition is excluded, document the operational consequence. Will it be routed to a separate process, presented differently or included in a later engineering scope? Identify who accepts that boundary and how the operator will recognise the exception. An exclusion that exists only in the trial team's notes is not a usable production instruction.
For adopters, this process makes the limits of a trial visible before committing to a rollout. For providers, it creates a defined sample obligation instead of an open-ended promise to handle every possible item. RobotAtom uses these requirements to scope the complete solution and its deployment evidence. Start with one part family and its real downstream task, then expand the register as the operating scope becomes clearer.
Sources
Material claims were reviewed against the following primary sources. External links open the publisher's website.
- Zivid application requirements — undated manufacturer guidance, reviewed 1 September 2026
- Robotiq re-grasp guidance — undated manufacturer guidance, reviewed 1 September 2026
- NIST/SEMATECH e-Handbook, Define Sampling Plan — publication date not shown, checked 27 September 2026
- NIST, Perception Performance of Robotic Systems — updated 24 April 2026, checked 27 September 2026
- Pickit, Collision prevention — current 4.1 documentation, date not shown, checked 27 September 2026
- NIST/SEMATECH e-Handbook, Selecting Sample Sizes — publication date not shown, checked 27 September 2026
This article provides general information. A robotics project still requires site-specific engineering, safety and regulatory review.