Why the results of experiments by the manufacturer differ from the results of actual equipment

When considering imaging tests, it is common to first submit a request for consideration to the manufacturer.
In many cases, several NG samples are deposited and asked to consider whether they are detectable or not.
Then, we ask them to provide us with a verification report, which we use as the basis for developing the actual equipment.
Then, when we deploy them to the actual equipment, it is not uncommon to find that something is different.
There are several possible reasons.
(1) Experiments conducted by manufacturers are often the result of "defect-by-defect" tuning.
(2) The environment of the actual device is not ideal compared to the ideal environment of the laboratory.
But this does not seem to be the case.
So-called "defective samples" have a very large amount of "noise" due to "lot-to-lot variation" and "degradation over time. In the actual inspection environment, the defects stand out more because they are from the same lot, are clean, and have less noise. In fact, the environment is often better in the actual equipment than in the laboratory. The manufacturer may be forced to tune the product.
After all, it is necessary to evaluate the product with a good sample at the product level.
How much of a product is judged to be good?
What kind of items are judged as defective?
How much difference is there between lots?
What is the operability of the product?
What is the operability?
What are the differences between lots?
I believe that the key to success or failure lies in the attitude of "finding out for ourselves.