Modeling methods note

Which experiment should the model help you choose?

A useful model helps us make a particular decision under stated conditions. Its prediction then needs a physical experiment that can put it to the test.

Simulate & optimizeReflections from our 2024–2026 modeling work

Start with the decision

I want a model to earn the next experiment. Imagine it ranks two candidate recovery trials. We could commit to the larger physical test, or first measure an uncertain input. The interesting work is in that choice. What could change the ranking, and which step would tell us more?

To answer, I need to know what decision the model serves, which conditions it represents, which inputs are measured or assumed, and which effects sit outside its boundary. Those details give the prediction a place where it can be useful. They also give an experiment something specific to challenge.

Keep three checks distinct

Shao, Hightower and Schindel connect that discipline to manufacturing digital twins through verification, validation and uncertainty quantification. Their 2022 paper ties credibility to intended purpose and calls for these activities throughout a twin’s life cycle. For me, that turns a broad modeling ambition into questions we can put to the evidence. NIST publication record, December 16, 2022.

Our next modeling study needs three different answers. Verification asks whether the implemented model correctly represents its specified formulation. Validation asks whether its predictions adequately represent the physical behavior relevant to the intended use. Uncertainty asks how incomplete knowledge and variation affect the answer. Calibration means adjusting a model using measurements; those same observations cannot, by themselves, provide an independent physical check. Each answer needs its own evidence.

Choose a test that could change the answer

Return to the hypothetical ranking. If an uncertain input or untested assumption is driving the preferred experiment, the first question is how much it matters. When plausible uncertainty could reverse the ranking, measuring that input may be more useful than committing to the larger trial. The later physical comparison would use stated conditions and separate observations to check the prediction.

Disagreement is where I would want the research to become more exacting. It could reveal an input problem, a missing physical effect or a prediction made beyond the model’s boundary. We would investigate the discrepancy and narrow or revise the claim before extending the model’s role. What changed in our understanding should be clear enough to guide the next experiment.

Optimization inherits the question

A model can optimize the objective we give it while leaving important parts of the job unanswered. A route might look attractive on modeled energy while material quality, handling constraints, elapsed time or downstream acceptance remain unresolved. Those questions still matter to the person who must use the result.

For the process team, a useful comparison needs clear decision criteria and an account of the effects left out. At this stage, scenario comparisons can help us organize physical experiments. Operating performance, safety qualification and net environmental benefit each need evidence for those separate outcomes.

Choose a prediction that can be challenged

Our 2024–2026 development brought together conceptual research, models and early prototypes, with seed support expanding the modeling work. Looking back at that foundation, I see a next step that connects a model to one physical process clearly enough to test it: calibrate with measured inputs, then check the predictions in a separate physical experiment before using them to guide operations. Modeling record, seed-development record, modeling and prototyping scope.

That would give us a model whose next experiment has a clear purpose, an independent physical check and a consequence if the prediction falls short. Which process decision can we define precisely enough for an experiment to show what needs to change?

Cite this article

Jackson, F. (2026). Which experiment should the model help you choose? SBX Research Journal.

Copy the complete reference, or open it below to copy manually.

View full citation

Jackson, F. (2026). Which experiment should the model help you choose? SBX Research Journal. https://www.shutterbugexchange.com/journal/model-guided-experiments.html