When you should use virtual DoE in the design process
You are reading the second part of an article trilogy about Design of Experiments (DoE). The trilogy covers the why, when and how to perform virtual DoE in the design process. By following these guidelines, you will experience a faster, smarter and cheaper way to get results that are vastly more reliable and ultimately a greater chance of succeeding with your product.
In the following, we will focus on when to apply virtual DoE to your design process, by starting before any physical tests are made, as a way of gaining insights into which physical tests to perform.
VIRTUAL TESTING CAN NEVER REPLACE REAL-LIFE TESTS FULLY,
but we can still gain insight for qualification of concepts
As the great statistician, George Box so eloquently put, “essentially, all models are wrong, but some are useful”. This is especially true when using metamodels for decision making. For starters, when selecting which factors to investigate, the engineer must at least have a hypothesis on which factors might have an influence. So, while these models can be useful, caution and experience should be applied.
Keys differences between physical and virtual testing
The important differences between physical and virtual DoEs are briefly listed:
- An advantage of virtual DoEs over their physical counterparts is that the resolution of virtual DoEs can be vastly higher. The reason for this is the lack of random error in computer simulations; in other words, physical tests often need replications and thus many precious test runs are spent on factoring out the noise, instead of using test runs on more different configurations of the product. With computer simulations, we do not have that problem and more variations can be tested for the same amount of test runs.
- Adding to the point above, testing for additional factors comes at no extra cost, since dimensions, forces, etc. are easily changed in the CAD model. The same does not apply in the physical world, as more variants of expensive prototypes are required.
- Although running computer simulations can be time-consuming, the cost and time spent on creating a metamodel is very low compared to running physical tests, where lead times on prototypes can be long.
The points above essentially highlight the importance of the when – the virtual testing can be done as soon as a preliminary model, early in the product development cycle, is constructed.
Simple problems, hard answers
Sure, computer simulations are deterministic in nature, and the underlying constitutive equations (relatively) simple, but the outcome can be very unpredictable indeed. Add to that, that even though computer simulations are relatively fast, testing a new configuration ad-hoc takes time for heavy simulations, and provides only that single data point. Running a batch of simulations according to a systematic test design is much more efficient. Therefore, front-loading testing activities as a virtual DoE can be a very cost-efficient method for sensitivity analysis early in the design process.
Why bother running virtual DoEs in the first place if we have to do all the testing with physical models?
DoEs studies are great for finding the optimal parameter values in terms of robustness: Numerical measures on the performance when the different factors vary can be used for determining the nominal settings, at which the sensitivity is the lowest, and for tightening or loosening tolerances. Remember though, this is an optimization of an already detailed concept, and the results gained from an early-stage virtual DoE should not be used for that purpose.
In other words, since the results are unverified in a virtual DoE, the numerical values cannot be taken at face value and used as exact measures for the product performance. Rather, the results should be used for only:
- Assessing the relative importance of the factors. This can be illustrated by e.g. a Pareto chart, as seen below, where the impact of each factor is ranked.
- Identifying the type of relationship to the performance parameter and that particular factor. The shape of the factor/performance curve, as seen below, illustrates how a virtual DoE with many measurement points can capture the nonlinear nature of the factor/performance relationship.
Because these virtual DoEs allow for testing many more factors than physical tests, picking out factors for a later-stage DoE, becomes more streamlined, especially since the number of available test runs is scarce with physical tests. In addition, because we will also have an idea on the type of relationships between these important factors and the performance parameter(s), we can determine how many measurement points we are likely to need in physical testing. In terms of early-stage usefulness, the results provide insights on which concepts are more sensitive than others and thus help in the decision-making.
Our next article will focus on how we actually perform these analyses, and we will dive into a hands-on example of the application of the methods.