Should We Start with Models or Measurement?

There is has been a running battle in and around measuring vs. modeling. It's come to a head with the COVID-19 Pandemic. There has been criticism on the models and their assumptions. There has been a cry for more data before releasing the economy and what speed the restrictions should be removed. While COVID is the issue today, there is a big issue going forward on any number of situations that involve Model vs. Measure. I would like to layout some guidance as a rule of thumb

Models are Great for Emergent Situations

In the absence of sufficient data for a situation that has rarely happened, models are ideal. Models are great for scenario planning, where there are numerous unknowns and untried situations. Models are helpful in operational planning where new approaches are being tried out before a significant investment of funds is necessary to change operations. Models are also helpful for people to understand complex aggregations of data from known situations. Models have a great impact when used properly and improved with real data over time.

Data is the Great Equalizer or Diviner of Truth

Having real data for known operational problems is the sweet spot of improvement and automation. There is little substitute for real data leveraged with know and true scientific methods or measures. While models might help visualize the data, the data is the key to assured decisions and lower risks. Some would say that all models are flawed; let's stick with the data. In fact, some would go as far as saying that the money and time used for modeling should be used to clean and normalize data and data sources. They have a good point for known and stable operational situations.

Models and Data Need Each Other

The real truth is that models nor data are ever perfect. Knowing this, we need to exercise wisdom in how to leverage both together, if possible. Models need more data to become more accurate, and data need models/algorithms to make the data digestible and in a form where the best the decisions have a chance to be made. I think the real issue is which one to start within any given situation. This is especially true where there is disagreement. Models let one try things out, and data, if correct and up to date, represent current conditions very well.

Net; Net:

Start with models for fuzzy and emergent situations. Start with data for known problem domains, hoping to discover opportunities.  Work towards a balanced and dynamic relationship between models and data no matter where you start.