Modelling and Simulation as Thought Experiments

In my last post, I talked about the potential use of linear regression in cost estimation.  Linear regression is a simple type of model.  I spent much of my career building and using simulation models and mathematical models to predict system behaviour.  These models, that were sometimes complicated, were simplifications of the real-world that could be solved using a computer.

I recently read Jim Manzi's new book, Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics, and Society. In it, he discusses the potential value of randomization to divide subjects into test and control groups during experiments. 

Manzi begins the book with a summary of the history of experimentation. He notes that the success of the physical sciences is directly related to the fact that the problems in those fields have low causal density. 

Manzi suggests that the reason that social science has made relatively little progress is that predicting human behaviour involves high causal density and is holistically integrated.

Manzi says that randomized field trials, that have proven useful in clinical trials, are being applied successfully by modern businesses. He suggests that they should be applied more widely in social science and public policy. He says that randomized field trials are the only scientific way to determine if the findings from social science research are valid and the proposed public policies will have the desired effect.

In Manzi's opinion, theory and experimentation are a continuous cycle of knowledge development. However, they are quite separate activities.  Theories can be developed in any manner one may wish.  However, experimentation involves a rigorous method that includes test and control groups and the ability to conduct replications.

By this reasoning, modelling and simulation can be considered an extensive form of theory development.  

For the predictions from models and simulations to be verified, one would need to conduct randomized field trials in the real-world.