Sterman, John D., "Appropriate Summary Statistics for Evaluating the Historical Fit of System Dynamics Models", 1983
ua435
System Dynamics models are often faulted for their reluctance to employ formal measures of goodness-of-fit when assessing the historical behavior of models. As a result, the validity of system dynamics models is often questioned even when the model’s correspondence to historical behavior is quite good. This paper argues that the failure to present formal analysis of historical behavior creates an impression of sloppiness and unprofessionalism. After reviewing the concept of validity in simulation modeling, the paper proposes a simple set of summary statistics appropriate for system dynamics models (the root-mean-square error and Theil inequality statistics). The statistics allow the error due to individual behavior modes to be analyzed, do not require the use of formal parameter estimation procedures, and can be conveniently computed. A large model of the U.S. economy is used to illustrate the use of statistics.