QUERY The Minimum Acceptable Model Standard (SD6742)

[SDMAIL Richard Dudley]

Posted by  Richard Dudley <richard.dudley at attglobal.net>

I think there is some BASIC model quality that we all expect when we read 
a paper or report based on a modeling exercise.   It's good to be open-
minded but there are limits. :-)
 
Even overview models and causal loop diagram without any equations have 
minimum standards. Presumably this is some of what is taught in courses on 
system dynamics modeling.
 
I think that we all expect some sort of standard, although that standard may 
be different depending on the type of model being used.
 
I agree that the way of looking at a problem (or system) might vary from one 
individual or group to another, and this may generate a lot of discussion and 
disagreement.    In fact this is the very reason a clearly laid out model is 
important -- to encourage discussion about the model structure and the 
assumptions made.
 
In order for discussion to take place -- before it can take place -- the model 
must be reasonably understandable to interested parties who would normally 
have (or be able to gain) sufficient knowledge to understand the concepts 
embedded in the model.
 
Thus, in addition to the longer list of what makes an ideal model, we need to 
consider the -minimum- qualities a model must have for discussion to start.
 
Here I am not thinking so much about the structural questions, which might be
 open to discussion at a later point, but about the minimum requirements 
 required for understanding of the model itself in its present form.
 
In other words, what do we need to see in a model in order to intelligently 
comment on that model even if, or especially if, we may not agree with its 
content, or outcomes. 
 
Here is a provisional minimal list of things that I think should be required, 
(most will note that these have been listed elsewhere)
 
1. Clearly presented model structure (not just an overview map)
2. Clearly stated model equations which have no obvious errors.
2. Units for all model components
3. The model is available for examination and such examination should allow 
   running/testing the model
4. Short description of each model component
5. The model should withstand basic validity tests -- such as reasonable 
   extreme conditions tests.
6. Output should be reasonable compared to the real world
7. Output should be consistent with what the authors claim is happening in the 
   model (even if we may not agree).
 
I would expect there would be a second tier of requirements related to model 
quality.  Such as things related to the more careful examination of model 
structure, or whether relationships are correctly formulated. etc.

I don't think the first tier of standards need to be particularly harsh, but 
models which are used to promote policies (for example) should pass minimum 
standards to allow discussion.  (Recall that my original post was related to 
widely distributed, in the literature, marginal models.... which might degrade 
the image of SD modeling).
 

 
Richard
Posted by  Richard Dudley <richard.dudley at attglobal.net>
posting date  Wed, 13 Feb 2008 01:45:57 -0800