REPLY Ensuring Quality of Models (SD6660)

[SDMAIL Fabian Fabian]

Posted by  Fabian Fabian <f_fabian at yahoo.com>

About Models QA:

In the abscence of an "ISO SDM", there are two types of filters that 
any model of any type could be subject to: Model Verification and Model 
Validation.

The latter has as a resource a specific battery of tests tailored for 
System Dynamics (as described in classic papers by Forrester & Senge; 
and Barlas). Apart from those 'objective' tests, model validation is a 
process of  confidence buildup during the model building process. That 
confidence is highly correlated with the perception of model ownership 
by the client (internal client, in answer to Richard's question). Thus 
the importance of considering designing a group model building process, 
including the model's client.

I cannot foresee the development of a universal ISO for the confidence 
buildup process, as each organization has its own DNA (genetic code), so 
each client would gain confidence in a different way. This diversity is
 also the reason for benchmarking not being useful as a SD Model QA 
 component.

The former (Verification) could be implemented by internal or external SD 
consultants who could verify if the model has methodological  and/or 
formulation flaws. Balci has developed some papers about model verification.

In closing, in my opinion there could be attempts of standardizing the 
verification process and even the implementation sequence of validation 
tests, aiming to defining a SD Model QA process.

However, we certaintly can't imagine one ISO component stating something like: 

Customer! Thou shall gain confidence on the model following THIS process, 
and in less than 3 months.

Be all well...

Fabian Szulanski
System Dynamics Centre
Instituto Tecnologico de Buenos Aires
Posted by  Fabian Fabian <f_fabian at yahoo.com>
posting date  Sun, 11 Nov 2007 05:57:19 -0800 (PST)