REPLY Ensuring Quality of Models (SD6661)

[SDMAIL Richard Stevenson]

Posted by  Richard Stevenson <rstevenson at valculus.com>

Some good points, Richard.

There are of course, many approaches to technical model validation.  
In fact, it's almost a subject in its own right.

My own (maverick, possibly) view, is that models are only valid if 
they actually change behaviours.  For consultants, this is far more 
about developing good client relationships and interactions than about 
'technical' model validation.  If, as a consultant, you can't 
technically validate your own models, then you are not a consultant at 
all - a charlatan, even? And if your client needs an external third 
party to validate your model - then you've failed anyway.

Rather, developing client confidence in a model is more a 'relationship' 
issue. Good consultants succeed by using models if they understand that 
the model is not an end in itself but, rather,  a communication tool with 
the client - to be developed, debated and ultimately validated through 
client interaction and informed 'buy in'.  As soon as a model becomes a 
'black box' to the client, then as a consultant you've lost the plot.

This is really where I  have significant differences with some SD 
academics and many 'consultants'.  In the main, I can confidently say 
that 90% of academics have never conducted anything more than 'superficial' 
client work - they feel good if they can get a client to sponsor an 
academic project.   And too many 'consultants' are actually doing bad 
client work - not because they can't build clever technical models but 
because they really don't understand what it takes to 'make a difference'.

What is a "difference"?  That's the 'a posterior' perspective.
Ultimately, the ONLY fundamental criterion of SD model validity is "what 
did it change, in what direction, and by how much?"  This is, of course, 
an 'a posterior' view.  And that's the problem with all planning tools - 
you can't 'sell' the value of the model in advance. If only we could "sell" 
SD on an 'a priori' basis!!

I'd be rich, for a start,  Our website (www.valculus.com) includes a number 
of short case examples of SD model projects that changed things, a lot (in 
retrospect!).

regards
Richard

Richard Stevenson
Valculus Ltd 
Posted by  Richard Stevenson <rstevenson at valculus.com>
posting date  Fri, 9 Nov 2007 17:23:41 +0000