REPLY Ensuring Quality of Models (SD6664)

[SDMAIL Richard Stevenson]

Posted by  Richard Stevenson <rstevenson at valculus.com>

Tom Fiddaman wrote:
>I'd call this influence, not validity.  A completely spurious model could be 
>involved in a process that changed things a lot. Influence and validity are 
>separate, individually necessary but not sufficient, conditions for a successful 
>project. Validity is about the "in what direction" part - i.e. do the contingent 
>predictions yielded by the model direct change in the right direction.

Kim Warren wrote:
> There is a puzzle in all this, though. Many of the most insightful SD
> models are simplified to high levels of abstraction. Yet skilled SD-ers
> somehow manage to generate confidence in the structure and findings.
> This seems an important skill, as it is often not practical to capture
> enough real-world detail to fully conform to the rule above. I may not
> be alone in welcoming advice on how to accomplish this.

I accept Tom's point about influence and validity being separate and necessary 
conditions.  And Kim's point is also neat; skilled SD-ers must have much more about 
them than SD itself.

So I guess my own point is that competent consultant practitioners must both:
(a) be capable of technically validating models themselves, using appropriate methods.  
    If the client can't trust you to do this, what's he paying for?
(b) be capable of working closely with the client team (workshops, facilitation, 
    personal relationships, etc) to create influence.  This is the hardest bit, by 
    far.  It's politics, usually.

And the point about 'a priori' and 'a posterior' validity and influence is also 
important.  The SD community has been remarkably shy of reviewing the influence of 
its client work 'down the track'.  It's common that 'the difference' takes years 
(even decades) to prove.  That's why it's still so difficult to justify this work 'a 
priori'.  There is often a fundamental mismatch (most often in the very long life-cycle 
industries where we can do most good) between short-term management interests and 
long-term investor interests.  See the pharmaceutical case example on our web site 
(www.valculus.com) for a classic example of mixed management and investor motives.

This reasoning is a factor in my initiative to move away from 'selling' SD as a 
strategy planning method in its own right.  Rather, we intend to focus on industries 
where short/long term discontinuity is an acknowledged issue - and where long-term 
value is the critical investor interest.  We then 'wrap up' SD in a wider methodology 
that includes rigourous financial valuation tools and an approach to understanding 
the long-term role of intangible assets.

'SD inside', indeed - like selling a house without selling the boiler.  But the point 
is - we address the issues that managers and investors increasingly have to communicate 
about.

The sales justification for this approach (we now call it 'resource-based valuation' 
because that's exactly what it is) is the 'a posterior' analysis of various client 
projects that my company Cognitus conducted for blue chip clients over the past couple 
of decades. In many cases we can now broadly see the 'value added' - and in a few cases 
it has been very substantial indeed.  We will be adding more case examples to the 
website over the next weeks and months, to make the point.

This really makes the critical distinction between validity and influence.  Validity is 
the easy bit!

Richard Stevenson
Valculus Ltd
Posted by  Richard Stevenson <rstevenson at valculus.com>
posting date  Tue, 13 Nov 2007 10:42:31 +0000