REPLY Structural or Behavioral Theory (SD6249)

[System Dynamics Mailing List]

Posted by  "Kim Warren" <Kim at strategydynamics.com>

Perhaps I don't understand important subtleties of the debates we keep
returning to about models being 'right' or not and theories of structure
and behaviour. But does this tendency reflect a lack of confidence in
what we do and its value to people with serious decisions to make? Maybe
this helps answer the other thread about why SD struggles to make
progress.

Geoff Coyle gave me a tip that has proved to be useful over many years.
A model should 'do what the real world does ..' [to a useful
approximation, according to the guidance in George Richardson's post] '
. AND for the same reasons'. 
An example that crops up in various forms in different situations goes
something like this ..
'I am worried about why my sales are falling and what to do about it ..'

. so we explain that with a model [A] that shows sales per customer
dropping - but when we look at the real-world data, this just isn't
happening. Instead, customer numbers are falling. 
. so we explain that with a model [B] that shows customer win-rate
falling and steady customer losses - but when we look at the real-world
data, that's not happening either. Instead, customer win-rates are OK,
but losses are rising. 
. so we explain that with a model [C] that shows customer win-rates,
loss-rates and sales/customer matching the real-world data. 
Is model [A] 'wrong'? - clearly
Is model [B] 'wrong'? - for sure
Is model [C] 'right'? - definitely. Well that specific piece is, at any
rate. 

In many years of doing this kind of thing, no user has ever, ever asked
the abstract question about whether models are right or wrong. Their
concern is simply 'Does this look like the real world I see, and does it
help me make better decisions than I was making before?' - in this case,
pay more attention to keeping our customers, and if we have limited
resources, check that this switch will not damage our win-rate and
sales/customer. 

If we must have a debate about theory here, I guess our mini-hypotheses
are that falling sales are caused by A, B, or C, but our single
structure >> behaviour theory encompasses all three. [We never in fact
build models A and B, because it's a waste of time testing those
hypotheses without including C]. 

Frankly, I don't care a cent whether this reflects anyone's mental model
or not [another question no-one has ever asked!] - the model does what
the real world does and it does so demonstrably for the same reasons
that the user can see playing out in the real world. What is the problem
with extending this principle right back through the system, and round
all the feedback loops - e.g. 'Are we losing customers because they are
not getting called on, or because product quality is bad?' Well, what
does the data say? And if we don't have the data, what might it look
like, and is it important enough to go find out?. 

Following this principle is tough when hard-to-see factors are involved,
like reputation or motivation, but there is often evidence even about
these things to support or refute what we think might be going on. I
guess there may also be problems when models have to be aggregated or
abstracted to a high-level so as to make the model tractable, like using
'pollution levels' to combine everything from water pollution to
household waste tips. .. but then should we not would worry if the model
made any sense to people when they can't see in it things they actually
see in the world around them?  

All of this makes me wonder if folk are getting anxious about whether
models are right because a hypothetical structure is developed without
actually looking at the data as they go? Might they then be struggling
to force a model whose structure is badly flawed to match only the 1-2
behaviour outcomes they originally set out to understand? If so, no
wonder it proves hard to justify the model to the user, and no wonder SD
has a dodgy reputation with management. 

I recall one situation that seemed to fit this possibility - an
insurance firm who didn't trust or use their SD profit model .. they had
a perfectly plausible causal loop diagram on which the model had been
built, but it strangely did not include policy-holders. Now I'm no
insurance expert, but I can't imagine how a model of insurance company
profits is ever going to work if it doesn't have policy-holders in it.
Have I somehow missed the point of this debate?

Kim Warren 
Posted by  "Kim Warren" <Kim at strategydynamics.com>
posting date  Fri, 9 Feb 2007 13:46:47 -0000