REPLY Models, Problems and Systems (SD7001)

[SDMAIL Jack Harich]

Posted by  Jack Harich <register at thwink.org>


Jack Harich wrote:
> '... the SD rule of don't model a system, model a problem. Or in 
> science, solve the specific problem first, and then develop a more 
> powerful generalization from that. The latter is the real payoff'. 

Dr John P Weldon replied:
> Strong indications, that all is not well with the above 'rule', are 
> provided by the following:
>     * The Ingalls case (/Interfaces/, K Cooper, 1980); one of the 
> brightest jewels in the SD crown.
>     * Corporate models that are used for system-wide activity and 
> resource control, planning, budgeting, and accountability etc. 

He then offers an improved rule:
> If a genuine 'problem' can be identified of smaller content and scope 
> than that of the system in which it resides, modeling shall be 
> confined to entities (below the number needed to model the system 
> itself) that can adequately explain and resolve the 'problem'. In all 
> other cases modeling of the system shall be regarded as appropriate 
> and necessary.
> The existing 'rule' is pernicious because it discourages modeling at 
> the *system level* in cases where that is appropriate. It also 
> contributes to confusion on the part of potential clients for SD 
> applications. The existing 'rule' is part of the baggage that needs to 
> be jettisoned, if the SDS is to have any real prospect of growing the 
> SD field.
I agree. But a model at the "system level" is not the same as the 
modeler who sets forth to model a "system" instead of a problem. In the 
second use of the word "system," the model will likely end up being of 
little value, due to being too large, too hard to comprehend, too 
expensive, too hard to maintain, and most of all, not enough focus on 
particular problems that arise in the system.

John Sterman, in Business Dynamics page 89, writes that:

"The most important step in modeling is problem articulation. What is 
the issue the clients are most concerned with? What problems are they 
trying to address? What is the real problem, not just the symptom of 
difficulty? What is the purpose of the model?

"A clear purpose is the single most important ingredient for a 
successful modeling study.

"Beware the analyst who proposes to model an entire business or social 
system rather than a problem. [This essentially says don't model a 
system, model a problem.] Every model is a representation of a system -- 
a group of functionally interrelated elements forming a complex whole. 
But for a model to be useful, it must address a specific problem and 
must simplify rather than attempt to mirror an entire system in detail."

And Jay Forrester writes in Urban Dynamics, page 113 that:

"The first step in modeling is to generate a model that creates the 
problem."

I can't find where I ran across the rule "Don't model a system, model a 
problem." But I think the above quotes try to say what that (overly?) 
short rule really says. We build models to solve problems, not to model 
whole systems and then hope such models will be useful.

For example, Sterman writes that "Ingalls spent several years pursuing 
the claim [an expected $500 million cost overrun], but traditional 
project management tools did not provide a means to quantify the ripple 
effects. Ingalls turned to system dynamics to quantify the delay and 
disruption created by Navy design changes. The model, developed by 
Pugh-Roberts Associates of Cambridge, Massachusetts, simulated all 
phases of the DD and LHA projects, from the award of the contract to the 
delivery of the last ship, then 5 years in the future."

So the impetus was to model a problem. That this required modeling all 
phases of the DD and LHA projects caused the model to be quite large. 
Yes it was a system. But it was not the whole system, even of the 
project part of Ingalls. It was a subset, just enough necessary to 
achieve the model goal, which was to quantify extra costs due to Navy 
design changes.

Now suppose someone had modeled Ingall's system of projects, with no 
thought to what problems it would be used for. The model would then 
include all sorts of things that would have been of little use here, 
like inclusion of safety principles (these are a big deal in engineering 
these days), staff retention and training, weather and supply chain 
disruptions, etc.

Instead, page 58 of Sterman shows how the dynamic hypothesis focused on 
these work phase stocks: Work to be Done, Work Really Done, Known Work, 
and Undiscovered Work. The model grew from there.

Even big system models are built to address problems. Company wide 
models are build so that management can make better decisions. They 
reduce the problem of not enough structural or quantitative data, which 
leads to better analysis and better solution alternatives evaluation. 
Such models emphasize areas where a particular company expects it will 
most run into problems, rather than modeling the whole equally.

National models do the same thing but the problems are different. But 
all national models focus somewhat on addressing particular areas, so 
that they may be better understood and managed.

The short rule is so terse it's easily misleading, so I like your 
attempt at improving it.

But to me the short form says it all: When you're modeling, don't think 
about mimicking a system. Instead, "generate a model that creates the 
problem" in the system of interest.

Thanks, John, for such a thoughtful post.

> Posted by "Kim Warren"
> SDMAIL Dr John P Weldon wrote:
>  
>> The 'rule' therefore needs to be substantially redefined along the 
>> lines below.
>>
>>     If a genuine 'problem' can be identified of smaller content and 
>> scope than that of the system in which it resides, modeling shall be 
>> confined to entities (below the number needed to model the system 
>> itself) that can adequately explain and resolve the 'problem'. In all 
>> other cases modeling of the system shall be regarded as appropriate 
>> and necessary.     
>
> I'd like to build on John's post, because I too have been puzzled by 
> where this mantra of 'model the problem, not the system' came from and 
> what its justification might be. We 'model systems' in all kinds of 
> ways, whether it is designing an aircraft, assessing the weather, 
> designing a piece of software and so on. And we use those models to 
> allow us to manage the resulting system, where that may be possible. 
It's a fine point I raised, with the maxim "Don't model a system, model 
a problem." Stated this way, it emphasizes the model boundary question: 
What should be in a model and what should not? What will help to 
understand and solve the problem should be included. The rest of what's 
in a system should not.

Thus one is actually never (?) modeling a system, but a problem 
expressed as part of a system. Perhaps that's the nub, as Mark Twain 
would have said. :-)

And now for a bit of humor. Pale faced analysts and this list need a bit 
of that now and then. See: 
http://www.timsheppard.co.uk/story/dir/twain.html
where Twain uses "nub" six times. This story has long been a favorite of 
mine, though the ending dialect is a little dated.

So what is the nub of this discussion?

> There seem to be two broad problems with just 'modeling the problem'
> 1. Unless we are very careful, we may miss factors that are 
> significant contributors to the problem because we decide on 
> boundaries that seem to make sense, but are not in fact comprehensive. 
> Consequently, we may solve *this* occurrence of the problem, but leave 
> the organization exposed to recurrences that are due to different 
> factors - e.g. we fix service quality now by modelling and identifying 
> how best to develop a service-support team, but next week the problem 
> comes back because the sales force manages to sell a different mix of 
> products.
>
> 2. We do not leave management with any means of continuing to manage 
> the system as a whole. This would seem to be like giving an aircraft 
> pilot a sequence of indicators and instructions to get out of a 
> tail-spin, but no overall control system or procedures for actually 
> flying the plane in general.   
Certainly good points. This builds on Sterman on page 89 (see earlier 
post) when he says "What is the real problem, not just the symptom of 
difficulty?" As far as I can tell, your suggestions go beyond the 
original problem and get to additional real problems, ones management 
probably doesn't even know they have. This is, of course, one of the 
hallmarks of a good analyst.
> I can see circumstances where modelling a problem would be valuable, 
> with caveats about 1 and 2 above, but would we not rather model to 
> provide a set of policies for managing things well, so we don't get 
> into trouble in the first place?
Touché! Proactive is always better than reactive solution of difficult 
problems, as the world is about to find out on the sustainability problem.


Jack
Posted by  Jack Harich <register at thwink.org>
posting date  Wed, 07 May 2008 20:53:12 -0400