Very Basic Question

[Jean-Jacques Lauble]

Hi Guido

It is very easy to be badly understood when one tries to explain something in written words.

When I wrote about why fear experts if it is the client who pays has nothing to do with liberalism.

It has to do with the final objective of SD. It is to satisfy the client in the long run of course.

I have built models now for 10 years, and I have the chance to be at the same time a modeller and a client.

The first years I followed very much expert’s advices (books, distant web courses etc..) and thought that it would necessarily generate my satisfaction as a client. It was not the case. With the time I do not follow anymore so called expert rules if it does not generate my satisfaction as a client. This does not mean that I do not consider any proposed method but it is my satisfaction as a client that counts. I have tool learned that SD is no magical solution and that it generates a return only if you have made the necessary efforts which are much higher than one can imagine and which a naive client may imagine. The main difficulty with SD is to make the client accept the necessary efforts to get a satisfactory return and from what I have seen so far, most of the time one does not see anywhere any client effort in any modeling work, and one does not seen anyway the client at all most of the time.

Another question is about the term of experts. What is an expert in SD? There is no definition of it.

I think too that if there was an authentic expert in SD, taking into account the promises of the method, he would be very rich, applying the method for his own affairs or for other's and be very much occupied using it with profit, and would have no time to participate with any forum.

But the satisfaction of a client is something much more tangible than expert’s advices, and on top of that it is the client who pays you.

I think too that overconfidence comes much more from modellers thinking that following established rules is enough for success especially with a method like SD which looks extraordinarily promising in theory.

Best regards.
JJ

[Guido Reichert]

Hi JJ,

I am sorry, if I misunderstood your statements.

The satisfaction of real client needs is what consulting should be aimed at. What I doubt is that there is a straightforward and immediate link between stated needs and real needs. But of course in the long run whatever happens - the client should be satisfied, so lets have the long-term satisfaction of the clients as the benchmark for successfully "applying" SD in the real world.

There seems to be in my humble (and still juvenile) perception still to be some need for a "proof" that SD is useful to clients in the long run. If you look in the theory most books I have read state that while there is a feeling that SD is very useful, there are no "proofs" or "strong indications" out there that decisions and change has happened in a significantly better way using SD that it would have if SD had not been used.

Where are the very, very satisfied clients that sponsor say the SD-conference. Most sponsors there seem to be offering consulting services or SD-tools. O.k. there is Boing and probably GM as a long time applicant - but who else? I have noted that the SD-society has just put forward a website collecting successful SD-case studies. Considering the age of the craft that should be full shouldn't it?

You have given a reason that quite resonates with what I am experiencing: Nobody seems to really have time to really get envolved in the modeling on the client side. In the end people seem to expect that you know what you are doing and that there is a nice presentation of results and maybe a convincing management cockpit with look and feel.

It seems Russell Ackhoff had a point: http://ackoffcenter.blogs.com/ackoff_center_weblog/files/Why_few_aopt_ST.pdf

Kind regards,

Guido

[Jean-Jacques Lauble]

Hi Guido

Stated needs are wishes and are immense.

Real needs take into account the realities: resources needed, time constraints, implementation problems etc.

This is why stated needs and real needs are different. This is why too I start with very small models that will give insights about the problem and will generate creativity and slowly make the difference with wishes and possibilities. The drawback is that it needs a permanent and strong involvement of the client, something that is rarely possible to realize.
People want to pay for something that will give results with no bothering, so that they can concentrate on other things. They may dream a while, and if they have not participated to the effort and not allowed for the necessary time, they will strongly be unsatisfied when they awake. But it is difficult to discuss about generalities. It all depends on the type of problem and its environment. Everything else is academic and not practical.

One thing that is sure is that if you cannot make your client permanently involved, he will not benefit from the study.
I once worked with a consultant who made a relatively big model with more than 30 parameters.
To study the model I had just to run it at least a billion times just to allow each parameter to take two values: medium low and medium high!
Of course I never took the time to study the model having no simpler models previously built to guide the study of this more complex model.

I may look pessimistic, but I think that in the long run it is better to be realistic and pessimistic than naively optimistic.

Regards.
JJ

[Etiënne Rouwette]

Dear Guido,

Your question on whether there is evidence 'out there that decisions and change has happened in a significantly better way using SD that it would have if SD had not been used' has been of interest to me for quite some time. I think there is some evidence. In 2002 Jac Vennix, Theo van Mullekom and I published a paper in System dynamics review that looked at 107 published cases on group model building. Group model building (GMB) is a generic term for building SD models with experts and/ or stakeholders. There were few studies comparing GMB to other interventions, and naturally those that do use a comparison group tend to build models with students. In those modeling projects there is no actual implementation and some authors (e.g. Colin Eden) argue that the decision making process in a student group is uncomparable to that of 'real' clients. So ideally you would like to have both a control group *as well* as real clients. The only examples that come to my mind are the studies by Steve Huz and Krys Stave (see references below).

In short, the results of the 2002 study for decisions and change are:

- of 107 cases, 35 report on commitment to a decision taken, in 31 of these the impact is positive;

- of 107 cases, 46 report on system changes. Of these, 42 out of 46 report implemented changes at the system level.

The paper also goes into the question to what extent the 107 cases are representative of 'all' GMB projects with clients. The overwhelming majority of cases reports positive results, which raises some suspicion. On the other hand, many studies are co-authored by clients.

You raise the point thar the client's stated needs do not necessarily reflect the real needs. Interestingly, a similar process seems to be operating in the client's perception of learning. In several projects, a client would state not to have learned (much) of participating in modeling sessions. Comparison of answers to a questionnaire before and after the sessions would however indicate clear changes in opinions on the problem modeled. When this was pointed out to them, most would say something like "now that you mention it, I did not know X which was mentioned in session Y. And yes, that might have changed my idea on what to do about this problem". So the conclusion was that people have little insight into their own learning - which turns out to be a finding that was widely known in cognitive psychology. See the reference below: Telling more than we can know: verbal reports on mental processes.

You also mention the case repository (http://cases.systemdynamics.org/). The idea in launching this was to add cases on the basis of publications in System dynamics review and conference papers, and to invite all discussion list participants to contribute their own cases. So far we have 36 cases but that there are many more examples not yet written up. So I would encourage anyone who has an idea on a case to send it in - see the website on how and where to do so.

Thanks,

Etiënne





Huz S, Andersen DF, Richardson GP, Boothroyd R. 1997a. A framework for evaluating systems thinking interventions: an experimental approach to mental health system change. System Dynamics Review 13(2): 149–169.
Nisbett, R., & Wilson, T. (1977). Telling more than we can know: verbal reports on mental processes. Psychological Review, 84(3), 231-259.
Rouwette E.A.J.A, Vennix J.A.M., Mullekom T. van. (2002). Group model building effectiveness: a review of assessment studies. System Dynamics Review 18(1): 5-45.
Dwyer, M., & Stave, K. (2008). Group model building wins: the results of a comparative analysis. Paper presented at the 2008 System Dynamics Conference.