A System Dynamics
Skills Inventory for Self Taught Practitioners
To begin
to answer the question: What constitutes System Dynamics Mastery?
By Gordon Kubanek
A report for the Policy Council of the System Dynamics Society. Feb.2002
Table of Contents
Introduction.............................................................................................................................. 1
Goal................................................................................................................................. 2
The
Matrix as a Learning Tool......................................................................................... 2
Background.............................................................................................................................. 2
Previous
work...................................................................................................................... 3
Cross
the Chasm.................................................................................................................. 3
System
Dynamics as a Trans-Discipline.............................................................................. 3
1.Reference
Based Discipline......................................................................................... 3
2.
Definition Based Discipline........................................................................................ 3
3.
Evolutionary Based Discipline.................................................................................... 3
The SD Skills Inventory......................................................................................................... 4
Dreyfus’
Five Stage Model for Skill Acquisition................................................................ 4
Stage
1: Novice............................................................................................................... 4
Stage
2: Advanced Beginner........................................................................................... 4
Stage
3: Competent......................................................................................................... 4
Stage
4: Practitioner........................................................................................................ 4
Stage
5: Expert................................................................................................................. 5
SD
Competencies................................................................................................................. 5
The
Systems Worldview.................................................................................................. 5
Computer
Modeling Skills............................................................................................... 5
Applications.................................................................................................................... 6
Communication
& Leadership.......................................................................................... 6
Skill
Acquisition.................................................................................................................. 7
The
Skills Inventory Matrix................................................................................................. 8
Conclusion............................................................................................................................... 9
References............................................................................................................................... 9
Introduction
Sustained growth of the field requires that the quality of the models built are sufficient to not discredit the SD method. Since most SD practitioners are self-taught and we have no certification program the skill set of these practitioners is bound to be highly variable. This makes any quality control initiative a challenge. This template is meant as a first step to finding a way for the Society to support the building of consistently high quality models by allowing practitioners to self-assess their level of knowledge and skills so that they build models appropriate to their abilities. As George Backus wrote on the SD list serve: [Oct.13’01]
“We remain an unidentified entity. When an
organization asks for a System Dynamicist or Systems Thinker who knows what to
expect?”
This question begs another question: Do SD practitioners really know what the field is & how best to use its tools in a given situation, given the fact that it’s conceptual boundary have not been defined in any generally accepted way? For better or worse, the answer to both of these questions is likely no.
Goal
The
purpose of this Matrix is to begin the journey of defining the meta-discipline
we call System Dynamics so practitioners know what is expected of them and
organizations contracting for a SD study know what to expect.
We have developed
a template for SD practitioners to assess their knowledge and skill with. This
guideline is meant for practitioners, especially those who are self-taught, to
be realistic about their ability to write SD models from which useful policy
recommendations can be made. In effect this template is a mirror against which
a modeller can “see” what his/her strengths & weaknesses are so that he/she
can get the training or readings required to work as the level of expertise
required. This beginning template is not meant to be complete but rather to
begin the journey towards developing a skill appraisement process that the SDS
can use to support practitioners.
The
Matrix as a Learning Tool
First: since our values rest on a belief of the DYNAMIC nature of systems this Matrix must allow for change, evolution and improvement as defined by the USER. Second: as another foundation of SD is the Learning Organization concept. Thus, this Matrix is meant as a Learning tool for practitioners to improve their skills in applying the ideas and tools of SD. In particular we need to help self taught practitioners realize that the change in SD skill level is NOT linear but exponential due to the positive feedback loop shown next. Without formal training or a tool such as this skills inventory these practitioners run the risk of believing themselves to be proficient when they are actually competent: between these two skill levels is a great divide referred to here as “The Chasm”.
Now is a good time to begin developing this template as concerns about quality have surfaced over the past couple of years. Now that Systems Thinking is becoming “mainstream” in Management circles and more books than ever are being published about SD we must be ready to avoid an “Overshoot & Collapse” scenario created by our own internal lack of capacity.
Background
Previous
work
This template develops and integrates the work done by Daniel Kim [1989], I.Martinez & G. Richardson [Best Practices in System Dynamics Modeling, Atlanta, 2001] & others. Thus, this template is not new but an updating and integration of proposals and research done by many concerned with this issue. This template is purposely brief. It is only a skeleton of what could easily blossom into more if & when the Policy Council supports this development. The Skill Acquisition model is that of Dreyfus [1980]. Clearly this initiative is working in parallel with the Curriculum Committee who is developing a body of knowledge that Institutions can use a basis for the designing their SD programs.
Cross the
Chasm
Proficient Competent
In particular we want to help practitioners “Cross
the Chasm” from being Competent
to becoming Proficient by giving them a tool to be measure the
improvement and learning against as shown below.
System
Dynamics as a Trans-Discipline
System Dynamics, like Mathematics, is a trans-discipline. As such, it only
becomes meaningful when applied to a problem found in a particular field: ie.
Performance Measurement. This means that the typical ways of measurement
knowledge and skill level do not apply. Informing Science, the field of
Information Systems, [ http://www.informingscience.com
] has investigated this issue and suggest this approach for dealing with
measurement of an individuals competency. We shall use the third concept to
develop this template.
1.Reference Based Discipline
This method determines which knowledge is and is not SD. We could use this approach to define the field. For example, SD is derived from Engineering Control Theory, Systems Theory, Organizational Behaviour, etc. Using this framework SD exists as the intersection of several disciplines. The risk is that we could be accused of having no “intellectual centre”. This approach runs the risk of concluding that SD is not even a field, “but rather an intellectual convocation that arose from the confluence of interests among individuals from many fields who continue to pledge allegiance to those fields” [ King,J (1993), “Editorial Notes”, Information Systems Research, 4(4), pp.291-298. ]
2. Definition Based Discipline
A definition based approach to determining what is SD uses words to describe its boundaries. One approach would be to classify the methods and topic that have been studied in the past. There are several limitations to this approach including:
a. it is descriptive, not proscriptive
b. it is static & unresponsive to change
c. classification itself has cultural bias built in
The second approach is to define SD by the functions it provides to its clientele. If we keep to a metasystem framework a simple three component model provides a great deal of explanatory power. We will use this as part of our approach.
3. Evolutionary Based Discipline
According to Research done by Information System this third approach is most suited to a meta-discipline like SD. The evolutionary approach examines the origin of the field and where the field is moving. It points to connections to reference fields and methods that make SD a true meta-discipline. [from http://www.informingscience.com ]
“We need
a culture of criticizing our work against standards we believe in.
so that
learning takes place.”
The SD Skills Inventory
A Template for Self Assessment of SD Mastery
The levels of mastery are based upon
descriptions by Benner [1984] of Dreyfus & Dreyfus’s [1980] five stage
model of skill acquisition. The skills inventory will be presented in four
stages:
1. Dreyfus’ Five Stage Model for Skill Acquisition
2. Competencies for the Five Stages applied to SD
3. Skill Level: descriptors with next steps
4. Skill Inventory Matrix as a Table
Dreyfus’
Five Stage Model for Skill Acquisition
Stage
1: Novice
They have no experience of SD. They must be taught context-free rules to guide actions, which is extremely limited and inflexible. However following rigid rules makes truly successful model building impossible, as the rules cannot tell anybody the most relevant task in an actual situation. Thus, the learner but be placed in situations that require flexible application of knowledge and skills learned as much as is possible.
Stage
2: Advanced Beginner
They have demonstrated marginally acceptable performance, have coped with some real situations so that they realize that a particular method or knowledge application is situation dependent. The advanced beginner should begin to formulate guiding principles [guidelines] that dictate actions based upon experience. However, they are weak at this, so need support in a controlled setting by a more advanced SD practitioner to ensure they do not get in over their head.
Stage
3: Competent
They have been practicing SD for several years and thus have experience to see their actions in terms of long-term impacts and are becoming conscious of the strengths and weaknesses. The competent SD modeller lacks the speed and flexibility of the proficient SD modeller but does have a feeling of mastery and the ability to cope with contingencies coming from the unexpected. There is a conscious, deliberate planning required to achieve efficiency and organized work.
Stage
4: Practitioner
They perceive situations as wholes rather than as unique situations full of details to be understand alone and are not longer dominated by guidelines. Perception is the key word: the perspective upon the problem is not thought out but “presents itself” based upon experience. They do this by seeing the meaning of a siutation in terms of its long term implications. They learn who plans need to be modified and thus quickly realize when a normal problem situation has become abnormal. The practitioner still uses guiding principles as guides, but a deep understanding of the situation is required before a maxim is used. Maxims reflect what would appear to the competent as unintelligible nuances of the situation as they mean difference things at different times. However, only one with a deep understanding of the situation can allow a maxim to provide direction as to what needs to be taken into consideration for a particular situation.
Stage
5: Expert
They no longer rely on any simple analytic principle [rule, guideline, maxim] as they can connect their understanding of the situation directly – almost intuitively – to the appropriate action. They have an intuitive grasp of each situation so that they can zero in on the key region of the problem without wasteful consideration for a large range of alternatives. Transferring and making explicit this insight is difficult as the expert operates from a deep understanding of each unique situation. Their challenge is to communicate this insight to others.
SD Competencies
What are the various skills I need?
The Systems Worldview
Are feedback, delays & complexity an
unconscious part of how I understand an issue?
Do I have knowledge of System Thinking values, terminology and tools?
|
Novice |
- interprets problems as a sequence of unconnected events with simple solutions and cause- effect relationships as linear |
|
Advanced Beginner |
- understands and
explains events as expressions of a System behaviour and identify causes of a
problem due to flaws in a System structure - has gained the
requisite skills and understanding of social psychology. - is aware that
complex dynamic behaviour of systems is governed by feedback & delay - understands dynamic complexity and the subjective nature of model building by including qualitative variables in models and working subject experts intimately |
|
Competent |
can describe a complex dynamic problem so that the relationships and the model boundary is clear |
|
Proficient |
- can integrate SD with complexity theory
concepts eg. Agent based modeling. - is aware of when to use the SD approach and
when NOT to use it - rather than viewing models as accurate descriptions of reality he/she knows that a model is only as useful as it is able to act as a catalyst for organizational change and changing of mental models |
|
Expert |
- places as great a
value on human perception of a problem as the quantitative data - can integrate the soft Systems Thinking & Hard Modeling Skills into one seamless whole |
Computer Modeling Skills
How complex a model can I build?
Can I use modeling
software and then use the model to help understand how & when to intervene
in a System?
|
Skill Level |
Expectations |
|
Novice |
- sees the need for
a quantitative policy methodology to deal with a complex, dynamic problem |
|
Advanced Beginner |
- can draw a
behaviour over time graph and simple causal loop diagram - can properly use
& relate stocks, flows, variables and connectors. - is able to build a
simple one or two stock model - is not aware of
the scope of SD in terms of organizational change, policy and simulations as
a rapid prototyping tool |
|
Competent |
- is able to take a
problem from a field that they are familiar with and translate it into an SD
map and model that shows the expected behaviour over time changes of the
situation - is able to one
software tool & all its functions [eg lookup or table functions, can
choose the appropriate delta & integration technique, can do causal
tracing of a model] |
|
Proficient |
- is able to focus
the modeling project to obtain maximum leverage from time invested eg. Can choose appropriate level of complexity for the intended
audience -. can use ALL the
power a SD modeling package |
|
Expert |
- can grasp the
essentials of a complex dynamic problem & model it |
Applications
Can I apply SD to
various fields?
Do I have the ability
to transfer concepts across diverse settings eg. Project Management
|
Skill Level |
Expectations |
|
Novice |
- does not have the
experience and knowledge gained from application to use a Systems model to
solve a real world problem |
|
Advanced Beginner |
- has gained the
requisite mathematical skills and understanding of calculus, social psychology, experience with
communicating proposals & implementing change - some experience
and limited skills in modeling but not able to be sure that the model is
accurate or useful for the given situation |
|
Competent |
- can use a SD model
or Systems perspective to add value to a project in another field |
|
Proficient |
-
has learned to
include soft variables such as attractiveness multipliers to allow an SD
model to investigate fuzzy human behaviours into a modeling process - applying SD to fields which have problems exhibiting dynamic complexity eg. Project Management, risk analysis, public policy -
working WITH others, preferably
subject experts from the problem domain the SD method is being applied to |
|
Expert |
- is able to consistently build models with groups where the model recommendations are implemented even if goes against previous policy - integrates use of SD software with other software
packages eg. Spread sheets, discrete event simulations methods, project
management, performance measurement and enterprise management software - can apply the SD method to any project
requiring an understanding of dynamics complexity |
Communication & Leadership
Can I convince others of my possibly counter intuitive
recommendations?
Do I have the ability
to convey to & convince others of my insights –eg. extract variables–
facilitate communication among those involved in the project.
|
Skill Level |
Expectations |
|
Novice |
- knows the basic
rules for group model building & follows them strictly - the ability to
discuss a problem from a System Thinking worldview |
|
Advanced Beginner |
- experience with
communicating proposals & implementing change in organizations |
|
Competent |
- has internalized
experiences of group model building and facilitation so that internal
guidelines are used for each new project |
|
Proficient |
- can involve others in a double loop
learning process to question their mental models and move teams to make
counter intuitive decisions - is able to convince others of their
insights - can explain the
advantages and appropriateness of the SD approach to others convincingly |
|
Expert |
- can communicate their
insights so anybody else can understand and appreciate the implications of
the Systems Dynamics approach - can convince others of the relevance of the method & can train
others in its application |
Skill Acquisition
Novice
Descriptor
Sees the need for a quantitative
policy methodology to deal with a complex, dynamic problem; however is stuck in
a simple linear thinking view of cause and effect. Curious.
Limitations
A. may not have the mathematical background
[Calculus] to understand stock and flow concepts
B. may have been so well trained in the Scientific
method that shifting from a simple linear cause effect view of solutions to a
seeing problems as complex & dynamic and full of delays and feedback is
very difficult.
Next Steps
Building a knowledge of the basic facts & SD ideas. Read
some classic SD texts, self study using on-line material like the Roadmaps
course from MIT, take a formal course from a University or one of the SD
software companies, join a local SD Chapter if possible, attend a System
Dynamics or Systems Thinking Conference.
Advanced Beginner
Descriptor
Knows when SD is the right
method for the job and is able to view problems as non linear complex
systems with feedback, delays and causal flow.
Limitations
A. unaware of how long
and difficult it will be to become a professional SD modeler,
B. still thinks of
models as slices of reality that can predict the future rather than as
learning tools that
help organizations develop robust policies that can allow the organization
withstand the
unexpected.
C. Most of my learning is being
alone.
Next Steps
A. take a formal course in SD and/or work with
a SD practitioner on projects
B. complete the year long MIT distance course or
equivalent. Read the seminal works in
the field [see Appendix A] and see how SD can be applied as a methodology to a
wide variety of problems.
Competent
Descriptor
Has mastered basic tools and adept at
drawing causal loop diagrams on the spot and able to build stock and flow
models with 3 to 10 stocks that are dimensionally consistent and have feedback
loops that describe real behaviour.
Limitations
A. cannot do
optimization of parameters, sensitivity testing, partial simulation by sectors,
B. cannot identify
policy levers and/or find variable with highest leverage. Not able to convince
a non-SD audience of the value of their insights.
Next Steps
A. attend World Conferences and other
conferences to see what experts in the Field are doing
B. learn about other fields where they can
apply the SD method & practice applying the SD method to fields
that you are not a subject expert
C. learn to use more than one of the SD modeling
software packages.
D. Find a mentor to learn from and work with. The SD professional now works with an experienced SD modeller in
fine tuning their skills – in particular in making their insights
comprehensible & relevant to
decision makers so the recommendation are IMPLEMENTED.
Proficient
Descriptor
Has applied
SD to real problems for several year. [under the guidance of a mentor]
Limitations
A. not expert in BOTH model
building & explaining/convincing/training an audience/client of the value
of insights gained from the SD worldview and model.
B. does not have all
the skills needed to carry out a total SD project alone and thus needs to work
in a team of at least 2 or 3
Next Steps
A. Continuously apply skills to new areas and work
with other people so that he/she do not become just a technical expert
B. Learn how to include human realities in a model and have non technical audience not feel threatened, over-whelmed or otherwise incapable of appreciating the insights of a System Analysis.
Expert
Descriptor
SD
modeller and trainer/teacher/consultant for many years who has earned
the respect of people inside and outside the field.
Limitations
Almost none; yet is constantly learning more by making SD relevant to others.
Next Steps
Share their skills and knowledge so it is not lost to the next generation.
The
Skills Inventory Matrix
A self taught practitioner can “see” how they are doing by tracking their advance through the Inventory Matrix by simply blocking in the appropriate skill level for each dimension. Note that the developing SD modeller should be very conscious when they believe they have moved from being a competent to proficient SD modeller as indicated by “The Chasm”.
Skill Level
|
Dimension |
Novice |
Advanced Beginner |
Competent |
The Chasm |
Proficient |
Expert |
|
Systems Worldview |
|
|
|
|
|
|
|
Computer Modeling Skills |
|
|
|
|
|
|
|
Applications |
|
|
|
|
|
|
|
Communication and Leadership |
|
|
|
|
|
|
Conclusion
It is author’s hope that this matrix will begin the process by which the SDS can support practitioners of SD in improving the quality of their work and allowing organization asking for SD work to know what to expect from a SD expert. The concept of SD as a trans-discipline is a key to understanding why it is difficult to define the field in the conventional sense of the word. This report is meant as a first step towards the SDS creating a pathway by which self taught practitioners can improve their skills and be clear about their current skill level so that they attempt projects commensurate to their abilities. The five skill levels framework and four competencies descriptions are the start of an expectations-based view of the SD practitioner that, it is hoped, will also aid the Curriculum committee in their efforts.
Finally, it is interesting to note that if
field of System Dynamics is to become clearly recognised as a unique discipline
it will have to begin taking on some of the characteristics of a Profession
listed below. [from the Professional
Engineers of Ontario]
1. A Code of Professional Ethics
2. Minimum academic credentials
3. Minimum professional experience terms/internships
4. Implement binding practice rules to which all licensed practitioners must adhere
5. Implement practice inspections for Consultants to ensure quality
6. Offer improved access to continuing education and training activities that will be valued by employers.
Implement clear definitions of sub-specialties within the profession in order to ensure that the work is certified only by practitioners qualified in that specific discipline.
Together we can move System Dynamics forward so that it becomes the methodology that helps managers make better Policy decisions when an understanding of dynamic complexity is required. As Jack Pugh put is well in Atlanta, 2001:
Let us create a
curriculum that SD practitioners can examine & argue about.
References
Benner, P. 1984 From Novice to Expert Addison-Wesley, CA.
Dreyfus,H. & Dreyfus,S. Feb.1980. A Five Stage model of the mental activities involved in directed skill acquisition. Unpublished report by the Air Force Office of Scientific Research [AFSC], USAF [contract F49620-79-0063], U. of California, Berkley.
Kim, Daniel, 1989 Curriculum Skills and competencies
King,J (1993), “Editorial Notes”, Information Systems Research, 4(4), pp.291-298.
I.Martinez & G. Richardson, Best Practices in System Dynamics Modeling, Atlanta, 2001
Professional
Engineers of Ontario www.peo.on.ca