WED 11:30 AM Parallel Session: Methodology Issues
Full Report: Modelling as Theory Building by Markus Schwaninger and Stefan Groesser; Presenter: Stefan Groesser
In the first presentation, Stefan Groesser discussed how system dynamics has been claimed to have a bright potential in theory building, which unfortunately has been little developed in the literature. Stefan started by presenting the criteria for evaluation of theory-building: refutability, importance, precision and clarity, parsimony, comprehensiveness, operationality, validity, reliability, fruitfulness, and practicality. Along with the example of a product launch model, which was built in cooperation with a Swiss industrial company, Stefan referred specifically to how some of the criteria for theory-building were evaluated in practice. The purpose of the model developed was to support the company’s management of product launches and more specifically to understand which factors influence the product-launching decision. Validation checks of the model were reported, which checked the fit of the model to the existing theory, which helped in gaining a better understanding of the model. Furthermore, Stefan referred to validity checks of model outputs against historical real world company data, which resulted in the explanation of eighty0five percent of the behaviour of the historical times series. As part of the discussion, an evaluation of the theory developed was provided as rated by two gatekeepers and the authors. An interesting aspect presented was the validation of the model according to the Validation Cube with three dimensions, including the level of validation, layers of reflection, and degrees of resolution. Stefan concluded his presentation that the model created applies to a large range of cases and is therefore close to a middle-range theory. This presentation gave a positive message that system dynamics modelling can be used for theory building.
The talk attracted the interest of the audience and some questions followed. When asked whether the model outputs were validated using a different set of data, Stefan replied that the data were validated using data from another case. Another attendee commented that the use of system dynamics for theory-building is especially promising for analyst theories.Explaining women’s careers at a Dutch university: Model building as a method for knowledge elicitation in gender analysis by Inge Bleijenbergh, Eelke Blonk, Lodewijk Schulte and Marloes van Engen; Presenter: Inge Bleijenbergh, Roudbound University of Nijmegen
In her presentation, Inge Bleijenbergh presented a practically oriented study on the careers of women academics in Netherlands. Inge started her presentation by pointing out that even though there are many female students in Dutch Universities, only ten point eight percent of women academics are full professors. Several explanations for the under representation of women in academia have been provided in the literature, but the presenter explained that these do not provide a full explanation of the situation. Therefore, further knowledge needs to be developed on the role of organizational culture and gender stereotyping. In understanding the issues involved system dynamics model-building is used as a tool for knowledge elicitation. The aim of the study was to report back to the executive board of the University, which was the decision maker. Group model-building was involved in this study in which all five authors participated in model-building and contributed different knowledge regarding the problem. The data used in the model resulted from a preceding qualitative study, involving forty-three semi-structured interviews (with decision makers as well as academics at different levels of their career) and nine focus groups. The five researchers created the system dynamics models as part of three modelling sessions led by a system dynamics facilitator. The results of the qualitative analysis found a mismatch between actual and perceived behaviour of female academics. There was indeed very small difference in the working times between men and women academics. It was interestingly found that parents (fathers or mothers) as opposed to women academics worked fewer hours. Then Inge presented the qualitative system dynamics model created, focusing on the four major feedback loops: the masculinity of norms, the productivity targets, visibility of women, and that of the networks and women. Then the theoretical conclusions of the study were highlighted. The system dynamics model-building was reported to have helped immensely in understanding the lower numbers of women in higher academic positions. From the qualitative model a mismatch was found between actual and perceived female academics, which highlighted the existence of gender stereotypes appearing in the model. From a methodological point of view, the study showed that system dynamics modelling supports knowledge sharing. The presentation ended with a positive message about gender stereotyping (see picture).
One of the questions raised by the audience was whether the authors had thought about quantifying the qualitative model developed. Inge replied that the authors were interested in creating a quantitative model. However difficulties were encountered in finding historical data from previous years.
Very Large System Dynamics Models - Lessons Learned, by Jacob J. Jacobson, Leonard Mayczynski, and Vincent Tidwell; Presenters: Jacob J. Jacobson and Leonard Mayczynski
Jacob J. Jacobson started the presentation talking about the lesson learned from developing several large system dynamics models, with the view to share experiences with other modellers. In the overview, Jake talked about 2 examples of large models the authors had recently worked on. One of the models is the Verifiable Fuel Cycle Simulation (VISION), built for two United States Department of Energy National Laboratories. The aim of this model was to analyze and compare the various proposed technology deployment scenarios with the use of a quick and transparent model. Jacob interestingly pointed out that it takes about twenty years to get a fuel certified in the United States. He continued with some statistics of the VISION model (forty-two modules and forty-two thousand variables!). Afterwards, Leonard Mayczynski took over to describe the next model, the MRG model, part of a cooperative water planning project for Rio Grande. The model represented a dynamic water budget, including two demand and supply components, including thirteen modules and eight hundred-ninety-nine variables. The lessons learned were presented successively by the presenters under the headings project management, modelling process, managing complexities, validation, and results. With respect to project management skills, it was mentioned that it was found valuable to have a plan at the beginning of the project to ensure clarity of what needs to be done, when, who does what, etc. With respect to modelling, the presenters pointed out the importance ofcausal loop diagrams, allowing time to develop a neat interface, evaluating the benefits of including detail complexity against the benefits gained, and showing significant results to the client early in the modelling process.
Furthermore, Jake and Len highlighted some useful techniques for managing complexity, such as the use of colours, modules, arrays, external data from Excel, etc. With respect to validation and verification of large models, some of the lessons learned were related to the importance of checking the final product given to the client. It was pointed out that a faulty model can irreparably damage the clients’ confidence. Furthermore, the need to validate the model at every stage was highlighted.
Jake and Len pointed out how the models have run for various years, supporting student projects as well as supporting negotiations and strategic projects. VISION was used to compile the Annual Report to the Congress. The MRG model was used successfully in the Middle East to support countries with shortages in water supplies. In summary, the authors drew attention to the fact that modelling large models is difficult and that it requires money and time. It is therefore necessary to educate the client. Two are the success measures of a study: does the model generate insights and is the application used after the model is developed? The presentation concludes with the statement that real success is the customer’s acceptance.
During questions, validation issues were discussed. In addition, the lack of incentive to publish in the private sector was raised, while it was appreciated that the insights developed would be beneficial for the system dynamics community. The presenters replied that due to the nature of client-oriented projects and funding limitations, there are no high incentives for publishing. However they admitted that an attempt is being made to develop practitioners’ guides.
The session was attended by twenty-five to thirty delegates.
Antuela A. Tako