Ten Heuristics for Modeling Projects

Home Browse Search

382 C. R. Nicolson and others and differences between alternative conjectures about the unknowns. The synthesis modelers should be encouraged and empowered to use their skills to help resolve the tensions between simplic- ity and detail that are inherent to any modeling project (Costanza and Sklar 1985; Starfield and Ble- loch 1991). Heuristic 8. Sensitivity analysis is vital at all stages of the modeling effort. Thorough sensitivity analysis involves testing not only different parameter values but also the assumptions and the effect of alterna- tive educated guesses at the underlying processes (see, for example, Starfield and others 1995; Star- field and Bleloch 1991). Sensitivity analysis is the only available means of determining what goes into the model and what level of detail is necessary. It is an essential tool for estimating the likely effects of alternative hypotheses for system processes. Sensi- tivity analysis should not simply be thought of as an automated process that tests all parameters, but rather an important part of the culture of modeling that is used for the thoughtful exploration of alter- native assumptions. It follows that the work of sen- sitivity analysis should be done by most (ideally, all) of the project team, not just the modeler. Because each person on the team brings a different perspec- tive to the problem, he or she is thus likely to run different eJCperiments and uncover different prob- lems. In fact, team efforts are essential both for identifying implicit assumptions (social norms do not change during two generations, for example) and for developing plausible alternative scenarios as part of the sensitivity analysis. Sensitivity tests are essentially mini-experiments. To be effective in shaping the prototype modeling process, the models supporting the mini-experi- ments need to run virtually in real time. Waiting days, weeks, or months for model results is too long. We found that the ability to work as a group to set up a model simulation, and then view the results within a minute or two, was principally responsible for most advances in developing model relationships that crossed disciplinary boundaries. Heurisric 9. Work hard at communicarion and budget for face-to face meetings. Effective communication lies at the heart of interdisciplinary research. Not only is it necessary for scientists to engage with one an- other to produce an integrated view of a system, their findings must also be explained clearly to stakeholders and to the public. In the Intemet era, communication can take many forms, including list-server memos, e-mails, phone calls, small face- to-face work groups, plenary team meetings, and public meetings. Each communication medium serves a different purpose, and it is dangerous to assume that simply because we have these tools at our disposal, people from different disciplinary backgrounds will automatically communicate effec- tively with each other. In the SAC Project, scientists often appeared to have reached a point of under- standing in their discussions, only to find out later that in fact they had two rather different things in mind (sometimes as the result of using the same words but meaning different things by them). Team members need to make an effort to become more familiar with each other's mental frameworks and to be cognizant of what specific people mean when they use certain words or concepts. This is one reason why rapid prototyping is so valuable. It leads quickly to a product that provides a common lan- guage and enables participants to say "No, that's not really what I have in mind." One way to foster better communication is by developing simulation models in easffy accessible modeling environments, such as spreadsheets. The goal is to work continually toward a culture of transparent and accessible models, so as to ensure that the models are understandable to everyone on the team. In this regard, we have found that spread- sheets have several advantages over traditional pro- gramming languages such as FORTRAN, BASIC, or C++. Most scientists are familiar with the spread- sheet environment and its basic concepts. Also, spreadsheets allow us to quickly and easily develop straw models as part of the dialogue among the participants, so we can constantly point to some- thing tangible and ask, "Is this what you mean?" The built-in graphing functions of spreadsheets en- able graphical model output with very little pro- gramming effort. Finally, because spreadsheets per- form calculations each time a cell is changed, they are powerful tools for sensitivity analysis. In addition to communication within the team, a second area requiring careful consideration is how the integrated work of the team will be communi- cated to the stakeholders and the public, a task that is vastly underrepresented in many scientific projects. Interdisciplinary research that affects peo- ple's lives directly must be explained to them in accessible language, stripped of its technical scien- tific terminology. The results need to be put into everyday terms, and it is crucial to spell out both the practical implications of the findings and the areas of uncertainty. Funding agencies need to be willing to support the outreach and extension part of in- terdisciplinary research, and scientists need the help of communication specialists to get their results into public discourse in a form that can be digested and discussed. The SAC Project's efforts at outreach in- clude the development of a simplified interactive