Ten Heuristics for Modeling Projects

Home Browse Search

380 C. R. Nicolson and others herd size. This suggests that the initial emphasis on herd population dynamics may have been some- what misplaced. In both of these examples, our problem definition led us to believe that certain factors were more important than they tumed out to be. It is also important to make sure that the linkages among different parts of the system are strong and that the system behavior is not dominated by a single component (in which case the problem does not necessarily call for an interdisciplinary ap- proach). Our experience, and that of other practi- tioners (for example, Holling 1978; Walters 1986), has shown that it is more fruitful to begin with the system itself and to look outward to the compo- nents rather than to look piecemeal at the system from within the perspective of the individual com- ponents. One danger in interdisciplinary modeling work is that people who are not fluent in systems modeling may not engage properly with the task. The solu- tion is not to recruit only model-oriented scientists (which would limit the scope and breadth of the synthesis), but rather to work at drawing nonmod- elers into the process. Prototype models that are simple enough to demonstrate and explain to all team members are an essential step in the educa- tion of nonmodelers. Heurisric 4. Allow the project s focus to evolve by not allocating all funds up front. This is a luxury seldom available to research scientists, given the current policy of multiyear, multi-investigator projects. However, one of the inherent difficulties with in- terdisciplinary research is that defining the problem often represents a major part of the project. Thus, a chicken-and-egg situation arises. The problem can- not be defined until a working team is in place, but it is impossible to know how deeply to involve specific team members until the problem has been defined. Even when the problem is apparently well defined, it is extremely hard to assess a priori which components determine the system dynamics most strongly until a first prototype of the synthesis work has been constructed. It is likely that the relative importance of the various components will only emerge during the study. We have already alluded to the initial hypothesis of climate change --?- vege- tation change - caribou herd dynamics --* caribou availability to human communities. By the time we discovered that this apparently central hypothesis was not a main driver of change, the project's funds had been allocated and could not easily be shifted to address newly evolving hypotheses. It might be better if funding agencies awarded preliminary planning funds (say, for the lst year or through the development of a first prototype model) and then funded the remainder of the project only when it was demonstrated that the conect mix of scientists was working together ef- fectively and attacking a well-defined problem. If all the funds are committed up front for the full dura- tion of the study, the project leadership has no flexibility to add new people as their expertise be- comes necessary or to reallocate funds from a com- ponent of the work that offers little to the inte- grated effort. Heurisric S. Ban all models or model components that are inscrutable. An "inscrutable" model is a black box in which the inner workings are inaccessible to all but the original developers. The user is required to take the output on faith. The problem with in- scrutable models is that people have no incentive to engage with them intellectually. If the model pro- duces any counterintuitive results, people cannot access the logic that led to those results. It is not surprising then that their usual reaction is to lose trust in the model rather than ask about the inter- mediate relationships that led to those final results. In the SAC Project, a complex model of caribou energetics (Hovey and others 1989; Kremsater 1991; Daniel 1993) was initially thought to be es- sential at the interface between vegetation change and caribou population dynamics. We realized later that what we really needed were models of herd distribution and movement, but a commitment had already been made to this energetics model. Until we developed a much simpler caribou population model, the project depended on output from a black box model that only a few people understood and used. A top-down, rapid-prototyping approach could have helped avoid this situation. Graphical "box and arrow" representations of the system (JBr- gensen 1986; Walters 1986) combined with the simplest possible component models, programmed using software that is easily accessible to all team members (such as spreadsheets), allow a team of scientists from different disciplinary backgrounds to understand and engage with the key relationships of the model. Heuristic 6. Instead of concentrating on one all-purpose synthesis model, invest in a suite of models, each with a well-defined objective. This heuristic applies particu- larly to the collaborative development stage of a project. It allows participants from a subset of dis- ciplines to engage with models that focus on the interfaces between those subsets. The idea of building a suite of models may seem to go against the very idea of interdisciplinary syn- thesis modeling, but meshing existing submodels together can be a difficult and time-consuming ex- ? X_