LOGIN

Your E-mail
Password  
Remember me
 Forgotten password
SCALETOOL IntroductionDriversBiodiversityPolicies and managementConnectivity and protected areas

Best Practice for model application

In the following you will find a collection of guidelines for best practice during model application. Topics covered are:

UNCERTAINTY, STOCHASTICITY, AND PARAMETER ESTIMATION


UNCERTAINTY, STOCHASTICITY, AND PARAMETER ESTIMATION


What sources of uncertainty are identified?
  • poor quality or low quantity of data (e.g. spatiotemporal sources of variance, like the existence and form of density dependence or the degree of spatial autocorrelation among local population dynamics, are difficult to discern without detailed, often long-term data)
  • difficulties in parameter estimation
    • Does parameter estimation address and capture the limitations of the field data?
    • Does parameter estimation translate uncertainty into parameter ranges,variances,or statistical distributions?
    • Is it possible to distinguish and quantify different sources of variability and potential biases?
      • sampling variability and errors result in poor estimates of population parameters,
      • process variability (e.g. variance in vital rates) may reflect true attributes of population dynamics
  • weak ability to validate model
  • effects of alternate model structures
Is there a proper distinction between uncertainty and stochasticity?

stochasticity - natural variation

One should distinguish among environmental, demographic, and genetic sources of stochasticity and address all sources of stochasticity, including catastrophes, in sensitivity analyses (White 2000). Catastrophes and disturbances (e.g., fires or floods) may differ from regular environmental stochasticity (e.g., fluctuations in temperature or rainfall) in amplitude, in the processes involved, and the nature of their effects on different life stages, individuals, populations, and the recovery of the environment itself (White 2000; Morris & Doak 2002).

uncertainty - lack of knowledge (epistemic uncertainty)

Uncertainty emerges from various sources, stochasticity being only one of them. Not all sources of uncertainty can be handled by having more or better data.

Important sources of uncertainty to consider and discuss are:
  • poor quality or low quantity of data
  • difficulties in parameter estimation and its translation into the model (see below)
  • weak ability to validate a model
  • effects of alternate model structures

Parameter estimation

Difficulties in parameter estimation are broadly expressed and discussed in the literature.

Important questions to ask when applying a PVA:
  • Does parameter estimation address and capture the limitations of the field data?
  • Does parameter estimation translate uncertainty into parameter ranges, variances,or statistical distributions?
  • Is it possible to distinguish and quantify different sources of variability and potential biases? These include sampling variability and errors, or, process variability which may reflect true attributes of population dynamics


DECISIONS CONCERNING VERIFICATION, VALIDATION AND CALIBRATION

At the onset of the model design and implementation, consider which data are available for calibration, parameterization, and verification. You might need to divide your data in advance or think what data may become available in the future.

Verificaton

Model verification is absolutely necessary: test that the model works according to its specifications, with no omissions and errors.
You may consider using planned simulation experiments, implementation in another software, or independent check by another modeller (see also Schmolke et al 2010).

Validation

Model validation is highly desired, but independent data are often absent.
You may thus wish to validate your model against a) predictions in the field, b) through the use of newly accumulated data, or c) monitoring of management outcomes.
Thinking in advance about the options may affect model application!

Calibration

Model calibration is desired especially when a model needs to match real observations or data, but can be challenging if data are scarce or unavailable for some parameters.

Important decisions to make:
  • choice of the parameters to be calibrated, and the ranges used
  • choice of the optimization method
Useful references on calibration: Schmolke et al. 2010, Grimm et al. 2005.


SENSITIVITY ANALYSES

Almost any model would require sensitivity analyses. You can perform local sensitivity analysis (per parameter) or, preferably, global sensitivity analyses.

Important issues to consider:

  • Choice of the model parameters to be addressed by a sensitivity analysis (see overview below).
  • Range of the parameter values to test;
  • Viability measure(s) to use as a response in sensitivity analysis (see table below).

Important questions to be answered when performing a sensitivity analysis

  • Are the parameters that have a strong effect on viability assessment identified?
  • Do dependencies between model parameters emerge from artefacts in model structure or a reflection of important ecological processes?
  • Are the parameter sensitivities analyzed individually? If yes, are there any synergistic effects that may prevent sensible interpretation of model sensitivity?
  • Is the importance of interactions among model parameters assessed?
  • Is the importance of nonlinearities in model response to parameter variation assessed?
For more details concerning sensitivity analysis see Saltelli & Annoni 2010, Saltelli et al 2006.

Potentially important parameters for sensitivity analysis

The following six parameters were found to have the highest importance in our analyses (from high to low):
  • Parameters concerning mortality
  • Parameters reflecting environmental stochasticity
  • Parameters concerning catastrophes
  • Parameters describing landscape management
  • Number of patches used in the model
  • Connectivity
We recommend concentrating on parameters concerning catastrophes, number of patches, and density dependence. For these parameters, we found a mismatch between their strength of effect and their inclusion in sensitivity analyses.


DECISIONS CONCERNING SIMULATION DURATION AND TIME HORIZON

Several viability measures for PVA are defined with respect to a given time horizon. In order to identify an appropriate time horizon (or several alternative ones), and the related simulation duration we suggest addressing the following questions:

Simulation duration should be long enough to:

  • identify long-term population trends
  • avoid transient dynamics due to initial conditions
  • distinguish among outcomes of alternative management scenarios
And short enough to:
  • limit the propagation of uncertainty over time
  • be relevant to specific, often pressing, conservation decisions

Irrespective of the chosen simulation duration, we suggest including a range of time horizons when reporting PVA results. See also: Choice of viability measures (below).


CHOICE OF VIABILITY MEASURES

The selection of a good viability measure, among the many available, is highly relevant for good communication. Yet it can also affect decisions on model application (e.g. simulation duration).

We strongly encourage selecting several viability measures, as they can complement each other.

Viability measure Meaning Calculation Recommendations for PVA
P0(t) Probability of extinction by time horizon t Count extinction events over multiple simulations versus the time at which they occur and plot their cumulative distribution over time. Report P0(t) for several time horizons; for consistency with international listing thresholds and to facilitate comparison across studies, report P0(100) as one of these time horizons.
PN Quasi-extinction risk Plot the minimum population size N observed during the course of each simulation iteration, against their cumulative distribution. Can be used when global extinction is not possible (Burgman et al. 1993); to advance comparability report outputs for multiple values of N, including N = 0 if possible, for comparison with P0(t).
Tm Intrinsic mean time to extinction Plot ln(1 - P0(t)) versus time t. The plot yields a straight line with slope 1/Tm (Grimm & Wissel 2004). Use Tm to enable approximating P0(t) for any time horizon based on P0(t) ≈ t/Tm; it is insensitive to initial simulation conditions, and may reveal generic information about extinction risk and viability.
EMP Expected minimum population size Record the smallest population size obtained in each simulation iteration. Rarely reported in PVA studies; a simple and effective measure which should be more frequently used especially for sensitivity analyses and when the risk of extinction is small (McCarthy & Thompson 2001).
Ne Expected population size Plot Ne over time to provide a simple and intuitive visualization of population behavior and comparison between scenarios. An important "currency" for decision makers, but the tail of distribution must be depicted to account for the range of potential outcomes; should be considered in conjunction with other measures of risk.
λ Mean intrinsic growth rate of a population Provides a simple measure of the potential for population growth. Useful for differentiating alternative population trends (Caswell 2002). However, recent analyses (Pe'er et al. unpublished) indicate risks in using lambda as a true representative of risks. Hence, it should be reported in conjunction with viability measures that provide a better measure of risk.
MVP Minimum Viable Population Run simulations with a range of initial population sizes to define the lowest threshold that maintains a viable population (i.e., predefined probability of survival over a given time horizon). Strictly speaking, not a viability measure but a measure of what would be required to achieve viability. Often relevant for policy decisions; provides intuitive information for communication; however oversimplifications may yield misinterpretation, therefore interpret and communicate carefully.
MAR Minimum Area Requirement Run simulations with a range of initial area (or other spatial attributes) to identify the area necessary to support a viable population. Same comment as MVP above. For further information see Pe'er et al. 2014.


PVA TO SUPPORT DECISION MAKING

For a PVA to be accepted by conservation managers, one should consider and rank multiple management options. When doing so, we recommend considering not only ecological benefits but also potential costs.

Questions to consider for making a PVA relevant for decision makers

  • Are various management options considered and ranked?
  • Are the considered management options realistic and relevant?
  • Are trade-offs between multiple management options considered and quantified?
  • Are the costs and benefits of multiple management options identified?
  • Are the most salient uncertainties characterized?
  • Are criteria for evaluating differences among management outcomes identified?

For further advice on performance criteria for ranking management options, see:
Lindenmayer & Possingham 1996; Beissinger & Westphal 1998; McCarthy et al. 2003; Bakker & Doak 2009.

For cases where alternative options have been quantified, see:
Curtis & Vincent 2008; Johst et al. 2011; Wintle et al. 2011.

References

Bakker VJ, Doak DF (2009) Population viability management: ecological standards to guide adaptive management for rare species. Frontiers in Ecology and the Environment 7: 158-165. doi:10.1890/070220.

Beissinger SR, Westphal MI (1998) On the use of demographic models of population viability in endangered species management. Journal of Wildlife Management 62: 821-841.

Caswell H (2002) Matrix population models: construction, analysis and interpretation. Sinauer Associates, Sunderland, Massachusetts, USA.

Curtis JMR, Vincent ACJ (2008) Use of population viability analysis to evaluate CITES trade-management options for threatened marine fishes. Conservation Biology 22: 1225-1232. doi:10.1111/j.1523-1739.2008.00994.x.

Grimm V, Revilla E, Berger U, Jeltsch F, Mooij WM, Railsback SF, Thulke H-H, Weiner J, Wiegand T, DeAngelis DL (2005) Pattern-oriented modeling of agent-based complex systems: lessons from ecology. Science 310: 987-991.

Grimm V, Wissel C (2004) The intrinsic mean time to extinction: a unifying approach to analysing persistence and viability of populations. Oikos 105: 501-511.

Johst K, Drechsler M, van Teeffelen AJA, Hartig F, Vos CC, Wissel S, Wätzold F, Opdam P (2011) Biodiversity conservation in dynamic landscapes: trade-offs between number, connectivity and turnover of habitat patches. Journal of Applied Ecology 48: 1227-1235

Lindenmayer DB, Possingham HP (1996) Ranking conservation and timber management options for leadbeater's possum in southeastern Australia using population viability analysis. Conservation Biology 10: 235-251. doi:10.1046/j.1523-1739.1996.10010235.x.

McCarthy MA, Andelman SJ, Possingham HP (2003) Reliability of relative predictions in population viability analysis. Conservation Biology 17: 982-989. doi:10.1046/j.1523-1739.2003.01570.x.

McCarthy MA, Thompson C (2001) Expected minimum population size as a measure of threat. Animal Conservation 4: 351-355.

Morris WF, Doak D (2002) Quantitative conservation biology. Theory and practice of population viability analysis. Sinauer, Sunderland.

Pe'er G, Tsianou MA, Franz KW, Matsinos GY, Mazaris AD, Storch D, Kopsova L, Verboom J, Baguette M, Stevens VM, Henle K (2014) Toward better application of Minimum Area Requirements in conservation planning. Biological Conservation 170: 92-102. doi:http://dx.doi.org/10.1016/j.biocon.2013.12.011.

Saltelli A, Ratto M, Tarantola S, Campolongo F, European C, Joint Res Ctr I (2006) Sensitivity analysis practices: Strategies for model-based inference. Reliability Engineering & System Safety 91: 1109-1125. doi:10.1016/j.ress.2005.11.014.

Saltelli A, Annoni P (2010) How to avoid a perfunctory sensitivity analysis. Environmental Modelling & Software 25: 1508-1517.

Schmolke A, Thorbek P, DeAngelis DL, Grimm V (2010) Ecological models supporting environmental decision making: a strategy for the future. Trends in Ecology & Evolution 25: 479-486.

White GC (2000) Chapter 9: Population viability analysis: Data requirements and essential analysis. In: Boitani L, Fuller TK (Eds) Research Techniques in Animal Ecology. Columbia University Press, New York, USA, 288-331.

Wintle BA, Bekessy SA, Keith DA, van Wilgen BW, Cabeza M, Schröder B, Carvalho SB, Falcucci A, Maiorano L, Regan TJ, Rondinini C, Boitani L, Possingham HP (2011) Ecological-economic optimization of biodiversity conservation under climate change. Nature Climate Change 1: 355-359. doi:10.1038/nclimate1227.

Copyright and disclaimer: SCALES and SCALETOOL

CONDITIONS OF USE: We explicitly encourage the use of SCALETOOL. SCALETOOL is freely available for non-commercial use provided you acknowledge SCALES as source. For more extensive access to databases (e.g. for statistical analyses or if you want to contribute data), tools, or background material, please contact the SCALES coordinator (send us email).

All rights reserved. USE THIS SOFTWARE AT YOUR OWN RISK. THE SCALES TEAM WILL NOT BE LIABLE FOR ANY DIRECT OR INDIRECT DAMAGE OR LOSS CAUSED BY THE USE OR THE INABILITY TO USE THIS SOFTWARE.

© 2010 - 2018 SCALES. All rights reserved.