^{1}

^{*}

^{2}

^{*}

^{3}

^{*}

Mark-recapture models are extensively used in quantitative population ecology, providing estimates of population vital rates, such as survival, that are difficult to obtain using other methods. Vital rates are commonly modeled as functions of explanatory covariates, adding considerable flexibility to mark-recapture models, but also increasing the subjectivity and complexity of the modeling process. Consequently, model selection and the evaluation of covariate structure remain critical aspects of mark-recapture modeling. The difficulties involved in model selection are compounded in Cormack-Jolly-Seber models because they are composed of separate sub-models for survival and recapture probabilities, which are conceptualized independently even though their parameters are not statistically independent. The construction of models as combinations of sub-models, together with multiple potential covariates, can lead to a large model set. Although desirable, estimation of the parameters of all models may not be feasible. Strategies to search a model space and base inference on a subset of all models exist and enjoy widespread use. However, even though the methods used to search a model space can be expected to influence parameter estimation, the assessment of covariate importance, and therefore the ecological interpretation of the modeling results, the performance of these strategies has received limited investigation. We present a new strategy for searching the space of a candidate set of Cormack-Jolly-Seber models and explore its performance relative to existing strategies using computer simulation. The new strategy provides an improved assessment of the importance of covariates and covariate combinations used to model survival and recapture probabilities, while requiring only a modest increase in the number of models on which inference is based in comparison to existing techniques.

Mark-recapture models are among the most widely utilized classes of models in quantitative population ecology. Historically, mark-recapture models were primarily used to estimate abundance [1,2] and survival rates [3,4]. The diversity and complexity of these models have advanced rapidly in recent years, and modern mark-recapture models are now commonly utilized to estimate a variety of vital rates and parameters related to processes such as immigration, state transition, and geographic transience [5-11]. A key factor in the utility of mark-recapture models is the ability to model natural parameters, such as survival rates, as functions of explanatory covariates [

An unavoidable consequence of using covariates is an increase in the complexity and subjectivity of the modeling process itself. Most mark-recapture experiments are not explicitly designed to test specific hypotheses, but rather generate observational data that may be used to investigate relationships and formulate hypotheses through exploratory modeling [

Several aspects of mark-recapture modeling continue to generate considerable interest in the ecological and biometrics literature, including the formation of a candidate set of models. Many investigators advocate developing a relatively small number of models that embody explicit biological or ecological hypotheses [15-17]. Other investigators have noted potential drawbacks with that approach in certain situations. In particular, inference is conditional on the candidate model set, which can cause significant bias if knowledge about the system under study is limited and well-fitting models are not included in the candidate model set [

The development of a candidate model set is a crucial component of the modeling process that is naturally subject, in part, to the particular objectives of individual investigations. If the primary objective is to test a priori hypotheses, developing a relatively small number of models that embody the hypotheses of interest is obviously sound [15-17]. In such investigations, identifying the covariates that seem most strongly associated with natural parameters, i.e., the survival and recapture probabilities of interest rather than covariate coefficients that are the parameters directly estimated, may be of more interest than estimates of those probabilities. In other investigations, the primary objective may be to estimate natural parameters, perhaps to describe and ultimately forecast population dynamics [

Once a candidate model set is formed, one must implement a strategy to evaluate the models. The most natural and logically unbiased strategy is to evaluate all candidate models. However, even when candidate models are developed with careful consideration of their biological relevance and plausibility [15-17], the number of models in the set can exceed reasonable limits. Large model sets can easily result when models are formed from combinations of sub-models for different classes of natural parameters, such as survival and recapture probabilities. In addition, the existence of animal classes with potentially unique vital rates or the need to model complex patterns of capture effort through time can substantially increase the number of models to be considered. As an example, a mark-recapture investigation of polar bears (Ursus maritimus) in which we were recently involved (unpublished), modeled survival probabilities for four age classes with 24, 10, 1, and 6 sub-models, respectively, totaling 1440 (= 24 × 10 × 1 × 6) sub-models for survival. The number of sub-models considered for recapture probabilities was a more modest 96, constructed to incorporate age-class heterogeneity and a complex pattern of capture effort through time. These sub-models were selected from a much larger collection of potential models identified using prior knowledge, expert opinion, and ecological relevance. The model set of all combinations of survival and recapture probability sub-models was therefore 138,240 (= 1440 × 96). Given such a large model space, which is common in mark-recapture investigations, an effective method of searching the model space without estimating the parameters of every candidate model is obviously advantageous.

Several investigators have developed strategies for evaluating a subset of all mark-recapture models within a Bayesian framework. For example, McCrea et al. [

The development of strategies to efficiently search a model space appears to have received less attention outside of a Bayesian framework, and the evaluation of model structure and covariate importance for the CormackJolly-Seber (CJS) mark-recapture model [25,26] remains an important area of inquiry. The CJS model is among the earliest open population models and one of the most simple structurally, being comprised of sub-models for survival (φ) and recapture (p) probabilities, but nevertheless continues to enjoy widespread application [27- 31]. An underappreciated consequence of the statistical dependence between sub-model parameters is that the covariate structure in one sub-model can influence the apparent predictive ability of covariates in the other sub-model, termed information “leak” by Catchpole et al. [

The most common strategies to search a CJS model space are the so-called pand φ-first strategies [25,26]. In the p-first strategy, one first searches the p dimension of the model space by pairing all candidate sub-models for p with a general and flexible sub-model for φ. One then searches the φ dimension after conditioning on a selected sub-model for p and pairing it with the remaining sub-models for φ. This stepwise method can be repeated as often as desired, but in practice one or two rounds of pairing a fixed sub-model with a number of candidates for the other sub-model are common. From among the resulting subset of models, a single model may emerge as overwhelmingly dominant and be selected as the preferred model [

Doherty et al. [

We introduce a new strategy, called the plausible combinations (PC) strategy, and investigate its performance by amending the simulations of Doherty et al. [

The strategy for model selection and covariate evaluation we propose is a generalization of the pand φ-first strategies which provides a more effective search of a model space. The most general sub-model for φ is first paired with all candidate sub-models for p. The resulting models are then used to identify a list of “plausible” submodels for p using a two-step procedure. In the first step, models with an AIC_{C} weight [_{C} due to the penalty caused by the number of parameters in the general φ sub-model. Consequently, models with a likelihood exceeding the minimum value of the likelihood among high weight models, L_{m}, are identified as “high likelihood” models in the second step. These models approximate the data as well as the high weight models and could become competitive from an AIC_{C} perspective if some of the sub-models for φ with fewer parameters provide a good approximation to the data. The sub-models for p associated with both high weight and high likelihood models are deemed “plausible” for p. Note that the only product of this two-step procedure is a list of plausible sub-models for p; the estimates themselves are discarded and not utilized further. A list of sub-models deemed plausible for φ is similarly developed by reversing the order of p and φ sub-model conditioning. The parameters of all combinations of plausible sub-models for p and φ are estimated and inference is based on the resulting subset of the entire model space, an approach termed the plausible combinations (PC) strategy.

The performance of the PC strategy was evaluated by replicating the essential elements of the simulations of Doherty et al. [_{1φ}, x_{2φ}, x_{3φ}, x_{4φ}), recapture probabilities (p), and four recapture probability covariates(x_{1p}, x_{2p}, x_{3p}, x_{4p}) for 15 capture occasions (

For each parameter set, the covariates were treated as fixed constants and φ and p (

simulate capture histories over 15 capture occasions. The true model was therefore fully time-variant (φ_{t}p_{t}) and the covariate models were used to approximate this temporal structure. This procedure mimicked the analysis of real data in which the covariates are known constants and the observed capture histories are a realization of random survival and recapture processes. Given that the mean of φ over all occasions was approximately 0.80 in all three parameter sets, an initial release of 100 marked individuals in the first occasion was followed by the release of 20 newly marked individuals in each of the subsequent 13 occasions to maintain approximately 100 marked individuals in the population through time. The 15th occasion was a recapture event only and no newly marked individuals were released. A total of 1000 sets of capture histories were randomly generated for each of the three parameter sets.

For each set of capture histories, the p-first strategy was implemented by pairing the most general sub-model for φ, consisting of an intercept and all four covariates additively, with all 16 sub-models for p. From among the resulting 16 models, the model having the minimum value of AIC_{C} was identified, after which the corresponding sub-model for p was paired with each of the remaining 15 sub-models for φ. The φ-first strategy was conceptually identical, with the order between φ and p submodel conditioning reversed. Inference under both the pand φ-first strategies was therefore based upon 31 (12%) of the possible 256 sub-model combinations. Under the AC strategy, the parameters of all 256 models were estimated. The PC strategy was implemented as previously described.

The fit of estimated models was evaluated by computing the relative sum of squares (RSS):

where represents an element of either φ or p for an individual capture occasion and is the corresponding estimate based on either a single model or computed by model averaging [

For each of the four model selection strategies, the RSS for φ, p, and their sum was computed using the model with the lowest AIC_{C}. The RSS statistics were similarly computed for the model-averaged estimates, using the models considered under each strategy. For comparative purposes, we also report RSS statistics for the one model among all 256 having the smallest total RSS, as well as the fully time-variant model used to generate capture histories. The mean and standard deviation of RSS statistics among the 1000 capture histories with each parameter set were computed.

The importance of parameter covariates and covariate combinations was evaluated by computing cumulative AIC_{C} model weights [

The AC, PC, and p-and φ-first evaluation strategies all yielded approximately equal means and standard deviations for the RSS statistic (_{C} model, and closer than the model-averaged estimates. The model with the minimum RSS had an average of 1 to 1.5 more parameters than the model with minimum AIC_{C}. Because of the increased variance associated with estimating the large number of parameters in the fully-parameterized timevariant model (φ_{t}p_{t}), all strategies produced smaller mean RSS values. Model-averaging produced a modest but consistent improvement in parameter estimation.

All strategies gave the largest cumulative weights to the individual covariates most highly correlated with φ and p (x_{1φ}, x_{2φ}, x_{1p}, and x_{2p}, _{3p} and x_{4p}). For example, the covariate x_{4p} in Parameter set 1 had a correlation with p of only −0.013, but its cumulative weight under the φ-first strategy was 0.509 (

_{t}p_{t}), and each of the four model selection strategies. For each strategy, results are shown for the model with the minimum AIC_{C} value (Min. AIC_{C}) among the models considered, as well the model-averaged estimates (Model Ave.).

_{C}) model weights for individual survival (x_{iφ}) and recapture (x_{ip}) probability covariates observed among estimates from 1000 simulations with each of three parameter sets under the p-first, φ-first, PC, and AC model-selection strategies. Bivariate correlations between the logit of the φ and p parameters and their covariates (Corr.) are taken from [

the covariates having small correlations with φ (x_{3φ} and x_{4φ}). The AC and PC strategies produced cumulative weights for these covariates that were intermediate between those of the pand φ-first strategies. However, note that the use of individual covariates was not balanced under the pand φ-first strategies (see DISCUSSION).

Substantial differences in the standard deviations of the cumulative weights for individual covariates were also observed among the strategies. For example, consider the p-first results for Parameter set 1 (_{3p} and x_{4p}, which both have low correlations with p, were more than double the values observed under the other strategies. Similar results were obtained for x_{3φ} and x_{4φ} under the φ-first strategy, and the results were consistent across the three parameter sets. Covariates that might be considered unimportant on the basis of pairwise correlations accumulated considerable AIC_{C} weight if they were included in the first sub-model selected and so were also included in the subsequent 15 models, but accumulated reduced weight otherwise, resulting in elevated variability.

The difference among the strategies was perhaps most clearly revealed by the cumulative model weights for combinations of covariates, which represent model structure more effectively than weights associated with individual covariates. The results were similar for all three parameter sets, so only the results for Parameter set 1 are shown (

The standard deviation of this φ sub-model was also

_{C}) model weights for combinations of survival (φ) and recapture (p) probability covariates observed among estimates from 1,000 simulations with parameter set 1 under the p-first, φ-first, PC, and AC model-selection strategies.

elevated under the p-first strategy; 0.130 versus 0.080. Sub-models for φ having fewer covariates therefore tended to have smaller mean weights and slightly reduced standard deviations under the p-first strategy. Conversely, under the p-first strategy, the more complex sub-model for p tended to accumulate less weight, submodels with fewer covariates (especially x_{1p} + x_{2p}) tended to accumulate more weight, and the cumulative weights of all sub-models were more variable than under the AC strategy. The results were symmetric for the φ-first strategy (

The numbers of models evaluated under the φ-first, p-first, and AC strategies are known in advance, while the number of models evaluated under the PC strategy cannot be predetermined. The frequency distribution of the number of models evaluated by the PC strategy for each of the three parameter sets is presented in

The PC strategy has several attractive characteristics. Under the pand φ-first strategies, one initially conditions upon a complex sub-model for either p or φ, while evaluating candidate sub-models for the other parameter class. The estimation results from these models are carried forward and constitute over half of the models that are evaluated (16 of 31 models in this investigation). The cumulative weights in Tables 2 and 3 suggest that these stepwise procedures tend to inflate the apparent importance of complex covariate models for one parameter class and, consequently, overly simplistic covariate models for the other parameter class. This phenomenon constitutes an information “leak” between sub-models [

space are disassociated from model results on which inference is based, and the selection of a flexible submodel in the initial modeling step has little influence on the ultimate parameter estimates and covariate weights obtained under the PC strategy. Unlike the pand φ-first strategies, the PC strategy is symmetric with respect to p and φ and the order in which they are considered is immaterial. Finally, the PC strategy explores the potential model space more effectively than the pand φ-first strategies by considering all models that provide a comparatively good approximation to the data, conditioned on the candidate model set, leading to an improved assessment of the importance of covariates and covariate combinations, and its numerical performance was indistinguishable from that of the AC strategy.

One potential disadvantage of the PC strategy is that the number of models to be evaluated cannot be predetermined, and will vary as a function of the observed capture histories and the candidate sub-models. Logically, one can expect the number of models evaluated to increase with the number of competitive sub-models, especially if there are numerous competitive sub-models for both p and φ. However, note that the PC strategy can be modified in straightforward fashion to influence the number of models evaluated. For example, the threshold AIC_{C} model weight of 0.01 used in this investigation could be increased, which would reduce the number of models evaluated. In addition, a threshold based on ΔAIC_{C} could be implemented. In any case, achieving performance equivalent to that of the AC strategy while estimating the parameters of far fewer models should outweigh concern about how many models need to be evaluated in many investigations, especially if parameter estimation can be performed efficiently, as is possible in the R programming environment [35,36].

The equivalent performance of all strategies with respect to parameter estimation, as measured by RSS, is intriguing. We suspect that this is a consequence of working with several additive covariate combinations that provide approximately equal explanatory power. However, based on our experience modeling real data, it is not uncommon for models with different covariate structures to produce parameter estimates with non-trivial differences but have similar model weights. The information “leak” between sub-models [

The issue of covariate balance in a model set is critical when using covariate weights to assess the explanatory utility of covariates. Covariates that are included in a large number of models can accumulate substantial weight even if they are not truly informative. Conversely, a covariate with meaningful explanatory value could conceivably accumulate lower weight than it merits if included in few models, although those models should tend to have relatively high weights. Of the model evaluation strategies considered in this investigation, only the AC strategy was balanced for individual covariates. Various approaches to adjust for unequal covariate balance within a model set have been implemented [34, 37], although no such adjustments were made in this investigation (

We would like to reemphasize the importance of deliberately formulating a candidate model set with careful consideration of prior information, expert opinion, ecological relevance, and research objectives or hypotheses [13,15,16,18]. Including every conceivable model and using an automated or semi-automated strategy to search the model space is data or model dredging [

Model selection and the evaluation of model structure will continue to be fundamental aspects of mark-recapture modeling. The PC strategy offers substantial advantages compared to other strategies in common use, producing parameter estimates and cumulative covariate weights that are nearly indistinguishable from those produced by the AC strategy, without substantially increasing the average number of models evaluated in comparison to the pand φ-first strategies. Although the performance of the PC strategy was investigated using CJS mark-recapture models, the conceptual approach is easily extensible to other classes of models in which the global model can be viewed as a combination of component sub-models, such as multistate CJS models. An investigation of the performance of the PC strategy with more complex models would be informative.

We would like to offer our sincere gratitude to Paul Doherty and Gary White for providing the parameter and covariate values on which the simulations were based, which strengthened the comparative basis of this investigation. We also thank Mark Udevitz and Karen Oakley, US Geological Survey, Alaska Science Center, and anonymous reviewers for constructive comments on drafts of the manuscript. Any mention of trade names is for descriptive purposes only and does not imply endorsement by the US Government, Western Ecosystems Technology, Inc., or Polar Bears International.