Do Different Methods Provide Accurate Probability Statements in the Short Term?

Abstract

The performance of uncertainty estimation procedures was evaluated with respect to accuracy. A confidence statement is said to be accurate if the confidence point achieves the desired probability coverage. A Monte Carlo experiment with 100 trials was conducted with a true population that experienced contrast between low and high fishing mortality. Observations for the last 25 years were drawn stochastically by adding measurement error. The assessment approaches were VPA-based (ADAPT and XSA with errors on the effort data). The Delta Method , parametric bootstrap and non-parametric bootstrap (NPB), and a Bayesian approach were used to quantify coverage and assess the accuracy of confidence limits of estimated interest parameters (F0.1, SSB and TACF0.1 in year 26) by comparing against the true values. Variations of the Delta Method and bootstrap were used to account for statistical estimation bias. The results indicated that accurate inference statements are possible with the different approaches and that bias correction can improve accuracy when it can be applied. The bias-corrected Delta-ADAPT and bias-corrected NPB-ADAPT applications performed best. Inference statements about F0.1 were more accurate than those for SSB or TAC.