Validating three methods for making probability statements in fisheries forecasts.
Patterson, K. R.; Cook, R. M.; Darby, Chris; Gavaris, S.; Mesnil, Benoit; Punt, A. E.; Restrepo, V. R.; Skagen, Dankert W.; Stefánsson, Gunnar; Smith, M.
Abstract
We report on a test of three methods that have been used by ICES in making probability statements about
fisheries forecasts to management agencies (XSA/WGMTERM, ICA/ICP, and a stochastic projection
method that was first applied to North Sea herring). The principle of the test is to calculate many stock
assessments and forecasts using historic data, and to compare probability statements about some quantities
(forecast biomass, relative biomass, catch etc.) with the latest and presumably more reliable estimates. For
most ICES and NAFO stocks where a sufficiently long time-series exists, we calculate retrospective (8 year
before present) assessments and then calculate medium term forecasts 5 years forwards. By comparing the
frequencies of expected and actual outcomes, we conclude that some methods can be used to make
reasonably reliable probability statements about relative biomass (e.g. biomass relative to biomass at some
time in the past), but that for most other quantities and most methods the accuracy of probability estimates
is very poor. In general, the methods tended to underestimate uncertainty and there was a relatively large
proportion of "unexpected" outcomes in the forecasts.
Publisher
ICESSeries
ICES CM documents2000V:06