Improving Fish Stock Assessments
Report of the
Committee on Fish Stock Assessment Methods
By
Ocean Studies Board
Commission on Geosciences, Environment, and Resources
National Research Council
 |
 |
[ to
the NAS website where the full report is available on-screen]
Preface Global marine fish harvest has plateaued and many important commercial stocks have been depleted. The Ocean Studies Board (OSB), at the request of the National Marine Fisheries Service (NMFS), has provided advice designed to improve management of marine fisheries in the United States (NRC, 1994a). Many of the changes suggested in the 1994 report Improving the Management of U.S. Marine Fisheries were incorporated in the reauthorized Magnuson-Stevens Fishery Conservation and Management Act (MSFCMA) in 1996. NMFS also has asked the OSB for advice on specific issues of Atlantic bluefin tuna population biology; the OSB presented its results in another 1994 report, An Assessment of Atlantic Bluefin Tuna (NRC, 1994b). Following the publication of that report, there was a widespread expression of the need for reviews of several other specific stock assessments. NMFS responded by requesting a broad review of the methods used in the United States for stock assessments. The results of that review are presented in this report. This study would not have been possible without the efforts of NMFS scientists who carried out blind runs of data provided by the study committee and participated in the committee's meetings. The cooperation between academic and agency scientists was commendable and actually led to advances in the state of the art of fish stock assessments. The OSB offers sincere thanks to NMFS analysts for their considerable efforts, including Ray Conser, Jeff Fujioka, Wendy Gabriel, Phil Goodyear, Jim Ianelli, Rick Methot, Jerry Pella, Clay Porch, Joe Powers, Mike Prager, Victor Restrepo, Gerald Scott, and Mike Sigler. Other individuals also contributed to the committee's work and deserve the thanks of the committee and the OSB: Jie Zheng (Alaska Department of Fish and Game), Andre Punt (Commonwealth Scientific and Industrial Research Organization, Australia), and David Fournier (Otter Research Ltd.). The results of this study will serve as an important foundation for the National Research Council review of the U.S. Northeast fisheries stock assessments that was mandated by Congress in 1996 as part of the MSFCMA reauthorization. The recommendations presented herein should also be useful to ongoing international activities related to fish stock assessments, such as those of the International Council for the Exploration of the Sea.
Marine fisheries provide a vital contribution to food supplies, employment, and culture worldwide. Therefore, matching fishing activities with natural fluctuations so as to avoid unsustainable harvests and population crashes is an important goal. In an ideal world, accurate and precise estimates of the abundance of fish stocks and their dynamics (how and why population levels change) would be available to set sustainable harvest levels to accommodate commercial and recreational demand. In reality, fishery management is based on imperfect estimation of the number, biomass, productivity, and age structure of fish populations and incomplete knowledge of population dynamics. The ocean is relatively opaque to light, and acoustic techniques of remote sensing are not yet sufficiently developed for general use in estimating fish populations. Thus, it is difficult to count fish through nondestructive means and fish usually must be caught to be counted, weighed, and measured. Standardized techniques have been developed to sample a relatively small proportion of fish from a population and to combine such data with commercial and recreational catch information to estimate population characteristics. These techniques yield stock assessments used by managers at state, regional, national, and international levels. In addition to monitoring the abundance and productivity of exploited fish populations, stock assessments can provide a quantitative prediction of the consequences of possible alternative management actions. The mechanisms that cause fish populations to change are poorly understood but include environmental and ecosystem effects, interactions among multiple species, and effects of humans through harvesting, pollution, habitat disruption, and other factors. Without accurate stock assessments and their proper use in management, exploited fish populations can collapse, creating severe economic, social, and ecological problems. Therefore, ensuring that stock assessment research progresses and that operational stock assessments use the best techniques for a given stock are fundamental for ensuring the sustainability of commercial and recreational marine fisheries. Stock assessment is a multistage process. Steps include (1) definition of the geographic and biological extent of the stock, (2) choice of data collection procedures and collection of data, (3) choice of an assessment model and its parameters and conduct of assessments, (4) specification of performance indicators and evaluation of alternative actions, and (5) presentation of results. This report concentrates on evaluating assessment models, with less extensive treatment of the other steps. Chapter 1 discusses these steps in greater detail. Techniques of stock assessment range from informal estimates to more sophisticated modeling approaches used to combine data of various types. Assessment models predict rates of change in biomass and productivity based on information about yield from fisheries and the rates at which fish enter the harvestable population (recruitment), grow in size, and exit the population (natural and fishing mortality). Stock assessments for fish living in the U.S. exclusive economic zone (3 to 200 nautical miles from shore) and for some highly migratory species are conducted by scientists from the National Oceanic and Atmospheric Administration's (NOAA's) National Marine Fisheries Service (NMFS) and independent species group commissions (e.g., the International Pacific Halibut Commission and the Inter-American Tropical Tuna Commission). In addition, interstate fishery management commissions were created to facilitate the coordination of state assessment scientists in working with each other and with federal scientists to assess and manage stocks shared among states in their coastal waters (within 3 nautical miles from shore on open coasts, as well as bays and estuaries). These organizations include the Atlantic States, Gulf States, and Pacific States Marine Fisheries Commissions. Some states (e.g., Alaska, Oregon, and Florida) also perform assessments for fisheries conducted in their own state waters. Fishery management organizations use the results of stock assessments
to design and implement various controls for the total catch that can be
removed from fish populations under their jurisdictions. Commercial catch
can be managed by specifying the amount of harvesting allowed; the areas
of fishing and times of the year that fishing can take place; the gear
that can be used; minimum fish size limits; and in some cases, the amount
of fish that any single fisher, community, company, or other entity can
catch. Recreational fisheries more often impose minimum size limits, daily
catch limits, seasons, and sometimes gear restrictions and requirements
to release fish that are caught.
The National Research Council (NRC) Committee on Fish Stock Assessment Methods was formed in early 1996 to review existing stock assessment methods and to consider alternative approaches for the future. The committee's statement of task was two-fold:1. Conduct a scientific review of stock assessment methods and models for marine fisheries management. 2. Compare models using actual and simulated data having a variety of characteristics, to test the sensitivity and robustness of the models to data quality and type. As part of this study, the committee asked selected stock assessment scientists to conduct blind runs of simulated data sets using five different models. Models tested included a production model, a delay-difference model, and three age-structured methods (described in detail in Chapter 3). The goal of the simulation study was to evaluate the performance of stock assessment methods for simulated fish populations for which the true population parameters were known (to the committee, but not to the analysts) and some of the assumptions usually made in stock assessments were violated. One type of data set was typical of the catch biomass, age composition of the catch, and catch per unit effort (CPUE) that are obtained from commercial and recreational fisheries. The other type of data set was typical of that collected by fishery-independent surveys. Each analyst was asked to evaluate five 30-year sets of simulated commercial and survey data, alone and in combination. The five data sets provided different combinations of parameters in terms of the following: • Increasing or decreasing stock size over time (population trend) • Constant versus changing age of fish caught (fishery selectivity) over time • Accuracy of catch reported by fishers • Ability of fishery and survey vessels to catch fish (fishery and survey catchability) The analysts were given essential information about fish growth and maturity, the probability of mis-estimating fish ages, and selected information about the structure of the populations and the data. Analysts were not provided information about natural mortality, catchability, selectivity, recruitment, or the amount of underreporting (although they were warned that underreporting might have occurred). In addition to the results of these basic analyses, (1) some analysts
repeated their model runs with the true average natural mortality (provided
by the committee), (2) key management variables were calculated by analysts
and the committee, and (3) retrospective analyses were conducted by the
committee to determine the persistence of over- or underestimation of population
parameters over time by the different models. Greater detail about the
study process is given in Chapter 5 and Appendix E.
The committee focused its examination on the data that are used in assessments,
model performance, use of harvest strategies, new assessment techniques,
periodic review and quality control of assessments and assessment methods,
and education and training of stock assessment scientists. The committee
based its recommendations on the results of the simulations and on its
collective experience. Caveats about how the analyses conducted for this
study compare to actual stock assessments are given in Chapter 5. Accomplishing
the recommendations of this report will require concerted and cooperative
action by all interested parties (academic and government scientists, fishery
managers, user groups, and environmental nongovernmental organizations)
to improve the stock assessment process and products.
Data Collection And Assessment Methods The committee concludes that stock assessments do not always provide enough information to evaluate data quality and to estimate model parameters, and it recommends a checklist that would promote more complete data collection for use in stock assessments. The results of the committee's simulations demonstrated that the availability of continuous sets of data collected by using standardized and calibrated methods is important for the use of existing stock assessment models. The best index of fish abundance is one for which extraneous influences (e.g., changes in gear and seasonal coverage, changes in fishers' behavior) can be controlled. The committee recommends that at least one reliable abundance index should be available for each significant stock. CPUE data from commercial fisheries, if not properly standardized, do not usually provide the most appropriate index. Likewise, CPUE data from recreational fisheries require standardization to serve as a good index of abundance. Fishery-independent surveys offer the best opportunity for controlling sampling conditions over time and the best choice for achieving a reliable index if they are designed well with respect to location, timing, sampling gear, and other considerations of statistically valid survey design. NMFS should support the long-term collection of fishery-independent data, using either the NOAA fleet or calibrated independent vessels. Diminishing the quality of fishery-independent data by failing to modernize NOAA fishery research vessels or by changing sampling methods and gear without proper calibration could reduce the usefulness of existing and future data sets. The simulation study demonstrated that assessments are sensitive to underlying structural features of fish stocks and fishery practices, such as natural mortality, age selectivity, catch reporting, and variations in these or other quantities. Auxiliary information in the form of indices or survey estimates of abundance, population structure information, and accurate estimates of other population parameters (e.g., natural or fishing mortality, growth, catchability) improves the accuracy of assessments. Formally reviewed sampling protocols for collection of commercial fisheries
statistics have not been implemented in many geographic regions. The lack
of formalized, peer-reviewed data collection methods in commercial fisheries
is problematic because bias and improper survey conduct may exist, with
unknown impact on data reliability. Greater attention should be devoted
to sampling design based on an understanding of the statistical properties
of the estimators for catch at age and other factors. Sampling and subsequent
analysis should also consider the issue of systematic biases that emerge
with factors such as misreporting. Formalized sampling protocols have been
developed for recreational fisheries in the form of the Marine Recreational
Fisheries Statistics Survey (MRFSS). MRFSS data and methods, albeit imperfect,
have undergone independent peer review, are readily available, and could
serve as a model for commercial fisheries. The committee recommends that
a standardized and formalized data collection protocol be established for
commercial fisheries nationwide.
Models Both harvesting strategies and decision rules for regulatory actions have to be evaluated simultaneously to determine their combined ability to sustain stocks. Simulation models should be realistic and encompass a wide range of possible stock responses to management actions and natural fluctuations consistent with experience. The committee recommends that fish stock assessments present realistic measures of the uncertainty in model outputs whenever feasible. Although a simple model can be a useful management tool, more complex models are needed to better quantify the unknown aspects of the system and to address the long-term consequences of specific decision rules adequately. Retrospective analyses performed by the committee showed that persistent over- or underestimation can occur over a number of years of assessment, regardless of which model is used. The committee recommends the use of Bayesian methods both for creating distributions of input variables and for evaluating alternative management policies. Other methods for including realistic levels of uncertainty in models also should be investigated. In the simulations, model performance became erratic as more variability or errors were introduced to data sets. Newer modeling methods offer promise for reducing bias in key parameter estimates, although using mathematically sophisticated assessment models did not mitigate poor data quality. Different assessment models should be used to analyze the same data to help recognize poor data and to improve the quality of assessment results. Results from such comparisons can be used to direct survey programs to improve data quality and to assess the degree of improvement in data achieved over time. Greater attention should also be devoted to including independent estimates of natural mortality and its variability in assessment models. Further simulation work of this kind is also needed to determine whether the simulation results and the conclusions based on these results remain the same over multiple replications. The committee believes that single-species assessments provide the best
approach at present for assessing population parameters and providing short-term
forecasting and management advice. Recent interest in bringing ecological
and environmental considerations and multi-species interactions into stock
assessments should be encouraged, but not at the expense of a reduction
in the quality of stock assessments.
Harvest Strategies Although the committee did not evaluate alternative harvest strategies, it believes that assessment methods and harvest strategies should be evaluated together because harvest strategies can affect stock assessments and the uncertainty inherent in stock assessments should be reflected in harvest strategies. Despite the uncertainty in stock assessments, fishery scientists may be able to identify robust management measures that can at least prevent overfishing, even if they cannot optimize performance. Conservative management procedures include management tools specific to the species managed, such as minimum biomass levels, size limits, gear restrictions, and area closures (for sedentary species). Management procedures by which the allowable catch is set as a constant fraction of biomass (used for many U.S. fisheries) generally perform better than many alternative procedures. However, errors in implementation due to assessment uncertainties could result in substantial reductions in long-term average harvests in some years if biomass estimates are highly uncertain. Assessment methods and harvest strategies need to be evaluated simultaneously to determine their ability to achieve management goals. Application of risk-adjusted reference points (based on fishing mortality or biomass) would immediately lead to reduced total allowable catch and thus create an economic incentive for investment in improved data gathering and assessment procedures to reduce the coefficient of variation of biomass estimates. There are at least four alternatives to harvesting a constant fraction
of exploitable biomass that may result in levels of total mortality that
are consistent with maintaining a fish stock. First, target fishing mortality
can be reduced as a stock decreases in size to reduce risks. Second, a
minimum biomass level can be established, below which fishing would be
halted (this is done for some U.S. fisheries). Third, the size of fish
captured can be increased by changing requirements for harvest gear. This
restriction might allow smaller fish to escape and spawn, but could be
ineffective if harvesters apply more effort to the larger fish. Finally,
geographic areas can be closed to limit mortality for sedentary species
if the distribution of organisms is well known and if the fishing mortality
in other areas is not increased. Area closures have been implemented or
proposed for many fisheries worldwide in the form of marine reserves and
sanctuaries.
New Approaches NMFS and other organizations responsible for fisheries management should support the development of new techniques that can better accommodate incomplete and variable data and can account for the effects of environmental fluctuations on fisheries. Such techniques should allow the specification of uncertainty in key parameters (rather than assuming constant, known values), should be robust to measurement error, and should include the ability to show the risks associated with estimated uncertainty. A few prominent recommendations for new approaches emerged from the study. Scientists that conduct stock assessments and organizations that depend on assessments should • incorporate Bayesian methods and other techniques to include realistic uncertainty in stock assessment models; • develop better assessment models for recreational fisheries and methods to evaluate the impacts of the quality of recreational data on stock assessments; • account for effects of directional changes in environmental variables (e.g., those that would accompany climate change) in new models; and • develop new means to estimate changes in average catchability, selectivity, and mortality over time, rather than assuming that these parameters remain constant. The results from the simulation exercise should be sobering to scientists, managers, and the users of fishery resources. The majority of the estimates of exploitable biomass exceeded true values by more than 25%; assessments that used accurate abundance indices performed roughly twice as well as those that use faulty indices. A disturbing feature of the assessment methods is their tendency to lag in their detection of trends in the simulated population abundance over time. For example, some methods with some types of data consistently overestimate exploitable biomass during periods of decreasing simulated abundance and underestimate exploitable biomass during periods of increasing simulated abundance. Although no stock assessment model was free from significant error in the simulations, it is also true that few of the models failed consistently. Hence, the message of this report is not that stock assessment models should not be used, but rather that data collection, stock assessment techniques, and management procedures need to be improved in terms of their ability to detect and respond to population declines. The simulation results and some actual fishery management examples suggest that overestimation of stock biomass and overfishing of a population can occur due to inaccurate stock assessments and that the overestimation can persist over time. The committee believes that the two most important management actions to mitigate this problem are: (1) to model and express uncertainty in stock assessments explicitly, and (2) to incorporate uncertainty explicitly into management actions such as harvesting strategies. The absence of adequate data is the primary factor constraining accurate
stock assessments. The differences between estimated and true values derived
from the simulated data were most likely not introduced by any mistakes
made by the analysts. Rather, the large differences that occurred under
some scenarios were primarily the result of poor data and model misspecification
stemming from incomplete knowledge of the true situation by the analysts.
The surplus production and delay difference models did not include the
ability to account for changes over time in key parameters for the simulated
populations. The simulated data sets were better structured for analysis
by age-structured methods; hence, these kinds of models performed better.
When they did not perform well, it was generally because the models used
biased information (e.g., the fishery CPUE index) or did not account for
changes in selectivity and catchability over time. Had the analysts been
told about these data features, it is likely that they could have compensated
for them and obtained better assessments. Some of the newer models appear
to be able to achieve such compensation through the introduction of process
errors. Nevertheless, modeling will never be able to provide estimates
that are as accurate as direct knowledge obtained by measurement and experimentation.
Thus, if future stock assessments are to avoid some of the past problems,
management agencies must devote the necessary resources to monitor and
investigate fish populations in a stable research environment that fosters
creative approaches.
Peer Review It is imperative that stock assessment procedures and results be understood
better and trusted more by all stakeholders. One means to achieve such
trust is to conduct independent peer review of fishery management methods
and results including (1) the survey sampling methods used in data collection,
(2) stock assessment procedures, and (3) risk assessment and management
strategies. When applied properly to stock assessments, peer review yields
an impartial evaluation of the quality of assessments as well as constructive
suggestions for improvement. Such reviews are most beneficial when conducted
periodically, for example, every 5 to 10 years, as new information and
practices develop. In addition, a complete review of methods for collection
of data from commercial fisheries should be conducted in the near future
by an independent panel of experts, which could lead to the adoption of
formal protocols.
Education and Training Reduction in the supply of stock assessment scientists would endanger
the conduct of fishery assessments by the federal government, interstate
commissions, and international management organizations and would hinder
progress in the development and implementation of new stock assessment
methods. NMFS and other bodies that conduct and depend on fish stock assessments
should cooperate to ensure a steady supply of well-trained stock assessment
scientists by using mechanisms such as personnel exchanges among universities,
government laboratories, and industry and by funding stock assessment research
activities. The training of stock assessment scientists should endow them
with skills in applied mathematics, fisheries biology, and oceanography.
Education of fisheries scientists should be organized and executed in such
a way that it complements and augments the NMFS research mission and leads
to improved management strategies for fisheries in the future.
|
|
NATIONAL ACADEMY PRESS • 2101 Constitution Avenue, N.W. • Washington, DC 20418 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competencies and with regard for appropriate balance. This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce Alberts is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the
charter of the National Academy of Sciences, as a parallel organization
of outstanding engineers. It is autonomous in its administration and in
the selection of its members, sharing with the National Academy of Sciences
the responsibility of advising the federal government. The National Academy
of Engineering also sponsors engineering programs aimed at meeting national
needs, encourages education and research, and recognizes the superior achievements
of engineers. Dr. William A. Wulf is president of the
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce M. Alberts and Dr. William A. Wulf are chairman and vice chairman, respectively, of the National Research Council. This report and the committee were supported by the National Oceanic and Atmospheric Administration. The views expressed herein are those of the authors and do not necessarily reflect the views of the sponsor. This study was supported by Contract No. 50-DGNC-3-00016 between the National Academy of Sciences and the National Oceanic and Atmospheric Administration. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that provided support for the project. Library of Congress Catalog Card Number 98-84433
Additional copies of this report are available from: National Academy Press
Copyright 1998 by the National Academy of Sciences. All rights reserved. Printed in the United States of America.
RICHARD DERISO (Cochairman), Inter-American Tropical Tuna Commission,
La Jolla, California
Staff EDWARD R. URBAN, JR., Study Director
|
 |
 |
 |
 |
