update on the state of video based monitoring in the northeast region ... · update on the state of...

Update on the state of video based monitoring in the northeast region (in two parts) 1/2

Nichole A. Rossi & Andrew W. Jones Northeast Fisheries Science Center

NEFSC Noon Seminar September 12th 2018

Groundfish EFP: Audit-Model• Project partners: The Nature Conservancy, Cape Cod Commercial

Fishermen’s Alliance, Maine Coast Fishermen’s Association and ~20 fishermen in five groundfish sectors

• Service providers: Ecotrust Canada and Gulf of Maine Research Institute

• Goals: Develop EM for discard catch accounting in the groundfish fishery; incentivize electronic Vessel Trip Reporting; refine fish handling protocols; inform a future audit-model program.

• Two EFPs issued in support of this project: • Electronic ASM: Vessels use EM when selected for coverage through

PTNS • 100% EM: Vessels use EM on 100% of trips

Groundfish EFP: Audit-ModelOn-the-Water Vessel Operations

• Haul-level electronic Vessel Trip Reporting (eVTR) • Species ID and length data collected for groundfish discards • Incentives for 100% accountability

Data Uses and Analysis • NOAA Fisheries: Audit-model simulations • SMAST: Testing EM uses beyond catch accounting; analyses

may include CPUE, discard composition, and spatial-temporal information

Groundfish EFP: Maximized Retention• Projects partners: Gulf of Maine Research Institute and 3 fishermen in one groundfish sector

• Service provider: Integrated Monitoring • Goals: Test the viability of maximized retention model for high-volume vessels; develop standards and protocols for at-sea operations; implement a 3rd party dockside monitoring program to support catch accounting shoreside

• EM used to verify discard compliance • One EFP was issued in support of this project: Vessels will use EM on 100% of trips.

Groundfish EFP: Maximized RetentionOn-the-Water Vessel Operations

• Haul-level electronic Vessel Trip Reporting • No discarding of allocated groundfish • Protocols that allow video reviewers to distinguish between allocated

and unallocated groundfish Dockside Operations

• Three-pronged dockside monitoring program to support project: 1. Verification of dealer-reported landings 2. Biosamples of undersized fish 3. Fish hold inspections

EM in the Midwater Trawl Fishery• NMFS worked with Saltwater Inc. (SWI) to conduct a study to

evalute EM as a tool to monitor catch retention (August 2016 - January 2018)

• 11 MWT vessels volunteered to participate in the study

• SWI and NMFS produced a final report on the study in January 2018

• NMFS and Council staff presented the EM the study findings in March 2018 at the NEFMC

• NMFS used costs from the EM study to estimate industry costs for EM in the herring fishery

EM in the Midwater Trawl Fishery• EM Review Panel determined EM is suitable for detecting discard

events in the herring MWT fishery

• NMFS determined that EM/PS is adequate substitute for ASM coverage aboard MWT vessels

• Herring Advisory Panel and Committee met on April 4th and supported approving EM/PS as a monitoring option for the MWT fleet

• NEFMC met on April 19th and the Council voted to approve EM/PS as an adequate substitute for ASM aboard MWT vessels and that an EFP be used to initially administer the EM/PS program

FIS/NOP Projects in 2018 https://www.st.nmfs.noaa.gov/data/fis/index

Electronic Monitoring Comparative Technology Study

• Comparison of still-image systems to traditional video-based systems leveraging the current work in northeast fisheries

• Goal: Explore still image or low frame rate systems and evaluate potential efficiencies

• Higher resolution

• Smaller total file size

• Compare catch data, storage requirements, review processes

• 15-month Contract awarded to Archipelago and AIS, Inc.

• Project can accommodate up to 5 vessels (operating both systems)

• Develop general processes for adoption and evaluation of new technology

Building a Groundfish Image Library from NEFSC Suvery Collections (R/V Bigelow)

• Collection of a set of high-quality images to advance a framework for machine learning (develop algorithms and software)

• Goal: Collect a large set of images from NEFSC survey (fall and spring)

• Create a data set that can be shared to support a national image library

• Develop algorithms to support video annotation

• Further automated image analysis in the region

• 14-month contract awarded to CVision Consulting, Kitware, and Gulf of Maine Research Institute

• Scheduled to install cameras on the Bigelow before Leg II (Sept 21st)

Detecting and Reducing Bycatching in the Herring Fishery

• Extension of previous work with Electronic Monitoring in the herring fishery

• Goal: Improve detection, identification, and weight estimation of bycatch (haddock) in the mid-water trawl herring fishery

• Explore the use of EM to identify bycatch discarded at dewatering box

• Evaluate the utility of echosounder technology to reduce bycatch

• 15-month contract awarded to Saltwater (early fall or late spring)

• Project can accommodate up to 3 vessels

• Information learned may improve catch cap monitoring and inform modifications to bycatch regulations

Quick Questions?

Update on the state of video based monitoring in the northeast region (in two parts) 2/2

Nichole A. Rossi & Andrew W. Jones Northeast Fisheries Science Center

NEFSC Noon Seminar September 12th 2018

Thank you!• Would not be possible without assistance from GARFO staff:

• Dan Linden, Brandt McAfee, Claire Fitzgerald • Support from other center staff (to varying degrees):

• Justin Potter, Corey Endres, Mike Palmer, Amy Martins, John Hoey, Chad Demarest, and Anna Henry

• As well as data and support from a number of other project partners:

Groundfish in New England

• Managed via catch shares • Discards & bycatch

estimated by observers • Relatively low monitoring

requirements • Fishery managers

considering increased monitoring

Reali

zed m

onito

ring c

over

age (

%)

0%

10%

20%

30%

Fishing year

2010

2011

2012

2013

2014

2015

2016

2017

Observer and Monitor Combined

Audit-model electronic monitoring (EM) may be a means to advance monitoring• Potential to remove ‘observer bias’ • Potential reduce costs (increase efficiency) • A way to increase the volume of fisheries dependent

data available to managers • Regional investigation of utility of audit EM for ~5 years • More concerted efforts for the past ~2 years • Currently ~20 vessels (smaller number active)

What do we mean by audit-model EM?

• Paired vessel trip report (VTR) and EM summary files

• Self reported VTR is primary data source for discard estimates of managed groundfish species

• Independent EM summary file used to verify a subset of VTR record (‘trust but verify’)

eVTR Species weights (self reported)

EM review Species weights

(independent)

Pass Difference

thresholds or other rules

eVTR Record of discards






EM review Species weights

(independent)

Fail Difference

thresholds or other rules

EM Record of discards

• Paired vessel trip report (VTR) and EM summary files

• Self reported VTR is primary data source for discard estimates of managed groundfish species

• Independent EM summary file used to verify a subset of VTR record (‘trust but verify’)

What do we mean by audit-model EM?

Need to develop business rules• Relatively few audit model EM programs • Business rules often based on economic realities • Other US regions looking to scale back EM to an audit

program from high coverage (slightly different) • Interest in setting rules based on data available

1.To insure data quality 2.But also seize on opportunity to reduce costs

Answering questions to inform business rules• Example of questions:

1. How many efforts ’pass’ given different thresholds? 2. How does an audit affect our estimates of discards?

• Exploring these types of questions provide insight into business rule specifics (e.g., video review rate)

• Looking to synthesize answers across questions

Data sets available• Two primary data comparisons are being used

1. A comparison of haul-by-haul NEFOP/ASM discard data and self-reported discard data collected from vessels participating in cooperative research programs

2. A comparison of a haul-by-haul EM discard summary and self-reported haul-by-haul discard data

• Currently performing similar analyses on both data sets

1. How many efforts pass given different thresholds?• We want to set thresholds that captain’s can achieve • Also explore what is to be gained by increasing review

rates • To get at this question we simulated fishing effort using

each existing data set • Explore thresholds applied at the trip and haul levels • Calculate the percent of passing and failing efforts at given

different thresholds • Also, using threshold for all species vs individual species

1. How many efforts pass given different thresholds?

−−−−−−−−−−−

−−−−

−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

70% Review 80% Review 90% Review



0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100

0%

25%

50%

75%

100%

0%

25%

50%

75%

100%

0%

25%

50%

75%

100%

Weight difference threshold (lbs)

Perc

ent p

assi

ng tr

ips

Each species has the same threshold − 200 iterationsHAUL−level percent passing trips for eVTR − EM

Mean number of passing hauls

Individual simulation

−−−−

−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−




0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100

0%

25%

50%

75%

100%

0%

25%

50%

75%

100%

0%

25%

50%

75%

100%


Perc

ent p

assi

ng tr

ips




200 iterations

−−−−

−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−

−−−−−−−−−−−




0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100

0%

25%

50%

75%

100%

0%

25%

50%

75%

100%

0%

25%

50%

75%

100%


Perc

ent p

assi

ng tr

ips





200 iterations


Taking this a bit farther we can explore different thresholds (e.g., using a trip total or individual spp.)

---------

------------

----------

-----------

----------

-----------

----------

-----------

----------

-----------

----------

-----------

----------

-----------

----------

-----------

----------

-----------70% Review 80% Review 90% Review



0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200 0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200 0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200

0%

25%

50%

75%

100%

0%

25%

50%

75%

100%

0%

25%

50%

75%

100%


Per

cent

pas

sing

trip

s

All species have same threshold except COD and HAL with have 1/4 the indicated threshold (x-axis)Trip-level percent passing trips for eVTR - EM

Mean number of passing trips


Taking this a bit farther we can explore different thresholds (e.g., using a trip total or individual spp.)

CAT

COD

FLBB

FLDAB

FLGS

FLSD

FLYT

HADD

HAL

POLL

POUT

RED

WHAK

0% 5% 10% 15%Percent of failing trip*

Spe

cies

Threshold (lbs)10

20

30

40

50

Using data from 2016, 2017, & 2018 accepted through the GARFO APIBreakdown of the percent `failing` trips using different wieght thresholds

* (of all trips in the data set)Percent of failing trips*

• Essentially, we want to know what the impact of using self-reported data on the accuracy and precision of discard accounting

• Interested in comparing discard estimates between 1. the audit model (potentially the future) 2. a ratio based estimator (the current method) 3. a census base estimate (as close to the truth as we can get)

• Again simulated fishing effort using each existing data sets • Calculate total discards for each species • Explore how different haul-level review rates affected each

estimate

2. How does an audit affect our estimate of discards

2. How does an audit affect our estimates of discards

Mean audit estimates (5 - 95% coverage)

Mean ratio estimate (15 % coverage)

Census estimates (100% coverage)

Proportion of hauls sampled


2. How does an audit affect our estimates of discards

Proportion of hauls sampled


Preliminary results• These two analyses and others suggest that review rate below

100% is clearly warranted • A review rate of 20-30% (of trips/hauls) might be sufficient to

develop discard estimates of similar quality to those currently being used (for all but three species)

• A weight tolerance of ~30 lbs means ~80% of hauls will pass and might be a ‘good’ threshold to begin the program with

• If comparisons are made at the trip level, higher tolerances are likely needed (between 50 and 100 lbs per spp per trip) to achieve a higher passing rate

What are we doing with these results?• Fishing year 2018 vessels fishing with 100% EM video review • But, VTRs being used as the estimate of discards (essentially an audit

rate of 100%) • Beginning of ‘trust but verify’ • Transition of a lower rate likely mid-year (soon) • Will also shift to initial pass/fail rate mid year (soon) • Continue to expand/repeat these analyses to evaluate the effectiveness

of pre-implementation business rules • Hope to publish to provide example of a method for other fisheries • Hope to combine with analyses of program cost analyses

Questions?

update on the state of video based monitoring in the northeast region ... · update on the state of...

Documents