An efficient protocol and data set for automated otolith image analysis

Information on fish age constitutes one of the most important biological variables for a fish stock, and an accurate estimation of the age structure of the fish populations is essential for the reliable management of these natural resources. The age of individual cod (Gadus morhua) is determined by manually examining the layered structure of otoliths, a calcium carbonate structure of the inner ear. Image‐based methods to age otoliths have been investigated for over 4 decades with varying results, but recent developments in automatic image analysis techniques are promising. The objective of this paper is to describe a method to efficiently image a manually broken otolith (avoiding the time‐consuming embedding and cross‐sectioning process) and to describe the organization and acquisition of imaged broken otolith images with associated metadata for a collection of north‐east Arctic cod otoliths. A single‐lens reflex camera was used for capturing photographs of the broken otoliths. A total of six images were acquired for each subject, consisting of three images in the first position with three different light exposures and three images in the second position with three different light exposures. This results in a simple and efficient procedure for capturing clear, satisfactory, and reproducible images of broken fish otoliths, and a more straightforward and less labour‐intensive alternative to the commonly used methods that involve embedding and cross‐sectioning of the otolith.


| INTRODUCTION
Information on fish age constitutes one of the most important biological variables, which is used for studying the life history (e.g. growth, sexual maturation) and population dynamics (Campana, 2001) of fish stocks. The age structure of fish populations is essential in any fisheries management system that is dependent on age-based analytical assessment models (Hilborn and Walters, 2013), and this includes a large proportion of commercially important fish stocks around the world (Hart and Reynolds, 2002). In most cases, this is obtained by physical sampling of individual fish, either from fish catches or from scientific surveys. Varieties of hard structures in fish, including opercula, vertebrae, spines or fin rays, are used for ageing purposes, but scales and otoliths are the most frequently used structures (Casselman, 1987;Campana and Thorrold, 2001).
Otoliths are calcified ear stones used by fish for balance and hearing (Panfili et al., 2002). They are metabolically inert and grow throughout the life of the fish (Campana and Neilson, 1985). Changes in the structure of the otolith are visible in the seasonally dominated habitats (Godiksen et al., 2010), where they form annual increments composed by two bands characterized by different opacity, that is the opaque and translucent zones (Katayama, 2018). These periodicities in the structure are widely used for ageing purposes (Høie and Folkvord, 2006), whereas seasonal zones can be visible in both whole (untreated) otoliths and/or after some form of preparation, such as cutting, breaking, burning or slicing Panfili et al., 2002;Vitale et al., 2019).
The north-east Arctic cod (Gadus morhua) is the world's largest cod stock and supports a large fishery in addition to being an important component of the Barents Sea ecosystem (Nakken, 2008;Yaragina et al., 2011). The fish is distributed across the Barents Sea and migrates to the Norwegian coast for spawning (Bogstad et al., 2016). Fishing occurs mainly in the Barents Sea and at the spawning grounds in the traditional Lofoten fishery (Opdal, 2010). Several scientific surveys in addition to catch sampling programmes are used to monitor the stock, and these monitoring programmes support agebased assessment models that are used for setting the total allowable catches (Olsen et al., 2010;ICES, 2019).
The age of individual cod is routinely determined by a trained expert examining the otoliths. This involves removal of the otolith, cleaning, breaking it in two pieces and reading it under a stereomicroscope. The average time to become a certified age reader at the Institute of Marine Research (IMR) is approximately 4 years. Decisions to scale up sampling may be compromised by significant resource demands and training lead times. Alternatively, some otolith laboratories use thin slices for the age determination, which require labour-intensive embedding of the otolith in epoxy resin and cutting with precise cut-off machines (ICES, 2013). This is a rather time-consuming process and is not always feasible for large sets of otoliths (Panfili et al., 2002), but offer clear surfaces suitable for traditional imaging techniques.
Imaging techniques are useful since they facilitate the development of online repositories of otolith pictures (Lombarte et al., 2006) and efficient organisation of inter-institutional exchange workshops (Appelberg et al., 2005). Different otolith imaging techniques have been developed, including 2-dimensional (2D) and 3-dimensional (3D) imaging (Schulz-Mirbach et al., 2013). However, 3D images are rarely used for age estimation purposes (Fisher and Hunter, 2018).
Image-based methods for ageing fish have been tried with varying results (Fisher and Hunter, 2018), but automatic image analysis technique is a rapidly developing field, and great strides have been made recently, using a technique known as 'deep learning'. Deep learning employs deep neural networks which are convolutional neural networks (CNN) that are organised in many layers (hence 'deep') and work directly on the image pixels with increased abstraction (LeCun et al., 2015). The method has been applied on images of Greenland halibut otoliths (Moen, 2018). The advantage of the Greenland halibut otoliths is that they can be imaged for ageing purposes without breaking them since annual increments are apparent on the whole otolith, whereas the cod otoliths need to be sectioned to make the increments visible. The rugged structure of the broken surface makes the imaging more challenging.
The objective of this paper is to describe a method to efficiently image a manually broken otolith (to avoid the embedding and cross-sectioning process) and to describe the organization and acquisition of broken otolith images with associated metadata for a subset of the historical north-east Arctic cod otolith material stored at the IMR. We have published the images and associative metadata as a reference internet archive for developing automated image-based methods.

| The surveys
Otoliths were taken from the two main fishery-independent surveys for north-east arctic cod: the joint Norwegian-Russian winter survey (Jakobsen et al., 1997) and the Norwegian Spawning Cod acoustic-trawl survey (Korsbrekke, 1997). The winter survey covers the Barents Sea and has been conducted since 1981. The Norwegian Spawning Cod acoustictrawl survey covers the Lofoten area and has been performed annually in March-April since 1982. The total number of trawl stations for the different surveys is given in Table 1.
On each of the surveys, the otoliths are sampled using a random-stratified sampling based on fish length for each trawl station. On the Norwegian-Russian Winter Survey, otoliths from individual fish are randomly sampled. Once there is one fish per 5-cm-length class, no further otoliths of that fish length are collected. This corresponds to a length-based random-stratified sampling scheme for each trawl station. On the Norwegian Spawning Cod acoustic-trawl survey, otoliths from five fish are sampled for each five-centimetre size class. On rare occasions (approximately five times per survey), the catches can be very large, and the otoliths from ten fish will be sampled for each size class. Therefore, the amount of otolith samples collected from a station is dependent on the specific survey and on the size distribution of the fish in each catch.
We subsampled the complete material by randomly picking trawl stations within each survey and imaged all otoliths sampled for one station. We aimed to balance the data set between survey and year, and the number of stations imaged is given in Table 1. The number of otoliths per station depends on whether the respective length groups were present in the catches or not. For some stations, the catch may be split into two subsamples. In those cases, we imaged all sampled otoliths for both subsamples.

| Sample selection method
At the surveys, otolith samples were organized into boxes containing sets of envelopes which were separated by rubber bands (Figure 1). The boxes were labelled with the survey name, the year and the trawl station numbers. Each set of envelopes represented one station (trawl haul), labelled by a trawl sample serial number. Each set included a different number of envelopes depending on the available number of cod for the respective length groups at that station at the time of collection. From each fish, one pair of sagittae cod otoliths (the largest of the three pairs of otoliths) were collected and stored in the envelopes. Each envelope was labelled with individual fish data such as fish ID, length, weight, sex and age.
One of the otoliths was broken in half as a result of the preceding age reading process. Sometimes both otoliths were broken depending on how easy the first broken otolith was to read for the age reader. Each envelope was labelled with a number; determined age; and weight, length and sex of the fish.
A sampling frame was created by placing serial numbers in random order separated by survey and year, resulting in simple random sampling design within a year and survey. During the image capturing process, stations were chosen in the order of the randomized sampling frame.

| Camera settings
The camera used for capturing photographs of the broken otoliths was a Canon model EOS 5D Mark II. It is a digital single-lens reflex camera with a full-frame complementary metal oxide semiconductor sensor of approximately 36 mm × 24 mm and with approximately 21.10 effective megapixels. The camera was attached to an LMscope (Micro Tech Lab) mount and connected to a computer (Figure 1) using the 'EOS Utility 2' (Canon U.S.A., 2019) application allowing communication between the camera and the computer. This application includes functions like downloading and displaying images, remote shooting and camera setting control.
The camera and the computer were connected to one another after making sure they were both turned on. The EOS Utility 2 application on the computer was manually initiated if it did not automatically start-up after connection. The 'Camera settings/Remote shooting' option was selected from the menu that appeared on the computer. The correct camera settings were applied to the camera ( Table 2). The auto exposure bracketing (AEB) setting was used to get three images of the same subject at three different light exposures. When the capture button is clicked three times successively, each picture will have a different light exposure setting (in order of: standard exposure, decreased exposure, increased exposure) (Figure 2) according to the exposure compensation amount and the AEB amount that was manually set by the user (Table 2).

| Light settings
For illumination, a LEICA CLS 150× photonic cold light source (Leica Microsystem) was used. The top and bottom dial were set to 4 and 6, respectively. Two photonic goosenecks were attached to the illumination system to allow for free configuration of light to the specimen. The goosenecks were adjusted to illuminate each side of the otolith from below the mount (Figure 1). This allows the light to beam up into the otolith from the bottom making the growth lines visible. The goosenecks may be moved slightly after each otolith was placed under the camera to adapt the lighting to fit each otolith, but the general configuration stayed constant. This was done to make sure parts of the otolith fracture surface are not too overexposed or too underexposed in order to get a good picture of all the growth lines as clearly as possible.

| Selection of otolith
The otolith half that seemed to have the clearest otolith growth lines were chosen from each envelope. In order to do this, a criterion was followed: the subjects chosen were ones where the fracture surface was flat enough so the image could be focused at all parts of the subject area as much as possible. If the break was rugged and it was difficult to see the growth lines clearly, otoliths that presented a clear core or nucleus were selected as this ensured that the right number of age zones were visible. A clean break at the core is beneficial when reliably determining the age.

| Preparation of otolith for imaging
In order to hold the otolith in place when capturing the images, a 'mount' was formed with a dark blue modelling compound, like Play-Doh™ (Figure 3). This modelling compound is the same as the age readers use. The otolith was placed onto the mount with the fracture surface facing towards the camera lens as horizontal as possible. The mounted subject was placed centrally in the camera frame, and the camera height was adjusted to gain a clear image without adjusting the magnification setting. After placement under the camera, a thin layer of moisture was added onto the surface of the otolith fraction to make the growth lines smoother and more visible.

| Image capture
In the EOS Utility application, the 'Live View Shoot' option was selected with the 'Depth-of-Field Preview' on. This allowed the subject to be viewed in real time on the computer screen ( Figure 1). After establishing this setting, the target shutter speed, before image capturing, should be 1/80 seconds. If it was not, then the goosenecks were slightly adjusted by pushing them close together or pulling them farther apart. Then, the Live View Shoot setting was reinitiated until the target shutter speed was reached.
A total of six images were acquired for each broken otolith subject, consisting of three images with different exposures for two different orientations of the otolith (Figure 2). For the first position setting, the subject was positioned so the ventral side of the otolith was near the bottom of the camera frame in an overall vertical arrangement. The spot metering box which is shown in the Live View Shoot window was positioned on the otolith as much as possible rather than on the T A B L E 2 Manual settings used on the Canon EOS 5D Mark II.
These can either be activated on the camera itself or through the EOS Utility 2 application on the computer. Mode and magnification must be activated on the camera itself

Mode
'Av' (Aperture-Priority AE) background. This ensured that the changing light exposures were respective to the otolith subject rather than the dark background. The three shots were taken by clicking the capture button three times successively. The three results were the same image but at three different light exposures ( Figure  2). Then the second position was arranged by turning the subject 180° so that the ventral side of the otolith was near the top of the camera frame, again, in an overall vertical arrangement. Three successive shots were taken again, and the spot metering box was positioned again. At the end of the image capturing process, a collection of six images should result ( Figure 2). If the otolith was very big and could not fit in the camera frame in the vertical direction, it was positioned to make it fit in the frame for the first position and then turned 180° for the second position. Capturing images of the same otolith in different positions will provide slightly different images of the same subject. This can provide additional data (like data augmentation), and also a useful control for automatic image analysis applications.

| Example images
The images taken during this study consisted of cod otoliths that were aged between 1 and 16 years old. Figure 4 shows the range of variability in different ages with respect to size

F I G U R E 2
Resulting images of AEB image capturing. The layered structure that is formed during growth is visible in the images. Dimensions of each image are 11.9 × 8.0 mm with an image resolution of 470.7 pixels/mm

F I G U R E 3
This picture displays the mount made of modelling compound and the contents of an individual envelope taken from a set of envelopes, or station of the otolith and quality of the image. For instance, an age 1 otolith does not have any growth lines yet, while the age 15 otolith has many growth lines, and the density can make them harder to distinguish.

| DATA ORGANIZATION
The data can be downloaded from the Norwegian Marine Data Centre and the data are licenced under the Creative Commons Attribution 4.0 International License.

| Images
The six images were automatically stored by the Canon software using a default name. They were then placed in folders organized as follows: Year This folder contains six image files with consecutive numbers Some sets of envelopes have the same serial number but were split into two sets with one labelled as 'Del 1' and the other labelled as 'Del 2'. This was done at the time of the survey when two samples came from one catch. This occurs either when (1) both juveniles and adults are present in the catch and they are separated because the juveniles are measured in millimetres (mm) and the adults are measured in centimetres (cm) or (2) when a large amount of one size is present (for example, if there are many medium-sized fish and very few large-sized fish in one catch). These two groups of different sized fish will be handled as two subsamples (parts 1 and 2) from one catch in order to get a better size distribution.

| Metadata file
The information about the trawl stations and individual samples are taken from the IMR's central data repository. The raw biotic data are given as XML files adhering to IMR's biotic ver 3 format. * The files are placed in the 'biotic' folder biotic and are named based on the surveys, that is: * https ://www.imr.no/forma ts/nmdbi otic/v3/nmdbi oticv3_en.html

F I G U R E 4
Example images of broken cod otoliths of different ages. Images were captured using the procedure created in this study. Dimensions of each image are 11.9 × 8.0 mm with an image resolution of 470.7 pixels/mm biotic_cruiseNumber_2017102_G+O+Sars.xml biotic_cruiseNumber_2017625_Kristine+Bonnevie.xml biotic_cruiseNumber_2017849_Helmer+Hanssen.xml biotic_cruiseNumber_2018836_Helmer+Hanssen.xml biotic_cruiseNumber_2018203_Johan+Hjort.xml biotic_cruiseNumber_2018202_Johan + Hjort.xml A table for the individual data is extracted from the XML files and presented in the 'indiv idual data.csv' file. The metadata contains the age and other parameters recorded for the individual otolith, c.f. Table 3 for an explanation of the fields.

Field name Description
Serialno A trawl station is uniquely identified within a year by its serial number.