A contrastive learning approach for individual re-identification in a wild fish population
DOI:
https://doi.org/10.7557/18.6824Keywords:
Individual, Recognition, Identification, Biometric, Temperate species, Fish, Deep learning, Siamese network, Embedding, CNNAbstract
In both terrestrial and marine ecology, physical tagging is a frequently used method to study population dynamics and behavior. However, such tagging techniques are increasingly being replaced by individual re-identification using image analysis.
This paper introduces a contrastive learning-based model for identifying individuals. The model uses the first parts of the Inception v3 network, supported by a projection head, and we use contrastive learning to find similar or dissimilar image pairs from a collection of uniform photographs. We apply this technique for corkwing wrasse, Symphodus melops, an ecologically and commercially important fish species. Photos are taken during repeated catches of the same individuals from a wild population, where the intervals between individual sightings might range from a few days to several years.
Our model achieves a one-shot accuracy of 0.35, a 5-shot accuracy of 0.56, and a 100-shot accuracy of 0.88, on our dataset.
References
J. Bromley, J. W. Bentz, L. Bottou, I. Guyon, Y. LeCun, C. Moore, E. Säckinger, and R. Shah. Signature verification using a “siamese” time delay neural network. International Journal of Pattern Recognition and Artificial Intelligence, 7(04):669–688, 1993.
J. Bruslund Haurum, A. Karpova, M. Pedersen, S. Hein Bengtson, and T. B. Moeslund. Re-identification of zebrafish using metric learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision Workshops, pages 1–11, 2020. doi: 10.1109/WACVW50321.2020.9096922.
E. M. Ditria, E. L. Jinks, and R. M. Connolly. Automating the analysis of fish grazing behaviour from videos using image classification and optical flow. Animal Behaviour, 177:31–37, jul 2021. ISSN 00033472. doi: 10.1016/j.anbehav.2021.04.018.
M. Goodwin, K. T. Halvorsen, L. Jiao, K. M. Knausgård, A. H. Martin, M. Moyano, R. A. Oomen, J. H. Rasmussen, T. K. Sørdalen, and S. H. Thorbjørnsen. Unlocking the potential of deep learning for marine ecology: overview, applications, and outlook. ICES Journal of Marine Science, 01 2022. ISSN 1054-3139. doi: 10.1093/icesjms/fsab255.
A. Gupta, E. Bringsdal, N. Salbuvik, K. M. Knausgård, and M. Goodwin. An accurate convolutional neural networks approach to wound detection for farmed salmon. In International Conference on Engineering Applications of Neural Networks, pages 139–149. Springer, 2022. doi: 10.1007/978-3-031-08223-8 12.
A. Gupta, E. S. Kalhagen, Ø. L. Olsen, and M. Goodwin. Hierarchical object detection applied to fish species: Hierarchical object detection of fish species. Nordic Machine Intelligence, 2(1), 2022. doi: 10.5617/nmi.9452.
K. T. Halvorsen, T. K. Sørdalen, C. Durif, H. Knutsen, E. M. Olsen, A. B. Skiftesvik, T. E. Rustand, R. M. Bjelland, and L. A. Vøllestad. Male-biased sexual size dimorphism in the nest building corkwing wrasse (Symphodus melops): Implications for a size regulated fishery. ICES Journal of Marine Science, 73 (10):2586–2594, nov 2016. ISSN 10959289. doi: 10.1093/icesjms/fsw135.
K. T. Halvorsen, T. K. Sørdalen, L. A. Vøllestad, A. B. Skiftesvik, S. H. Espeland, and E. M. Olsen. Sex- and size-selective harvesting of corkwing wrasse (Symphodus melops) — a cleaner fish used in salmonid aquaculture. ICES Journal of Marine Science, 74(3):660–669, mar 2017. ISSN 1054-3139. doi: 10.1093/icesjms/fsw221.
K. T. Halvorsen, T. Larsen, H. I. Browman, C. Durif, N. Aasen, L. A. Vøllestad, A. Cresci, T. K. Sørdalen, R. M. Bjelland, and A. B. Skiftesvik. Movement patterns of temperate wrasses ( Labridae ) within a small marine protected area. Journal of Fish Biology, 99(4):1513–1518, jul 2021. ISSN 0022-1112. doi: 10.1111/jfb.14825.
E. Hoffer and N. Ailon. Deep metric learning using triplet network. In International workshop on similarity-based pattern recognition, pages 84–92. Springer, 2015. doi: 10.1007/978-3-319-24261-3 7.
G. Jocher, A. Stoken, A. Chaurasia, J. Borovec, et al. ultralytics/yolov5: v6.0 - YOLOv5n ’Nano’ models, Roboflow integration, TensorFlow export, OpenCV DNN support, Oct. 2021. URL https://doi.org/10.5281/zenodo.5563715.
K. M. Knausgård, A. Wiklund, T. K. Sørdalen, K. T. Halvorsen, A. R. Kleiven, L. Jiao, and M. Goodwin. Temperate fish detection and classification: a deep learning based approach. Applied Intelligence, 52(6):6988–7001, 2022. doi: 10.1007/s10489-020-02154-9.
D. Li, H. Su, K. Jiang, D. Liu, and X. Duan. Fish face identification based on rotated object detection: Dataset and exploration. Fishes, 7 (5):219, 2022. doi: 10.3390/fishes7050219.
V. Lopez-Vazquez, J. M. Lopez-Guede, S. Marini, E. Fanelli, E. Johnsen, and J. Aguzzi. Video image enhancement and machine learning pipeline for underwater animal detection and classification at cabled observatories. Sensors, 20(3):726, 2020. doi: 10.3390/s20030726.
S. M. Lundberg and S.-I. Lee. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30, 2017.
E. Meidell and E. S. Sjøblom. Fishnet: A unified embedding for salmon recognition. Master’s thesis, NTNU, 2019.
O. Moskvyak, F. Maire, F. Dayoub, A. O. Armstrong, and M. Baktashmotlagh. Robust re-identification of manta rays from natural markings by learning pose invariant embeddings. In 2021 Digital Image Computing: Techniques and Applications (DICTA), pages 1–8. IEEE, 2019. doi: 10.1109/DICTA52665.2021.9647359.
S. Schneider, G. W. Taylor, S. Linquist, and S. C. Kremer. Past, present and future approaches using computer vision for animal reidentification from camera trap data. Methods in Ecology and Evolution, 10(4):461–470, 2019. doi: 10.1111/2041-210X.13133.
S. Schneider, G. W. Taylor, and S. C. Kremer. Similarity learning networks for animal individual re-identification-beyond the capabilities of a human observer. In Proceedings of the IEEE/CVF winter conference on applications of computer vision workshops, pages 44–52, 2020.
C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
I. Uglem, G. Rosenqvist, and H. S. Wasslavik. Phenotypic variation between dimorphic males in corkwing wrasse. Journal of Fish Biology, 57(1):1–14, jul 2000. ISSN 00221112. doi: 10.1006/jfbi.2000.1283.
B. G. Weinstein. A computer vision for animal ecology. Journal of Animal Ecology, 87(3): 533–545, 2018. doi: 10.1111/1365-2656.12780.
Downloads
Published
Issue
Section
License
Copyright (c) 2023 Ørjan Langøy Olsen, Tonje Knutsen Sørdalen, Morten Goodwin, Ketil Malde, Kristian Muri Knausgård, Kim Tallaksen Halvorsen
This work is licensed under a Creative Commons Attribution 4.0 International License.
