Image recognition to the rescue of natural history museums by enabling curators to identify specimens on the fly

New Research Idea, published in RIO Journal presents a promising machine-learning ecosystem to unite experts around the world and make up for lacking taxonomic expertise.

In their Research Idea, published in Research Ideas and Outcomes (RIO Journal), Swiss-Dutch research team present a promising machine-learning ecosystem to unite experts around the world and make up for lacking expert staff

Guest blog post by Luc Willemse, Senior collection manager at Naturalis Biodiversity Centre (Leiden, Netherlands)

Imagine the workday of a curator in a national natural history museum. Having spent several decades learning about a specific subgroup of grasshoppers, that person is now busy working on the identification and organisation of the holdings of the institution. To do this, the curator needs to study in detail a huge number of undescribed grasshoppers collected from all sorts of habitats around the world. 

The problem here, however, is that a curator at a smaller natural history institution – is usually responsible for all insects kept at the museum, ranging from butterflies to beetles, flies and so on. In total, we know of around 1 million described insect species worldwide. Meanwhile, another 3,000 are being added each year, while many more are redescribed, as a result of further study and new discoveries. Becoming a specialist for grasshoppers was already a laborious activity that took decades, how about knowing all insects of the world? That’s simply impossible. 

Then, how could we expect from one person to sort and update all collections at a museum: an activity that is the cornerstone of biodiversity research? A part of the solution, hiring and training additional staff, is costly and time-consuming, especially when we know that experts on certain species groups are already scarce on a global scale. 

We believe that automated image recognition holds the key to reliable and sustainable practises at natural history institutions. 

Today, image recognition tools integrated in mobile apps are already being used even by citizen scientists to identify plants and animals in the field. Based on an image taken by a smartphone, those tools identify specimens on the fly and estimate the accuracy of their results. What’s more is the fact that those identifications have proven to be almost as accurate as those done by humans. This gives us hope that we could help curators at museums worldwide take better and more timely care of the collections they are responsible for. 

However, specimen identification for the use of natural history institutions is still much more complex than the tools used in the field. After all, the information they store and should be able to provide is meant to serve as a knowledge hub for educational and reference purposes for present and future generations of researchers around the globe.

This is why we propose a sustainable system where images, knowledge, trained recognition models and tools are exchanged between institutes, and where an international collaboration between museums from all sizes is crucial. The aim is to have a system that will benefit the entire community of natural history collections in providing further access to their invaluable collections. 

We propose four elements to this system: 

  1. A central library of already trained image recognition models (algorithms) needs to be created. It will be openly accessible, so any other institute can profit from models trained by others.
Mock-up of a Central Library of Algorithms.
  1. A central library of datasets accessing images of collection specimens that have recently been identified by experts. This will provide an indispensable source of images for training new algorithms.
Mock-up of a Central Library of Datasets.
  1. A digital workbench that provides an easy-to-use interface for inexperienced users to customise the algorithms and datasets to the particular needs in their own collections. 
  2. As the entire system depends on international collaboration as well as sharing of algorithms and datasets, a user forum is essential to discuss issues, coordinate, evaluate, test or implement novel technologies.

How would this work on a daily basis for curators? We provide two examples of use cases.

First, let’s zoom in to a case where a curator needs to identify a box of insects, for example bush crickets, to a lower taxonomic level. Here, he/she would take an image of the box and split it into segments of individual specimens. Then, image recognition will identify the bush crickets to a lower taxonomic level. The result, which we present in the table below – will be used to update object-level registration or to physically rearrange specimens into more accurate boxes. This entire step can also be done by non-specialist staff. 

Mock-up of box with grasshoppers mentioned in the above table

Results of automated image recognition identify specimens to a lower taxonomic level.

Another example is to incorporate image recognition tools into digitisation processes that include imaging specimens. In this case, image recognition tools can be used on the fly to check or confirm the identifications and thus improve data quality.

Mock-up of an interface for automated taxon identification. 

Using image recognition tools to identify specimens in museum collections is likely to become common practice in the future. It is a technical tool that will enable the community to share available taxonomic expertise. 

Using image recognition tools creates the possibility to identify species groups for which there is very limited to none in-house expertise. Such practises would substantially reduce costs and time spent per treated item. 

Image recognition applications carry metadata like version numbers and/or datasets used for training. Additionally, such an approach would make identification more transparent than the one carried out by humans whose expertise is, by design, in no way standardised or transparent.

*

Follow RIO Journal on Twitter and Facebook.

*

Research publication:

Greeff M, Caspers M, Kalkman V, Willemse L, Sunderland BD, Bánki O, Hogeweg L (2022) Sharing taxonomic expertise between natural history collections using image recognition. Research Ideas and Outcomes 8: e79187. https://doi.org/10.3897/rio.8.e79187

In a hole in a tunicate there lived a hobbit: New shrimp species named after Bilbo Baggins

Digital illustration by Franz Anthony.

Two new species of tiny symbiotic shrimps are described, illustrated and named by biology student at Leiden University Werner de Gier as part of his bachelor’s research project, supervised by Dr. Charles H. J. M. Fransen, shrimp researcher of Naturalis Biodiversity Center (Leiden, the Netherlands).

Inspired by the extremely hairy feet of one of the species, the authors decided that they should honour Middle Earth’s greatest halfling, Bilbo Baggins.

Aptly named Odontonia bagginsi, the new shrimp joins the lines of other species named after Tolkien’s characters such as the cave-dwelling harvestman Iandumoema smeagol, the golden lizard Liolaemus smaug and the two subterranean spiders Ochyrocera laracna and Ochyrocera ungoliant.

Photo by Charles Fransen.

The newly described shrimps were collected during the Ternate expedition to the Indonesian islands of Tidore and Ternate, organised by Naturalis Biodiversity Center and the Indonesian Institute of Sciences (LIPI) in 2009.

Typically for the Odontonia species, the new shrimps do not reach sizes above a centimetre in length, and were found inside tunicates. It is believed that these symbiotic crustaceans are fully adapted to live inside the cavities of their hosts, which explains their small-sized and smooth bodies.

Photo by Charles Fransen.

Unlike most Odontonia species, which live inside solitary tunicates, the new species Odontonia plurellicola was the first one to be associated with a colonial tunicate. These tunicates have even smaller internal cavities, which explains the tiny size of the new species.

To determine the placement of the new species in the tree of life, the scientists compared the shrimps’ anatomical features, including the legs, mouthparts and carapace. As a result, they were assigned to Odontonia. Further, the available genetic information and Scanning Electron Microscope (SEM) images of the unusual feet of the newly discovered shrimp provided a new updated identification key for all members of the species group.

“Being able to describe, draw and even name two new species in my bachelor years was a huge honour. Hopefully, we can show the world that there are many new species just waiting to be discovered, if you simply look close enough!” says Werner de Gier, who is currently writing his graduate thesis at Naturalis Biodiversity Center and working together with Dr. Charles Fransen on crustaceans.

###

Original source:

de Gier W, Fransen CHJM (2018) Odontonia plurellicola sp. n. and Odontonia bagginsi sp. n., two new ascidian-associated shrimp from Ternate and Tidore, Indonesia, with a phylogenetic reconstruction of the genus (Crustacea, Decapoda, Palaemonidae). ZooKeys 765: 123-160. https://doi.org/10.3897/zookeys.765.25277

Digital illustration by Franz Anthony.