Image recognition to the rescue of natural history museums by enabling curators to identify specimens on the fly

New Research Idea, published in RIO Journal presents a promising machine-learning ecosystem to unite experts around the world and make up for lacking taxonomic expertise.

In their Research Idea, published in Research Ideas and Outcomes (RIO Journal), Swiss-Dutch research team present a promising machine-learning ecosystem to unite experts around the world and make up for lacking expert staff

Guest blog post by Luc Willemse, Senior collection manager at Naturalis Biodiversity Centre (Leiden, Netherlands)

Imagine the workday of a curator in a national natural history museum. Having spent several decades learning about a specific subgroup of grasshoppers, that person is now busy working on the identification and organisation of the holdings of the institution. To do this, the curator needs to study in detail a huge number of undescribed grasshoppers collected from all sorts of habitats around the world. 

The problem here, however, is that a curator at a smaller natural history institution – is usually responsible for all insects kept at the museum, ranging from butterflies to beetles, flies and so on. In total, we know of around 1 million described insect species worldwide. Meanwhile, another 3,000 are being added each year, while many more are redescribed, as a result of further study and new discoveries. Becoming a specialist for grasshoppers was already a laborious activity that took decades, how about knowing all insects of the world? That’s simply impossible. 

Then, how could we expect from one person to sort and update all collections at a museum: an activity that is the cornerstone of biodiversity research? A part of the solution, hiring and training additional staff, is costly and time-consuming, especially when we know that experts on certain species groups are already scarce on a global scale. 

We believe that automated image recognition holds the key to reliable and sustainable practises at natural history institutions. 

Today, image recognition tools integrated in mobile apps are already being used even by citizen scientists to identify plants and animals in the field. Based on an image taken by a smartphone, those tools identify specimens on the fly and estimate the accuracy of their results. What’s more is the fact that those identifications have proven to be almost as accurate as those done by humans. This gives us hope that we could help curators at museums worldwide take better and more timely care of the collections they are responsible for. 

However, specimen identification for the use of natural history institutions is still much more complex than the tools used in the field. After all, the information they store and should be able to provide is meant to serve as a knowledge hub for educational and reference purposes for present and future generations of researchers around the globe.

This is why we propose a sustainable system where images, knowledge, trained recognition models and tools are exchanged between institutes, and where an international collaboration between museums from all sizes is crucial. The aim is to have a system that will benefit the entire community of natural history collections in providing further access to their invaluable collections. 

We propose four elements to this system: 

  1. A central library of already trained image recognition models (algorithms) needs to be created. It will be openly accessible, so any other institute can profit from models trained by others.
Mock-up of a Central Library of Algorithms.
  1. A central library of datasets accessing images of collection specimens that have recently been identified by experts. This will provide an indispensable source of images for training new algorithms.
Mock-up of a Central Library of Datasets.
  1. A digital workbench that provides an easy-to-use interface for inexperienced users to customise the algorithms and datasets to the particular needs in their own collections. 
  2. As the entire system depends on international collaboration as well as sharing of algorithms and datasets, a user forum is essential to discuss issues, coordinate, evaluate, test or implement novel technologies.

How would this work on a daily basis for curators? We provide two examples of use cases.

First, let’s zoom in to a case where a curator needs to identify a box of insects, for example bush crickets, to a lower taxonomic level. Here, he/she would take an image of the box and split it into segments of individual specimens. Then, image recognition will identify the bush crickets to a lower taxonomic level. The result, which we present in the table below – will be used to update object-level registration or to physically rearrange specimens into more accurate boxes. This entire step can also be done by non-specialist staff. 

Mock-up of box with grasshoppers mentioned in the above table

Results of automated image recognition identify specimens to a lower taxonomic level.

Another example is to incorporate image recognition tools into digitisation processes that include imaging specimens. In this case, image recognition tools can be used on the fly to check or confirm the identifications and thus improve data quality.

Mock-up of an interface for automated taxon identification. 

Using image recognition tools to identify specimens in museum collections is likely to become common practice in the future. It is a technical tool that will enable the community to share available taxonomic expertise. 

Using image recognition tools creates the possibility to identify species groups for which there is very limited to none in-house expertise. Such practises would substantially reduce costs and time spent per treated item. 

Image recognition applications carry metadata like version numbers and/or datasets used for training. Additionally, such an approach would make identification more transparent than the one carried out by humans whose expertise is, by design, in no way standardised or transparent.

*

Follow RIO Journal on Twitter and Facebook.

*

Research publication:

Greeff M, Caspers M, Kalkman V, Willemse L, Sunderland BD, Bánki O, Hogeweg L (2022) Sharing taxonomic expertise between natural history collections using image recognition. Research Ideas and Outcomes 8: e79187. https://doi.org/10.3897/rio.8.e79187

Journal of Hymenoptera Research links Crocodile Dundee, Toblerone, Game of Thrones & Alien

A myriad of species and genera new to science, including economically important wasps drawing immediate attention because of their amusing names and remarkable physical characters, in addition to work set to lay the foundations for future taxonomic and conservation research, together comprise the latest 64th issue of Journal of Hymenoptera Research (JHR).

The species Qrocodiledundee outbackense

Two genera (Qrocodiledundee and Tobleronius) named after the action comedy Crocodile Dundee and the chocolate brand Toblerone are only a couple of the 14 new genera from the monograph of the microgastrine wasps of the world’s tropical regions, authored by Dr Jose Fernandez-Triana and Caroline Boudreault of the Canadian National Collection of insects in Ottawa. In their article, the team also describes a total of 29 new species, where five of them carry the names of institutions holding some of the most outstanding wasp collections.

Another curiously named species of microgastrine wasp described in the new JHR issue, is called Eadya daenerys in reference to Daenerys Targaryen, a fictional character known from the best-selling book series A Song of Ice and Fire by George R. R. Martin, and the blockbuster TV show Game of Thrones. Discovered by University of Central Florida‘s Ryan Ridenbaugh, Erin Barbeau and Dr Barbara Sharanowski as a result of a collaboration between biocontrol researchers and taxonomists, the new species might not be in control of three dragons, nor a ruler or protector of whole nations. However, by being a potential biocontrol agent against a particular group of leaf beetle pests, it could spare the lives of many eucalyptus plantations around the world.

The species Tobleronius orientalis

Furthermore, a wasp named Dolichogenidea xenomorph, which parasitises other eucalyptus pests, is also named after a character from a sought-after franchise. The scriptwriters of the horror sci-fi movie series Alien are thought to have been thinking of parasitic wasps when they came up with the character Xenomorph, remind authors Erinn Fagan-Jeffries, Dr Steven Cooper and Dr Andrew Austin. Additionally, the team from University of Adelaide and the South Australian Museum point out that the species name translates to ‘strange form’ in Greek, which perfectly suits the characteristic remarkably long ovipositor of the new wasp.

The species Eadya daenerys

In another paper of the same journal issue, Dr. Jean-Luc Boevé, Royal Belgian Institute of Natural Sciences, Diego Domínguez, Universidad Técnica Particular de Loja, Ecuador, and Dr David Smith, Smithsonian’s National Museum of Natural History, USA, publish an illustrated list of the wasp-related sawflies, which they collected from northern Ecuador a few years ago. They also provide a checklist of the country’s species.

In conclusion, the fifth paper, authored by Serbian scientists Dr Milana Mitrovic Institute for Plant Protection and Environment, and Prof Zeljko

The species Dolichogenidea xenomorph

Tomanovic, University of Belgrade, studies ways to extract DNA from dry parasitoid wasps from the natural history archives decades after their preservation. In their work, they make it clear that such projects are of great importance for future taxonomic and conservation research, as well as agriculture.

***

The open access Journal of Hymenoptera Research is published bimonthly by the scholarly publisher Pensoft on behalf of the International Society of Hymenopterists.

***

Original sources:

Boeve; J, Dominguez D, Smith D (2018) Sawflies from northern Ecuador and a checklist for the country (Hymenoptera: Argidae, Orussidae, Pergidae, Tenthredinidae, Xiphydriidae). Journal of Hymenoptera Research 64: 1-24. https://doi.org/10.3897/jhr.64.24408

Ridenbaugh RD, Barbeau E, Sharanowski BJ (2018) Description of four new species of Eadya (Hymenoptera, Braconidae), parasitoids of the Eucalyptus Tortoise Beetle (Paropsis charybdis) and other Eucalyptus defoliating leaf beetles. Journal of Hymenoptera Research 64: 141-175. https://doi.org/10.3897/jhr.64.24282

Fagan-Jeffries EP, Cooper SJB, Austin AD (2018) Three new species of Dolichogenidea Viereck (Hymenoptera, Braconidae, Microgastrinae) from Australia with exceptionally long ovipositors. Journal of Hymenoptera Research 64: 177-190. https://doi.org/10.3897/jhr.64.25219

Boeve; J, Dominguez D, Smith D (2018) Sawflies from northern Ecuador and a checklist for the country (Hymenoptera: Argidae, Orussidae, Pergidae, Tenthredinidae, Xiphydriidae). Journal of Hymenoptera Research 64: 1-24. https://doi.org/10.3897/jhr.64.24408

Mitrovic M, Tomanovic Z (2018) New internal primers targeting short fragments of the mitochondrial COI region for archival specimens from the subfamily Aphidiinae (Hymenoptera, Braconidae). Journal of Hymenoptera Research 64: 191-210. https://doi.org/10.3897/jhr.64.25399