Wouldn’t it be great if an app or computer program could identify a bird from its vocalizations or from a photo?

The future is here, thanks to machine learning.

Today, we are going to look at the intersection of identification of birds and advanced computing. In particular, I will concentrate on machine learning, a type of computation considered a part of the field of artificial intelligence. The app with lots of buzz is Merlin, developed by the Cornell Laboratory of Ornithology. It is available free on Android and Apple IOS platforms.

Machine learning is used to create Merlin. The application is given a training set of bird vocalizations and photos whose identity is known. The program then seeks to teach itself how to identify each of those vocalizations or images. Once trained, the program is then given unknown vocalizations or images and provides an identification. The programmer can check whether those identifications are correct. If not, the program is informed of the wrong decision and will revise its learning. So, the application should get better and better over time.

When you first open the app, you choose a part of the world where you do your birding. You also will be prompted to download a photo ID file. For identification of images in Merlin, you either take a photo of a bird in the field or use a photo from your library. Merlin will ask where the photo was taken and will provide its best reckoning of your species, along with other secondary choices.

Merlin first appeared in 2014. Initial versions could identify 400 common North American species from images. Now, using the vast photo library at the Cornell Lab of Ornithology, the app is well trained. Over 8,000 species can be identified now, 80% of the world’s bird species.

How good is Merlin at identifying birds from photographs? I have found it is remarkably accurate if you have a decent photograph. Some users report a 90% or higher rate of correct identifications.

To me, Merlin’s ability to identify sounds is more useful and is nothing short of astounding. You press the sound ID button on the app and a recording begins. Obviously, getting close to the singing or calling bird is ideal, but smartphone microphones are impressive these days.

Merlin will then identify the sound of your mystery bird as well as any other birds that are vocalizing. It’s cool to put your phone out in your backyard to discover all the birds that are vocalizing. Merlin gets over 90% of the sound identifications right.

You can imagine how this technology can be used to census birds. My friend Randy Downer used Merlin while conducting a Breeding Bird Survey in June. The survey involves three-minute stops at 50 sites. At each stop, Randy compared his sound identifications with Merlin, finding a high level of concordance.

A similar result was found by a survey volunteer in North Carolina. At one site, the observer was listening intently to a tanager, trying to decide if it was a summer tanager or a scarlet tanager. He missed some rough-winged swallows flying overhead, but Merlin didn’t.

At another site, Merlin had six more species than the observer. A major flaw? No, that stop had a northern mockingbird singing and its repertoire of mimicked species fooled Merlin. I think this result is really a testament to how good mockingbirds are at mimicry.

With the use of any identification software, a skeptical perspective is warranted. Particularly for species that Merlin identifies that are rare or beyond the usual range, take a hard look and see if other species are more likely. If you hear hoof beats, think horses, not zebras.

Although Merlin is widely appreciated, a significant minority oppose its use in conducting censuses or just identifying birds because of possible errors and because Merlin could be a barrier to developing our own skills in auditory and visual identification.

Herb Wilson taught ornithology and other biology courses at Colby College. He welcomes reader comments and questions at whwilson@colby.edu

Copy the Story Link