Скачать или смотреть SOUND BASED BIRD CLASSIFICATION - Sound Recognition - Bird Noise Recognition -Research- Colaboratory

This project was designed to be a collaboration on a real-life problem which machine learning can help to solve with a typical structure of a data science project including data research and analysis, data preparation, creation of models, analysis of results (or model improvement) and the final presentation.
After the weeks of work, the group has managed to build a solution that predicts the right bird’s name with 87% accuracy on the test sample.
Are you curious about the solution that has been built? We invite you to travel into a world of birds songs.

The birds’ problem
The birdsong analysis and classification is a very interesting problem to tackle.
Birds have many types of voices and the different types have different functions. The most common are song and ‘other voices’ (e.g. call-type).
The song is the “prettier” — melodic type of voice, thanks to which the birds mark their territory and get partners. It is usually much more complex and longer than “call”.
Call-type voices include contact, enticing and alarm voices. Contact and attracting calls are used to keep birds in a group during flight or foraging, for example in the treetops, alarm ones to alert (e.g. when a predator arrives). Most often these are short and simple voices.

Example:
The song is a simple lively rhythmic verse with a slightly mechanical sound, e.g. “te-ta te-ta te-ta” or three-syllable with a different accent, “te-te-ta te-te-ta te-te-ta”

sample
The call has a rich repertoire. Joyful “ping ping” voices, cheerful “si yut-tee yut-tee” and the chattering “te tuui”. In the autumn you can often hear slightly questioning, more shy “te te tiuh”. He warns with a hoarse crackling “yun-yun-yun-yun”. The ramps fill the forest with persistent penetrating “te-te-te te-te-te”.

Samples - https://www.xeno-canto.org/464650

Why can sound-based bird classification be a challenging task?
There are many problems you can encounter:
background noise — especially while using data recorded in a city (e.g. city noises, churches, cars)
multi-label classification problem — when there are many species singing at the same time
different types of bird songs (as described earlier)
inter-species variance — there might be a difference in birdsong between the same species living in different regions or countries
data set issues — the data can be highly imbalanced due to the bigger popularity of one species over another, there is a large number of different species and recordings can have different length, quality of recordings (volume, cleanliness)

So, how were the problems solved in the past?
Recognizing birds just by their songs might be a difficult task but it does not mean it is not possible. But how to handle those problems?
To find the answer there was a need to dive into research papers and discovered that most of the work happened to be initiated by the various AI challenges, such as BirdCLEF and DCASE. Fortunately, winners of those challenges usually describe their approaches, so after checking the leader boards some interesting insights were obtained:
almost all winning solutions used Convolutional Neural Networks (CNNs) or Recurrent Convolutional Neural Network (RCNNs)
the gap between CNN-based models and shallow, feature-based approaches remained considerably high
even though many of the recordings were quite noisy the CNNs worked well without any additional noise removal and many teams claimed that noise reduction techniques did not help
data augmentation techniques seemed to be widely used, especially the techniques used in audio processing such as time or frequency shift
some winning teams successfully approached it with semi-supervised learning methods (pseudo-labeling) and some increased AUC by model ensemble
But how to apply CNNs, neural networks designed to extract features from images to classify or segment them, when we only have sound recordings? Mel-frequency cepstrum (MFCC) is the answer.

Time to model!
After the creation of mel-spectrograms with high pass filter out of 10s lasting audio files, data were split it into train (90%), validation (10%), and test set (10%).

If you are interested in seeing the code in a jupyter notebook, you can find it here:

https://github.com/m-kortas/Sound-bas...