Скачать или смотреть Using CUSTOM AI to paint Beethoven's Fur Elise's emotions

This video was created using three different AIs:

The first one is a custom built machine learning model built by me, inspired on Chowdhury et al's work (https://archives.ismir.net/ismir2019/.... I used the soundtrack's stimulus set as a database, where the training data is 360 soundtracks classified on 8 different emotion by professionals.

The custom model loads the soundtrack, uses the Bag of Symbolic Fourier Approximation Symbols approach to obtain the features in each song, and then uses a CatBoostRegressor to predict the sentiment of the songs, which essentially outputs 8 different emotions for each song.

Once we predict the sentiment of the songs, we then plug these in into midjourney, using the progress command to let us see how the images are created in real-time.

It is quite interesting to see how some of the images seem to start very similarly, but end up with very different outcomes. It seems that the first 5-8 seconds construct the original image, while the last 5-8 seconds creates all the additional detail and slowly brings up the detail to the desired quality.

The images were later upscaled to 4K using ESRGAN models.