Скачать или смотреть Dark Vision: Video Frame Extracting and Tagging

Video should soon represent up to 90% of all consumer internet traffic. This is a lot of information, often referred to as “dark data”, that is not simply searchable like a row in a database.

What if we could apply the same technologies to videos to make sense of these “dark data”? That’s what I did, helped again by IBM Cloud and Watson services, including Natural Language Understanding, Speech to Text, Visual Recognition, Cloudant etc.

Also leveraging the IBM Github project called Dark data . The sample application, called Dark Vision, processes videos by extracting frames and tagging these frames independently. No custom models were created to train against; so the results are just default. Here are three images uploaded. The first being that of a fruit basket. Keywords for the image include banana, fruit, food, melon, olive colour and lemon yellow color.

The second image of Ashton Kutcher, and the image keywords are male, person, official, investor. A second female face is also identified in the background. The third image of Ginni Rometty, is tagged with woman portrait photo, person, blue coat. With face detection of 87% female. Two YouTube videos from GeoBeats were used for the video analysis. Both being approximately 90 seconds in length. The first being on New York. From the video, image keywords with percentages are listed, with audio keywords. Concepts are listed and emotional analysis from the transcript. The video transcript is also available. Frames can be searched for keywords like tower, watercraft, sky etc. Going to the second video of Paris.

The full transcript is again available, with Audio key words. The emotion is also analysed. Concepts are identified like Paris, Mona Lisa, Louvre etc. Again the video can be searched with words like Tower, arch, Eiffel.

This is a good example how the IBM Cloud and Watson elements can be orchestrated to fashion unique solutions to real world problems.