Multimedia Opinion Mining

A BMBF funded Project from 2015 - 2017
of the MADM group at DFKI

introduction


The Internet is full of opinion hidden under tons of irrelevant and unstructured data. From micro-blogging platforms like Twitter to video repositories such as YouTube the users express sentiments about products, brands, institutions and governments. This project aims to address opinion analysis from a holistic point of view, where text and structured data (i.e. Linked Data from DBPedia, Freebase) together with images and videos are considered in order to derive a more precise opinion assessment.

demos


Capttitude (Captions with Attitude) is a system able to produce affective captions with an emotional component. Two different methods are used, which as a result provide two variants of a text caption with emotional content; a Convolutional Neural Network (CNN) together with a long-short-term-memory (LSTM) network, and in the second approach a graph-based concept and syntax transition (CAST) network.

The project DeepEye faces the challenge of recognizing natural catastrophes in satellite images and to enrich the information with multi-media content gained from the social media. With an image analysis, the different spectral bands of satellite data are combined and the geographic areas affected by a natural catastrophe will be extract.

With the Yahoo Flickr Creative Commons 100 Million (YFCC100m) dataset, a great novel dataset was introduced to the computer vision and multimedia research community. To maximize the benefit for the research community and utilize its potential, this dataset has to be made accessible by tools allowing to search for target concepts within the dataset and mechanism to browse images and videos of the dataset.

We present a novel approach towards sentiment analysis based on the semantic understanding of visual content. To research this goal, we automatically construct a large-scale ontology of semantic concepts.

Every day real-world event happen. These events are reflected in social media and can be detection as trending topics within multiple streams. This demon crawls and archives trending topics from Google, Twitter and Wikipedia and visualizes them to inform us about the current Zeitgeist.


to find PUBLICATIONS related to these demos please click here.

vision


This project aims to address the challenge of opinion mining of multimedia content from the web. This comprises a large-scale multi-modal analysis of social media streams and their underlying network dynamics considering different media channels such as Twitter, YouTube, Flickr, Google, and Wikipedia. The main contribution of MOM is four-fold as illustrated below:

1. Trending Topics

social media data is analyzed utilizing information from structured data sources to detect and track trending topics.

2. Sentiment Analysis

large amounts of multimedia content are analyzed with respect to its sentiment and opinion. This follows a holistic approach i.e. the analysis of a rich set of different modalities such as textual, visual, and tempo-visual content.

3. Social Network Analysis

this information is further enriched by network analysis to grasp structural diffusion as well as global and local impact of information in social networks.

4. Forecasting

the possibility of forecasting the progression of identified opinion topics into the near future is investigated.

team

The following people contribute to the project.


Prof. Dr. Prof. h.c. Andreas Dengel

Head of Department

Dr. Damian
Borth

Project Lead

Dr. Christian
Schulze

Researcher

Jörn
Hees

Researcher

Sebastian
Palacio

Researcher

Joachim
Folz

Researcher

Benjamin
Bischke

PhD Student

Philipp
Blandfort

PhD Student

Andrey
Guzhov

PhD Student

Tushar
Karayil

PhD Student

Federico
Raue

PhD Student

Victor
Campos

Intern

William
Tong

Intern

Prakriti
Bhardway

Master

Carlos
Gallego

Master

Aman
Gautam

Master

Sebastian
Kalkowski

Master

Alireza
Koochali

Master

Hans
Schmitz

Master

Jan Niklas
Böhm

Bachelor

Jessica
Chwalek

HiWi

Shailza
Jolly

HiWi

Magnus
Wiese

HiWi


The following people are former member of the project.

Patrick
Helber

PhD Student

Peter
Burkert

Master

Tobias
Hamann

Master

Tim
Hertweck

Master

Jalpa
Punjabi

Master

Sebastian
Säger

Master

Ahsan
Naeem

Seminar

Rajesh
Chidambaram

HiWi

Ayushman
Dash

HiWi

Mihai
Fieraru

HiWi

Monit Shah
Singh

HiWi

Kshitij
Kumar

Internship

Contact Us

Dr. Damian Borth
German Research Center for Artificial Intelligence
Trippstaedter Straße 122
67663 Kaiserslautern
Germany

Phone: (+49 631 20575 4184)