INTRODUCTION

The Internet is full of opinion hidden under tons of irrelevant and unstructured data. From micro-blogging platforms like Twitter to video repositories such as YouTube the users express sentiments about products, brands, institutions and governments. This project aims to address opinion analysis from a holistic point of view, where text and structured data (i.e. Linked Data from DBPedia, Freebase) together with images and videos are considered in order to derive a more precise opinion assessment.

NEWS


06.10.2015:


MADM presents at the press event covering Googles engagement with the DFKI



The DFKI announced that Google became a shareholder of the DFKI. The MADM group presented their Eye Tracking technology and Visual Sentiment Analysis (SentiBank) with Deep Convolutional Neural Networks (CNN), which was names as one of the key technologies by Google. In this context, MADM was also covered in an TechCrunch article.


15.09.2015:


"Financial Data Science" a discipline combining Data Mining and Financial Statistics



A new discipline emerges - “Financial Data Science” - which combines approaches from data mining and financial statistics leveraging large-scale data processing, machine learning, statistics, and financial modeling.


Nowadays the Internet, as a major platform for communication and information exchange, provides a rich repository of people’s opinion and sentiment about a vast spectrum of topics. Such knowledge is embedded in multiple facets, such as comments, tags, browsing actions, as well as shared media objects. The analysis of such information either in the area of opinion mining, affective computing or sentiment analysis plays an important role in behavior sciences, which aims to understand and predict human decision making and enables applications such as brand monitoring, stock market prediction, or political voting forecasts. (read more)

07.09.2015:


"Multimedia Opinion Mining for Social Good" presented at UN PRI Event



In the context of the BMBF sponsored project “Multimedia Opinion Mining”, Dr. Damian Borth, gave a guest lecture at the ICMA Center of the Henley Business School about how tools developed at MADM can be used for social good. The lecture presented novel way of mining social multimedia to gain insights about critical issues around environmental, social, and governance issues.

04.09.2015:


Dr. Damian Borth part of assessment committee for the Investment Innovation Benchmark (IIB)



Dr. Damian Borth served as a member of the assessment committee at the Investment Innovation Benchmark (IIB) summit in London. Damian was appointed to the committee due to his background in machine learning and large-scale data processing and will serve as an expert to evaluate FinTech submissions. The assessment committee consists of member from academia and some of the largest pension fund worldwide.


03.08.2015:


"YFCC100M: The New Data in Multimedia Research" got accepted at CACM



Our paper describing the 100 million images and videos Flickr dataset: “YFCC100M: The New Data in Multimedia Research” got accepted at “Communication of the ACM”. The paper describes the up-to-date largest dataset in the computer vision and multimedia research community. This work was a collaboration between Yahoo Inc., ICSI Berkeley, and the Lawrence Livermore National Laboratory. A pre-print is available on Arxiv.org.


26.07.2015:


Interview with Hannoversche Allgemeine Zeitung about Computer Vision Research



Dr. Damian Borth gave an Interview to Hannoversche Allgemeine Zeitung about the success of deep learning in computer vision research, recent progress to recognize sentiment and emotions in visual content, and how artificial intelligence (AI) can be used for social good.


25.6.2015:


Dr. Damian Borth moderates a Technology Panel at Bloomberg, New York, USA



With the advent of FinTech, disciplines like machine learning, natural language processing, and large-scale data mining are moving into the finance and investment industry. To discuss the potential of FinTech, Dr. Damian Borth was invited to moderate a Technology Panel at FinTech Bloomberg event. The panel was including CEOs and CTOs of leading startups in the area such as: eRevalue, TruValue and Bloomberg Labs.

02.06.2015:


Dr. Damian Borth speaks at RI Europe about Financial Data Science, London, UK



Dr. Damian Borth speaks about Big Data and Environmental, Social and Governance research and introduces together with Prof. Andreas Hoepner the concept of “Financial Data Science” at the RI Europe - a major conference in the responsible investment community. Financial Data Science embraces the processing of large datasets with machine learning methods and welcomes approaches such as opinion mining from public available data on the web.


14.04.2015:


Call for paper for Multimedia COMMONS Workshop is out



The CfP to the Multimedia COMMONS workshop (MMCommons) is out! We invite the community to attend MMCommons and explore the possibilities for novel research, future data challenges, and new benchmarks offered by this large-scale open dataset (YFCC100m dataset). The MMCommons will be held at the ACM Multimedia 2015, Brisbane, Australia around October 26-30.

01.04.2015:


Dr. Damian Borth appointed new head of MADM group



By April 2015, Damian Borth became the new head of MADM. Before this position Damian was a postdoctoral research fellow at the International Computer Science Institute (ICSI) with Dr. Gerald Friedland and at UC Berkeley with Prof. Trevor Darrell where he was involved in various project sponsored by DARPA, IARPA, and the Lawrence Livermore National Laboratory. His research focuses concept detection from social multimedia including trending topic detection, visual sentiment analysis, and multimedia opinion mining.

01.04.2015:


Invited talk about "Data Science and Opinion Mining" at the London School of Economics, UK



The project Multimedia Opinion Mining (MOM) was presented in the context of Data Science during an UN PRI event at the London School of Economics. Dr. Damian Borth introduced data science and opinion mining and outlined how it can be used for social good. The event was later interrupted by an evacuation due to the Holborn Fire in London.


31.03.2015:


Dr. Damian Borth gives an invited talk at the University of Reading, UK



Dr. Damian Borth speaks about “Mining Insights from Public Data” at an event of the University of Reading and outlines concepts and goals of the Multimedia Opinion Mining (MOM) project to the audience.

18.03.2015:


Dr. Damian Borth invited to BMBF dinner with Dr. Georg Schütte in Berkeley, USA



During a visit of the Bundesministerium für Forschung und Entwicklung (BMBF) in the Bay Area, the BMBF invited DAAD scholarship researcher from ICSI to join an exclusive dinner with Deputy Secretary Dr. Georg Schütte. Dr. Damian Borth introduces the BMBF funded project Multimedia Opinion Mining (MOM) to the delegation during the dinner.

01.03.2015:


BMBF funded project Multimedia Opinion Mining (MOM) starts



Multimedia Opinion Mining (MOM), a 2.5 years long project funded by the BMBF, aims to address the challenge of opinion mining in social multimedia content. The main goals of MOM are: (1) analyze social media data to detect and track trending topics. (2) understand multimedia content with respect to its sentiment and opinion. (3) discover individual persons or groups able to influence propagation of topics in social networks. And finally, (4) forecasting the progression of identified topics into the near future. The project will be coordinated within MADM and lead by Dr. Damian Borth.

DEMOS







VISION

This project aims to address the challenge of opinion mining of multimedia content from the web. This comprises a large-scale multi-modal analysis of social media streams and their underlying network dynamics considering different media channels such as Twitter, YouTube, Flickr, Google, and Wikipedia. The main contribution of MOM is four-fold as illustrated below:

1. Trending Topics

social media data is analyzed utilizing information from structured data sources to detect and track trending topics.

2. Sentiment Analysis

large amounts of multimedia content are analyzed with respect to its sentiment and opinion. This follows a holistic approach i.e. the analysis of a rich set of different modalities such as textual, visual, and tempo-visual content.

3. Social Network Analysis

this information is further enriched by network analysis to grasp structural diffusion as well as global and local impact of information in social networks.

4. Forecasting

the possibility of forecasting the progression of identified opinion topics into the near future is investigated.

Research Activities

The MOM project defines the following research activities and work packages.

PUBLICATIONS

  • AudioSentibank: Large-scale Semantic Ontology of Acoustic Concepts for Audio Content Analysis


    Sebastian Sager, Damian Borth, Benjamin Elizalde, Christian Schulze, Bhiksha Raj, Ian Lane, Andreas Dengel
    arXiv preprint arXiv:1607.03766, 2016

    [paper]

  • Multimedia COMMONS Workshop 2016 (MMCommons’16) - Datasets, Evaluation, and Reproducibility


    Bart Thomee, Damian Borth, Julia Bernd
    ACM Multimedia Workshops, 2016

    [website]

  • Generating Captions with Emotions using Concept and Syntax Transition Networks


    Philipp Blandfort, Tushar Karayil, Damian Borth, Andreas Dengel
    ACM Multimedia, Grand Challenge, 2016 (accepted)

    [challenge website]

  • Contextual Enrichment of Remote-Sensed Events with Social Media Streams


    Benjamin Bischke, Damian Borth, Christian Schulze, Andreas Dengel
    ACM Multimedia, Grand Challenge, 2016 (accepted)

    [website]

  • Which languages do people speak on Flickr? A Language and Geo-Location Study of the YFCC100m Dataset


    Alireza Koochali, Sebastian Kalkowski, Damian Borth, Christan Schulze, Andreas Dengel
    ACM Multimeda Community-Organized Multimodal Mining: Opportunities for Novel Solutions (MMCOMMONS) Workshop, 2016 (accepted)

    [YFCC100m Browser]

  • Introducing Concept And Syntax Transition Networks for Image Captioning


    Philipp Blandfort, Tushar Karayil, Damian Borth, Andreas Dengel
    ACM Int. Conf. on Multimedia Retrieval (ICMR), 2016
  • An Evolutionary Algorithm to Learn SPARQL Queries: Finding Patterns for Human Associations in DBpedia


    Jörn Hees, Rouven Bauer, Joachim Folz, Damian Borth, Andreas Dengel
    arXiv preprint arXiv:1607.07249, 2016

    [paper]

  • Trending Topic Aggregation by News-based Context Modeling


    Sebastian Fuchs, Damian Borth and Adrian Ulges
    KI, 2016 (accepted)
  • YFCC100m: The New Data in Multimedia Research


    Bart Thomee, David Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Dough Poland, Damian Borth, Li-Jia Li
    Communication of the ACM, 2016, (YFCC100m Dataset Paper)

    [paper] [watch video] [YFCC100m Browser]

  • Kickstarting the Commons: The YFCC100M and the YLI Corpora


    Julia Bernd, Damian Borth, Carmen Carrano, Jaeyoung Choi, Benjamin Elizalde, Gerald Friedland, Luke Gottlieb, Karl Ni, Roger Pearce, Doug Poland, Khalid Ashraf, David A. Shamma, and Bart Thomee
    ACM Multimeda Community-Organized Multimodal Mining: Opportunities for Novel Solutions (MMCOMMONS) Workshop, 2015
  • Real-time Analysis and Visualization of the YFCC100m Dataset


    Sebastian Kalkowski, Christian Schulze, Andreas Dengel, Damian Borth
    ACM Multimeda Community-Organized Multimodal Mining: Opportunities for Novel Solutions (MMCOMMONS) Workshop, 2015

    [YFCC100m Browser]

  • What makes a Beautiful Landscape beautiful: Adjective Noun Pairs Attention by Eye-Tracking and Gaze Analysis


    Mohammad Al-Naser, Seyyed Saleh Mozaffari Chanijani, Syed Saqib Bukhari, Damian Borth, Andreas Dengel
    ACM Workshop on Affect and Sentiment in Multimedia (ASM), 2015
  • Aesthetic Photo Enhancement using Machine Learning and Case-Based Reasoning


    Joachim Folz, Christian Schulze, Damian Borth, Andreas Dengel
    ACM Workshop on Affect and Sentiment in Multimedia (ASM), 2015
  • The New Data and New Challenges in Multimedia Research


    Bart Thomee, David Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Dough Poland, Damian Borth, Li-Jia Li
    arXiv, preprint arXiv:1503.01817, March, 2015

    [preprint]

  • Large-Scale Deep Learning on the YFCC100M Dataset


    Karl Ni, Roger Pearce, Kofi Boakye, Brian Van Essen, Damian Borth, Barry Chen, Eric Wang
    arXiv, preprint arXiv:1502.03409, February, 2015

    [paper]

PEOPLE

The following people contributed to the project (past and current).

Prof. Dr. Prof. h.c. Andreas Dengel

Fachbereichsleiter

    Dr. Damian Borth


    Project Lead

      Dr. Christian Schulze

      Researcher

        Jörn Hees

        Researcher

          Sebastian Palacio

          Researcher

            Joachim Folz

            Junior Researcher

              Philipp Blandfort

              PhD student

                Tushar Karayil

                PhD student

                  Prakriti Bhardwaj

                  Master

                    Benjamin Bischke

                    Master

                      Peter Burkert

                      Master

                        Tobias Hamann

                        Master

                          Sebastian Kalkowski

                          Master

                            Alireza Koochali

                            Master

                              Jalpa Punjabi

                              Master

                                Ahsan Naeem

                                Master

                                  Sebastian Säger

                                  Master

                                    Rajesh Chidambaram

                                    HiWi

                                      Jessica Chwalek

                                      HiWi

                                        Mihai Fieraru

                                        HiWi

                                          Aman Gautam

                                          HiWi

                                            Monit Shah Singh

                                            HiWi

                                              Kshitij Kumar

                                              Internship

                                                CONTACT









                                                • Dr. Damian Borth
                                                • Trippstaedter Str. 122
                                                  67663 Kaiserslautern
                                                  Germany
                                                • +49 (631) 20575-4184
                                                • damian.borth@dfki.de