The Internet is full of opinion hidden under tons of irrelevant and unstructured data. From micro-blogging platforms like Twitter to video repositories such as YouTube the users express sentiments about products, brands, institutions and governments. This project aims to address opinion analysis from a holistic point of view, where text and structured data (i.e. Linked Data from DBPedia, Freebase) together with images and videos are considered in order to derive a more precise opinion assessment.
MADM presents at the press event covering Googles engagement with the DFKI
The DFKI announced that Google became a shareholder of the DFKI. The MADM group presented their Eye Tracking technology and Visual Sentiment Analysis (SentiBank) with Deep Convolutional Neural Networks (CNN), which was names as one of the key technologies by Google. In this context, MADM was also covered in an TechCrunch article.
"Financial Data Science" a discipline combining Data Mining and Financial Statistics
A new discipline emerges - “Financial Data Science” - which combines approaches from data mining and financial statistics leveraging large-scale data processing, machine learning, statistics, and financial modeling.
Nowadays the Internet, as a major platform for communication and information exchange, provides a rich repository of people’s opinion and sentiment about a vast spectrum of topics. Such knowledge is embedded in multiple facets, such as comments, tags, browsing actions, as well as shared media objects. The analysis of such information either in the area of opinion mining, affective computing or sentiment analysis plays an important role in behavior sciences, which aims to understand and predict human decision making and enables applications such as brand monitoring, stock market prediction, or political voting forecasts. (read more)
"Multimedia Opinion Mining for Social Good" presented at UN PRI Event
In the context of the BMBF sponsored project “Multimedia Opinion Mining”, Dr. Damian Borth, gave a guest lecture at the ICMA Center of the Henley Business School about how tools developed at MADM can be used for social good. The lecture presented novel way of mining social multimedia to gain insights about critical issues around environmental, social, and governance issues.
Dr. Damian Borth part of assessment committee for the Investment Innovation Benchmark (IIB)
Dr. Damian Borth served as a member of the assessment committee at the Investment Innovation Benchmark (IIB) summit in London. Damian was appointed to the committee due to his background in machine learning and large-scale data processing and will serve as an expert to evaluate FinTech submissions. The assessment committee consists of member from academia and some of the largest pension fund worldwide.
"YFCC100M: The New Data in Multimedia Research" got accepted at CACM
Our paper describing the 100 million images and videos Flickr dataset: “YFCC100M: The New Data in Multimedia Research” got accepted at “Communication of the ACM”. The paper describes the up-to-date largest dataset in the computer vision and multimedia research community. This work was a collaboration between Yahoo Inc., ICSI Berkeley, and the Lawrence Livermore National Laboratory. A pre-print is available on Arxiv.org.
Interview with Hannoversche Allgemeine Zeitung about Computer Vision Research
Dr. Damian Borth gave an Interview to Hannoversche Allgemeine Zeitung about the success of deep learning in computer vision research, recent progress to recognize sentiment and emotions in visual content, and how artificial intelligence (AI) can be used for social good.
Dr. Damian Borth moderates a Technology Panel at Bloomberg, New York, USA
With the advent of FinTech, disciplines like machine learning, natural language processing, and large-scale data mining are moving into the finance and investment industry. To discuss the potential of FinTech, Dr. Damian Borth was invited to moderate a Technology Panel at FinTech Bloomberg event. The panel was including CEOs and CTOs of leading startups in the area such as: eRevalue, TruValue and Bloomberg Labs.
Dr. Damian Borth speaks at RI Europe about Financial Data Science, London, UK
Dr. Damian Borth speaks about Big Data and Environmental, Social and Governance research and introduces together with Prof. Andreas Hoepner the concept of “Financial Data Science” at the RI Europe - a major conference in the responsible investment community. Financial Data Science embraces the processing of large datasets with machine learning methods and welcomes approaches such as opinion mining from public available data on the web.
Call for paper for Multimedia COMMONS Workshop is out
The CfP to the Multimedia COMMONS workshop (MMCommons) is out! We invite the community to attend MMCommons and explore the possibilities for novel research, future data challenges, and new benchmarks offered by this large-scale open dataset (YFCC100m dataset). The MMCommons will be held at the ACM Multimedia 2015, Brisbane, Australia around October 26-30.
Dr. Damian Borth appointed new head of MADM group
By April 2015, Damian Borth became the new head of MADM. Before this position Damian was a postdoctoral research fellow at the International Computer Science Institute (ICSI) with Dr. Gerald Friedland and at UC Berkeley with Prof. Trevor Darrell where he was involved in various project sponsored by DARPA, IARPA, and the Lawrence Livermore National Laboratory. His research focuses concept detection from social multimedia including trending topic detection, visual sentiment analysis, and multimedia opinion mining.
Invited talk about "Data Science and Opinion Mining" at the London School of Economics, UK
The project Multimedia Opinion Mining (MOM) was presented in the context of Data Science during an UN PRI event at the London School of Economics. Dr. Damian Borth introduced data science and opinion mining and outlined how it can be used for social good. The event was later interrupted by an evacuation due to the Holborn Fire in London.
Dr. Damian Borth gives an invited talk at the University of Reading, UK
Dr. Damian Borth speaks about “Mining Insights from Public Data” at an event of the University of Reading and outlines concepts and goals of the Multimedia Opinion Mining (MOM) project to the audience.
Dr. Damian Borth invited to BMBF dinner with Dr. Georg Schütte in Berkeley, USA
During a visit of the Bundesministerium für Forschung und Entwicklung (BMBF) in the Bay Area, the BMBF invited DAAD scholarship researcher from ICSI to join an exclusive dinner with Deputy Secretary Dr. Georg Schütte. Dr. Damian Borth introduces the BMBF funded project Multimedia Opinion Mining (MOM) to the delegation during the dinner.
BMBF funded project Multimedia Opinion Mining (MOM) starts
Multimedia Opinion Mining (MOM), a 2.5 years long project funded by the BMBF, aims to address the challenge of opinion mining in social multimedia content. The main goals of MOM are: (1) analyze social media data to detect and track trending topics. (2) understand multimedia content with respect to its sentiment and opinion. (3) discover individual persons or groups able to influence propagation of topics in social networks. And finally, (4) forecasting the progression of identified topics into the near future. The project will be coordinated within MADM and lead by Dr. Damian Borth.
VISIONThis project aims to address the challenge of opinion mining of multimedia content from the web. This comprises a large-scale multi-modal analysis of social media streams and their underlying network dynamics considering different media channels such as Twitter, YouTube, Flickr, Google, and Wikipedia. The main contribution of MOM is four-fold as illustrated below:
|Sebastian Sager, Damian Borth, Benjamin Elizalde, Christian Schulze, Bhiksha Raj, Ian Lane, Andreas Dengel. AudioSentibank: Large-scale Semantic Ontology of Acoustic Concepts for Audio Content Analysis. In arXiv preprint arXiv:1607.03766, 2016.
|Bart Thomee, Damian Borth, Julia Bernd. Multimedia COMMONS Workshop 2016 (MMCommons’16) - Datasets, Evaluation, and Reproducibility. In ACM Multimedia Workshops, 2016.
|Philipp Blandfort, Tushar Karayil, Damian Borth, Andreas Dengel. Generating Captions with Emotions using Concept and Syntax Transition Networks. In ACM Multimedia, Grand Challenge, 2016 (accepted). |
|Benjamin Bischke, Damian Borth, Christian Schulze, Andreas Dengel. Contextual Enrichment of Remote-Sensed Events with Social Media Streams. In ACM Multimedia, Grand Challenge, 2016 (accepted).|
|Alireza Koochali, Sebastian Kalkowski, Damian Borth, Christan Schulze, Andreas Dengel. Which languages do people speak on Flickr? A Language and Geo-Location Study of the YFCC100m Dataset. In
ACM Multimeda Community-Organized Multimodal Mining: Opportunities for Novel Solutions (MMCOMMONS) Workshop, 2016 (accepted).|
|Philipp Blandfort, Tushar Karayil, Damian Borth, Andreas Dengel. Introducing Concept And Syntax Transition Networks for Image Captioning. In ACM Int. Conf. on Multimedia Retrieval (ICMR), 2016.|
|Jörn Hees, Rouven Bauer, Joachim Folz, Damian Borth, Andreas Dengel. An Evolutionary Algorithm to Learn SPARQL Queries: Finding Patterns for Human Associations in DBpedia. In arXiv preprint arXiv:1607.07249, 2016. |
|Sebastian Fuchs, Damian Borth and Adrian Ulges. Trending Topic Aggregation by News-based Context Modeling. In KI, 2016 (accepted).|
|[TSF+16]||Bart Thomee, David Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Dough Poland, Damian Borth, Li-Jia Li.YFCC100m: The New Data in Multimedia Research. In Communication of the ACM, 2016. (YFCC100m Dataset Paper)|
|Julia Bernd, Damian Borth, Carmen Carrano, Jaeyoung Choi, Benjamin Elizalde, Gerald Friedland, Luke Gottlieb, Karl Ni, Roger Pearce, Doug Poland, Khalid Ashraf, David A. Shamma, and Bart Thomee. Kickstarting the Commons: The YFCC100M and the YLI Corpora. In ACM Multimeda Community-Organized Multimodal Mining: Opportunities for Novel Solutions (MMCOMMONS) Workshop, 2015. |
|[KSDB15]||Sebastian Kalkowski, Christian Schulze, Andreas Dengel, Damian Borth. Real-time Analysis and Visualization of the YFCC100m Dataset. In ACM Multimeda Community-Organized Multimodal Mining: Opportunities for Novel Solutions (MMCOMMONS) Workshop, 2015.|
|[ANCB+15]||Mohammad Al-Naser, Seyyed Saleh Mozaffari Chanijani, Syed Saqib Bukhari, Damian Borth, Andreas Dengel. What makes a Beautiful Landscape beautiful: Adjective Noun Pairs Attention by Eye-Tracking and Gaze Analysis. In ACM Workshop on Affect and Sentiment in Multimedia (ASM), 2015.|
|[FSBD15]||Joachim Folz, Christian Schulze, Damian Borth, Andreas Dengel. Aesthetic Photo Enhancement using Machine Learning and Case-Based Reasoning.
In ACM Workshop on Affect and Sentiment in Multimedia (ASM), 2015.|
|Bart Thomee, David Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Dough Poland, Damian Borth, Li-Jia Li. The New Data and New Challenges in Multimedia Research. In arXiv, preprint arXiv:1503.01817, March, 2015. |
|Karl Ni, Roger Pearce, Kofi Boakye, Brian Van Essen, Damian Borth, Barry Chen, Eric Wang. Large-Scale Deep Learning on the YFCC100M Dataset. In arXiv, preprint arXiv:1502.03409, February, 2015.|
The following people contributed to the project (past and current).
Prof. Dr. Prof. h.c. Andreas Dengel
Dr. Damian Borth
Dr. Christian Schulze
Monit Shah Singh
German Research Center for Articial Intelligence