With the Yahoo Flickr Creative Commons 100 Million (YFCC100m) dataset, a great novel dataset was introduced to the computer vision and multimedia research community. To maximize the benefit for the research community and utilize its potential, this dataset has to be made accessible by tools allowing to search for target concepts within the dataset and mechanism to browse images and videos of the dataset.

Every day real-world event happen. These events are reflected in social media and can be detection as trending topics within multiple streams. This demon crawls and archives trending topics from Google, Twitter and Wikipedia and visualizes them to inform us about the current Zeitgeist.

We present a novel approach towards sentiment analysis based on the semantic understanding of visual content. To research this goal, we automatically construct a large-scale ontology of semantic concepts.

Capttitude (Captions with Attitude) is a system able to produce affective captions with an emotional component. Two different methods are used, which as a result provide two variants of a text caption with emotional content; a Convolutional Neural Network (CNN) together with a long-short-term-memory (LSTM) network, and in the second approach a graph-based concept and syntax transition (CAST) network.

The project DeepEye faces the challenge of recognizing natural catastrophes in satellite images and to enrich the information with multi-media content gained from the social media. With an image analysis, the different spectral bands of satellite data are combined and the geographic areas affected by a natural catastrophe will be extract.

Image is not available


The Internet is full of opinion hidden under tons of irrelevant and unstructured data. From micro-blogging platforms like Twitter to video repositories such as YouTube the users express sentiments about products, brands, institutions and governments. This project aims to address opinion analysis from a holistic point of view, where text and structured data (i.e. Linked Data from DBPedia, Freebase) together with images and videos are considered in order to derive a more precise opinion assessment.


Dr. Damian Borth giving a keynote in Bern at the Swiss AI Day

Significant developments in the use of AI can currently be observed in the health sector, insurances and finance institutions. For this, Dr. Damian Borth and many more gather at the Swiss AI Day in Bern to give talks about the future of AI in different industries. Dr. Borth in particular, gave the opening keynote about Deep Learning as one element of AI’s future.

Dr. Damian Borth at Rise of AI Conference, Berlin

Giving a keynote at Rise Of AI Conference in Berlin, Dr. Damian Borth talks about the future of AI in society and industries. Together with many other contributors, this year’s conference takes place in the Deutsches Technikmuseum in Berlin. A video of the Borth’s keynote can be found here [X].

Dr. Damian Borth gives An Interview to the “Karriereführer” magazine

Dr. Damian Borth gives the outlook for Deep Learning as a game changer for the business world as the Internet once was. In an interview for the Karriereführer Dr. Borth explains how important Deep Learning is for several industries as well as how it might optimize sales.

Dr. Damian Borth gives interview for for Terra X documentary

The german TV program Terra X (ZDF) is interviewing Dr. Damian Borth as part of its upcoming documentary. The documentary aims to reveals the history and present effects of technology and science on society, also giving an outlook on Artificial Intelligence and its impact to our world.

Dr. Damian Borth gives a Talk at the Kreativviti Conference, Pirmasens

At the Trade Fair for Innovation, Creative and Cultural Industries (Kreativviti Conference) in Pirmasens, Dr. Damian Borth has been present to talk about the topic “AI - Artificial artist or creative machine?”. A full program in german can be found here [X].

DFKI Deep Learning Competence Center (DLCC) at the CeBit 2017

The DFKI has been represented in various ways at this year's CeBit in Hannover. The Deep Learning Competence Center was presenting work in the areas of text, image, video and social media analysis at the DFKI exhibition booth. Deep Learning as subarea of ​​Artificial Intelligence is currently responsible for remarkable breakthroughs. The DLCC has presented its exhibits DeepEye - Deep Learning for disaster protection and crisis management - and Capttitude, a deep learning process for the generation of image texts with emotions.

Dr. Damian Borth gives Future Talk at the CeBit 2017

Dr. Damian Borth also has been invited to giving a talk at the CeBit 2017 as part of the “FutureTalk: the innovative conference forum” series. The well-attended talks presented the DFKI Deep Learning Competence Center and the latest developments in the area of deep learning. In particular, the work on Generative Adversarial Networks (GAN) was of interest. With the topic “Deep Learning: What it is and Why now?”, Dr. Damian Borth has been inviting the audience after the talk to an open discussion about the topic.

Dr. Damian Borth has been invited to give a keynote at Amazon in Luxembourg

Dr. Damian Borth was invited by Amazon Web Services (AWS) to Luxembourg to give a keynote about Artificial Intelligence, Machine Learning, and Deep Learning. The keynote was given in the context of a workshop at Amazon where several AWS cloud computing tools have been extended by machine learning approaches.

Dr. Damian Borth at Empolis Executive Forum in Berlin

The annual Empolis Executive Forum held in Berlin has invited Dr. Damian Borth to give the event’s opening keynote. With the theme of the event: “Turning Data Into Decisions - How Machine Learning turns Big Data into Big Business”, Dr. Damian Borth’s talk "Machine Intelligence through Deep Learning - Learning to understand Data" has contributed to the discussion about how industry can use the latest achievements of Machine Learning for their business. A column by Holger Schmidt is found here [x].

‘Visual Sentiment Analysis with Deep Convolutional Neural Networks’ - Dr. Damian Borth at the Deep Learning Meetup Munich

Dr. Damian Borth gives a talk about his research on Visual Sentiment Analysis with Deep Convolutional Neural Networks (CNN). On social networks sites like Twitter, Facebook & Co. communication is turning more and more towards visual content such as with images and videos. Being able to extract sentiment from visual content is a challenging task as presented during Dr. Borth’s talk at the Deep Learning Meetup in Munich. He introduced the Visual Sentiment Ontology (VSO) which consists of thousands of Adjective Noun Pair (ANP) concepts. Further, he talked about SentiBank and an associated deep convolutional neural network (CNN), which they use to find up to 2089 ANPs in images. This framework is able to analyse the sentiments of visual contents much better then by texts. If you want to know more about this research, please have a look at the DFKI-Newsfeed. The Meetup website you can find here.

Dr. Damian Borth give invited keynote at the Allianz Global Investors (AGI) EMEA Meeting in Berlin

The Allianz Global Investor (AGI) asset manager has invited Dr. Damian Borth for a keynote on the future of artificial intelligence to his annual EMEA Meeting in Berlin at the Pariser Platz. In addition to Dr. Borth's keynote “Our Future with Artificial Intelligence”, the day's event was also opened by Prof. Michael Hüthner, Director of the Cologne Institute for Economic Research. The main theme of the event was digitalization in the financial world and the potential of artificial intelligence as a driving force for the economy.

Dr. Damian Borth gives an invited keynote at the Allianz Forum by AIM

The annual “Allianz Forum” has set the 2017 topic to new markets, risks in investment strategies, and the future of investment. In this context, Dr. Damian Borth has hold a keynote presentation about Deep Learning with the topic: “The New Artificial Intelligence: What It Can Do And What Not”. During this event Jürgen Stark (EZB) was speaking, as well as Ruchir Sharma (Morgan Stanley).

Dr. Damian Borth became a member of the "Steering Group: AI and Society" of the Volkswagen Foundation

Dr. Damian Borth has been named into the “Steering Group: AI and society” of the Volkswagen Foundation as one of five experts. In addition to professors from Freiburg, Munich, Pforzheim and Konstanz, Dr. Borth represents the DFKI in this topic. “The current developments, which is expected in the context of the digitization of society, have the potential to transform society in a substantial way,” states the foundation. The aim of the Steering Group is to define a funding program devoted to this theme for 2018. The Volkswagen Foundation is an independent non-profit foundation based in Hannover. With a funding volume of around 150 million euros per year it is the largest private German science-promoting foundation. In more than 50 years of its existence the Volkswagen Foundation has supported more than 30,000 projects with a total of more than 4.2 billion euros.

Dr. Damian Borth at the World Economic Forum, Switzerland

Dr. Damian Borth participated at this year’s World Economic Forum in Davos. Invited by the ZEIT Dr. Borth discussed the current affairs in the field of Artificial Intelligence in Switzerland which is recognized as an more importantly growing topic of global interest in the scope of politics, economic growth, and societal impact.

‘Deep Learning: the Revolution in artificial intelligence’ - Dr. Damian Borth at the Handelsblatt annual conference

Digitization, innovation and agility are the keywords of this year's Handelsblatt annual meeting for Strategic IT Management 2017. These keywords do not only describe the ever-increasing amount of data that companies have to deal with, but also the innovation pressure on companies such as Artificial Intelligence, Machine Learning and Data Science. As the director of the Deep Learning Competence Center at the DFKI, Dr. Damian Borth has been invited to speak about "Deep Learning: The Revolution in Artificial Intelligence" in front of 250 CIO from all over Europe. In the words of Mrs. Ilse Aigner, Dr. Borth was able to introduce the DFKI as innovation provider in Germany and establish the broad topic of Deep Learning as a disruptive force in various industries. DFKI was thus the only research institute from Germany to be present alongside the CIO / CDOs of VW, Volvo, Ford, Porsche, Continental, AXA, Bosch and others. Together with many more speakers from several special fields the conference is being held until 25th January. A program brochure in english is available [X], as well as an interview with Dr. Borth on YouTube.

Dr. Damian Borth gives a keynote at US-Germany Artificial Intelligence Policy Workshop in Washington DC, USA

With the White House releasing a follow-up report of the October report about AI transforming our society, Dr. Damian Borth has been invited by the German Embassy in Washington DC to give a keynote about state-of-the-art of Artificial Intelligence and Deep Learning at the German-US AI Policy workshop.The discussions during the workshop at New America follows the White House ‘Artificial Intelligence, Automation, and the Economy’ report’s line of argumentation to embrace Artificial Intelligence as influencing our daily life and disrupting our industries. Dr. Damian Borth’s discussion focus was on the need for smart regulation in the artificial intelligence space.

Dr. Damian Borth gives an invited talk at Invesco

At this year's quand day Dr. Damian Borth was invited to give a talk about the potential of Artificial Intelligence, Machine Learning, and Deep Learning extending the domain of Financial Data Science at Invesco in Frankfurt am Main.

Tim Althoff gives talk at the DFKI Deep Learning Competence Center

Tim Althoff - PhD candidate from Stanford - is visiting the DFKI in Kaiserslautern to give a talk about his research topic Leveraging Data Science and Social Networks To Improve Human Well-Being and Health, in which he presents a new method based in network analysis, data mining, causal inference and computational linguistic to harness the power of massive data to improve human health. For this he addresses how to leverage large-scale datasets from several sources to study if and how mobile health applications and online social communities can improve health and well-being overall.

Dr. Damian Borth moderates the Machine Learning panel at VDI CESIS 2016 in Berlin

Dr. Damian Borth has the great honor to moderate an expert panel discussion about machine learning including Dr. Ralph Herbrich (Managing Director at Amazon Development Center, Germany), Michael Adendor (Senior Technical Staff Member, Architect – Predictive Maintenance and Quality, Watson IoT, IBM, Canada) and Sami Abu-El-Haija (Sami Abu-El-Haija, Machine Learning and Computer Vision, Google Research, USA) as participants. The panel discussion at VDI CESIS 2016 conference focused on recent advances in machine learning and its role in the cloud.

Dr. Damian Borth has been selected as Part of the BW Foundation “Neurorobotik” Review Committee

The Baden-Würtemberg Foundation has selected Dr. Damian Borth as one memeber of the “Neurorobotik” program review committee. Neurobotik (‘Newrobotic’) is an interdisciplinary topic which gathers the aspects of neuroscience, robotics and computer science. The Baden-Württemberg foundation has granted an overall budget of EUR 4 Mio for the program.

Dr. Damian Borth gives invited talk at the Daimler Digital Life Day

Dr. Damian Borth has been invited to Daimler’s Digital Life Tech Talk Event in Stuttgart. Speaking about Artificial Intelligence as driving force of digitalization Dr. Damian Borth has presented recent developments and projects of the DFKI Deep Learning Competence Center. Impressions can be found here [X].

Dr. Damian Borth gives an invited talk at the MENSA Germany MIND Event

Providing an insight look at current research activities at the Deep Learn Learning Competence Center at the DFKI, Dr. Damian Borth has been talking at the MENSA annual MIND Academie event which took place this year in Mannheim.

Dr. Damian Borth participated at the Deep Learning Panel of the ACM Multimedia

Dr. Damian Borth was invited as one of four participants to the Deep Learning Panel discussion at ACM Int. Conference on Multimedia. The panel discussed the recent momentum of deep learning in context of media technology research.

Invited Talk at Karlsruhe Institute of Technology (KIT) about Deep Learning

Dr. Damian Borth was invited by the faculty of architecture to speak about deep learning and artificial intelligence at the “Architecture meets Technology Symposium” at KIT in Karlsruhe.

The Investment Innovation Benchmark (IIB) invited Dr. Damian Borth to give a keynote about Artificial Intelligence

The IIB summit in Stockholm invited Dr. Damian Borth to give a keynote about the topic: “The Global Pulse: Addressing Sustainable Development Goals with Artificial Intelligence”. In particular the usage of satellite image data was discussed as disrupting force in this area. In parallel, the investment journal “Responsible Investor” covers Dr. Damian Borths research in a features article focusing on the usage of AI for Good.

UC Berkeley’s Prof. Stella Yu visits DFKI DLCC

Prof. Stelly Yu from UC Berkeley visits DFKI Deep Learning Competence Center (DLCC) to give a talk about deep learning and satellite image analysis and discuss potential collaborations with DFKI DLCC and UC Berkeley. More infos to Stella Yu here [x] [x].

Dr. Damian Borth gives a TED talk at TEDxStuttgart

With this year’s theme ‘(e)Motion’ Dr. Damian Borth was honored to give a TED talk at the TEDxStuttgart about “Our future with Artificial Intelligence” and the question what if machines would become aware of themselves and how this would transform the current understanding of us as humans. To read more about the event and to watch Dr. Damian Borth’s talk click here [X].

Third Financial Data Science Association Conference in Frankfurt

Discussing the topic of why big data and artificial intelligence being ready to disrupt investment security markets, Dr. Damian Borth provides insights about this topics at the third Financial Data Association Conference. In particular the issue of artificial intelligence vs. human intelligence, big data processing and analytics for the finance industry were discussed in his talk and panel discussion about Financial Data Science.

Dr. Damian Borth gives a talk at GTC Europe in Amsterdam

Presenting PricewaterhouseCooper’s and the DFKI’s approach to detect financial fraud as anomalies in journal entries Dr. Damian Borth and Marco Schreyer speak about the future of audits and forensics in large scale financial and accounting data by the use of Deep Learning and Stacked Autoencoders. [X].

NVIDIA awards DFKI Deep Learning Competence Center with NVIDIA AI Lab Award

At this years NVIDIA GPU Technology Conference (GTC) in Amsterdam, NVIDIA CEO and co-founder Jen-Hsun Huang announced during his opening keynote the DFKI to join its NVIDIA AI Lab program. Only two labs in Europe have been awarded this honor and the DFKI Deep Learning Center is one of them. Along with this, DFKI will be provided with founding and the NVIDIA DGX-1 supercomputer, as one of the first research institutes in Europe. Dr. Damian Borth says: “this will complements the ambitions of the Deep Learning Competence Center to enhance basic research and industrial knowledge transfer in the field of AI.” (read more)
In this context Dr. Damian Borth gives a short interview at the NVIDIA GTC Europe about the potential of using the DGX-1.

Dr. Damian Borth at the ISC High Performance Conference in Frankfurt

Dr. Damian Borth gives an invited talk on "Deep Learning for Visual Classification of Adjective Noun Pairs in Context of Image Captioning and Video Analysis" at the ISC High Performance Conference in Frankfurt. In addition to Dr. Borth's talk, three other talks, including one by Andrew Ng, were devoted to the topic of Deep Learning. A summary of the event can be found on Youtube.

Invited talk about usage of Deep Learning and Satellite Image Analysis

At this year’s Situation Awareness Workshop at ICSI in Berkeley, California, USA, Dr. Damian Borth was invited to present recent work around deep learning and satellite image data analysis for situation awareness and natural disaster monitoring. The workshop was set up as an interdisciplinary event connecting domain experts with machine learning experts from UC berkeley, University of Stanford, Lawrence Berkeley Laboratory, and DFKI.

Workshop At First National Science Foundation In Berkeley

Along with many other events, Dr. Damian Borth was one out of six provocateurs to discuss the topic of Cybermanufacturing Ecosystem and Infrastructure at the Cybermanufactoring Workshop on Enabling Composable & Modular Manufacturing through Abstractions in Berkeley.

Innovation Advisory Board PwC and NVIDIA at the DFKI Kaiserslautern

Dr Damian Borth - head of the DFKI Deep Learning Competence Center - and Werner Ballhaus - head of the Innovation Advisory Board of PwC - have welcomed the internal and external visitors while a visit from PwC and NVIDIA at the DFKI at Kaiserslautern. This cross research-area event was opened by Prof. Didier Stricker. He presented the DFKI as a Research institute to our guests and gave many examples of several areas of AI.
After the lunch break, Dr. Damian Borth gave a presentation about “Deep Learning for Computer Vision”. Likewise Prof. Joseph van Genabith and his team have presented “Deep Learning for Language and Text”. As external visitors, Ralph Hinsche and Axel Koehler from NVIDIA had the opportunity to present the topic “GPU Accelerated Learning”. Finally, Bernd Reimer and Marco Schreyer have presented the current research from PwC in the scope of Data Analysis, Visualization and “Fraud and Anomaly Detection”.
This event was completed with a tour through the SmartFactory-KL and a live demonstration of the Augmented Reality-Guide from the Spin-Off IOXP.

Dr. Damian Borth gives a Talk at the “IT Themenabend”, Intel Visualisierungszentrum DFKI Saarbrücken

To contribute to the discussion about deep neural networks, Dr. Damian Borth spoke about Deep Learning for visual sentiment analysis and opinion mining at the IT-Themenabend in Saarbrücken. Together with Dr. Mario Fritz this event provided a great opportunity to get an overview about recent advances in Deep Learning.

Dr. Damian Borth Gives A Talk At ISC, Frankfurt a.M.

Dr. Damian Borth talks about Deep Learning for Visual Classification of Adjective Nound Pairs at ISC. Furthermore, he introduces the SentiBank and the Yahoo Flickr Creative Common 100 million (YFCC100m) dataset, for which more information can be found here [x].

DFKI has been selected to join the NVIDIA’s GPU Research Center Program

Since its continuous work in the area of GPU research in the field of text, image, video and social media analysis, DFKI Deep Learning Competence Center has been chosen to join NVIDIA’s GPU Research Center. With the status as a GPU research center, the researchers from DFKI have access to latest devices and software for GPU computing to further explore the wide field of GPU utilization for deep learning.

Interview with WDR and Dr. Damian Borth about Artificial Intelligence & Deep Learning

Dr. Damian Borth gave an Interview to the WDR 5 radio program to talk about the definition of Artificial Intelligence and current state-of-the-art in the Deep Learning following the potential usage of AI in everyday life. Here a snippet of the interview [x].

Dr. Damian Borth At Twitter In Berlin, Germany

‘We consider data the fabric of modern life,’ says Dr. Damian Borth in his invited talk about Visual Sentiment Analysis with Deep CNNs at Twitter in Berlin. Working closely with Twitter at the Multimedia Opinion Mining project, this gave both Dr. Borth and Twitter a unique opportunity for exchange of ideas and discussion of potential collaboration activities.

Dr. Damian Borth Gives An Interview To The German Newspaper ‘Welt am Sonntag’

In current time Artificial Intelligence is even more present in daily life. Considering the recent success of Google’s Alpha Go in the game Go, Dr. Damian Borth gives a slight overview [x] of how AI is working, in particular Deep Learning.

Dr. Damian Borth talks at the Quo Vadis 2016, Berlin

Along with other talks from experts from several institutions Dr. Damian Borth presents the concept of Deep Learning for Visual Classification of Adjective Noun Pairs and its Application at the Quo Vadis 2016. Further he introduces the SentiBank and the Yahoo Flickr Creative Common 100 million (YFCC100m) dataset, for which you can see more information here [x].

Dr. Damian Borth Gives A Talk At The HfG Karlsruhe

Covering the topics “Art and Artificial Intelligence?” Dr. Damian Borth had the opportunity to present recent advanced in deep learning to students from KIT and HfG Karlsruhe. He presented insights about machines starting to create art at the HfG Karlsruhe.

Dr. Damian Borth invited to Expert Discussion about AI at German Live TV Show

The 3sat-TV-Show “Scobel” dedicated a live show on the 28.04.2016 to the topic “Die Roboter-Rivalen”. The reason for this show was triggered by the media hype about the victory of Google’s computer program “AlphaGO” against one of the world’s best Go-players, the south korean Lee Sedol. “AlphaGO” is basically using two categories of neuronal networks and a tree search. So, it is based on a Deep Learning procedure. In this context, Dr. Damian Borth and Thomas Metzinger were invited to discuss the need of ‘ethik of machines’ in terms of recent successes in artificial intelligence. In particular, whether it is possible for machines to pass the Turing test, in order to analyze and evaluate data sets independently and autonomously.

Interview with welt.de about Artificial Intelligence

Dr. Damian Borth gave an Interview to welt.de about how computers are able to learn from unstructured data and to deal with searching for pattern in order to find meaning.

Dr. Damian Borth at Hitachi Social Responsibility Event in Brussels

Speaking about Industry 4.0 and Social Innovation, Dr. Damian is invited to present his opinion about the interplay between Artificial Intelligence and the rapidly changing working environment at Hitachi in Brussles.

Dr. Damian Borth gives an Interview for German TV channel “RTL Nachjournal”

The current news of a go match between a human player and an artificial intelligence in which the AI has won, is one of the topic Dr. Damian covers in the Interview with RTL. He explains the principles of artificial intelligence and how the machine was able to defeat the human player by using latest advanced in deep learning.

DFKI With Many Projects At The CeBIT in Hannover, Germany

Together with the MOM, the SentiBank and many more project, the DFKI was present at this years CeBIT in Hannover. A short impression can be found here [x].

Dr. Damian Borth gave a Keynote about “Artificial Intelligence” at WIV Family Office Annual Meeting in Frankfurt am Main

The WIV Family Office invited Dr. Damian Borth to give a keynote about “Artificial Intelligence” at their annual meeting in Frankfurt am Main. Dr. Damian was speaking in front of as selected group of guest including beyond others Prof. Rombach from Fraunhofer IESE, and member of the board of the University of Kaiserslautern.

Dr. Damian Borth as a speaker at the E-Finance Lab and IBM Conference in Frankfurt am Main

Dr. Damian Borth gives a talk at the Joint Spring Conference 2016 of E-Finance Lab and IBM to discuss the topic of “Identifiers and Identification Management in the Financial World and Beyond - Requests, Solutions, and Applications” with other participants from science and practise. A video of Borth’s speech about “Identifier Matching in Responsible Investment – a Machine Learning Approach” is to be found here [x].

Title Story Of Communication of the ACM - February Issue

With the recent advances in Deep Learning researchers demand more and more data for supervised training. Through collaboration between several researchers, The YFCC100M, one of the largests publicly available datasets for deep learning is made available over the last weeks. The YFCC100m consists of 100 million images and videos from Flickr including its metadata, all made available as creative common licence. This dataset and its description were published in the current issue of the Communication of ACM (“YFCC100M: The New Data in Multimedia Research” by Bart Thomee, David Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Dough Poland, Damian Borth) as title story and cover sheet. Dr. Damian Borth as part of the group of researchers welcome this novel dataset and hope for exciting discoveries in the research community. More information about the YFCC100M and its browser can be found at www.yfcc100m.org.

Google Tech Talk In Zurich About Visual Classication with DeepSentiBank: Visual Classification of Adjective Noun Pairs and its Application for CSA Detection

Dr. Damian Borth and Dr. Christian Schulze both introduce the concepts of Adjective Nound Pairs (ANP) and Visual Sentiment Ontology (VSO) together with SentiBank, which has been already shown to have a broad spectrum of application domains. To do experiments on several datasets, applying multiple low-level features are needed, like real world CSA provided by several European law enforcement partners.

1st Financial Data Science Association Conference in Zurich

Having the goal to improve the synergies between data science and financial markets more than 30 international participants joined the first Financial Data Science Association Conference. Beside several plenary sessions and panel discussions, Dr. Damian Borth spoke introduced the principles for Financial Data Science and discussed usage of Artificial Intelligence for Good.

Dr. Damian Borth Giving A Keynote at Reporting 3.0 in Berlin

Being invited to give a keynote on Driving the Next Systemic Transformation by Big (Sustainability) Data at the Reporting 3.0, Dr. Damian Borth showed in a vivid kind of way how to handle this issue with for example current technology in satellite techniques, or how machines can imitate the art style of several artists to change a given picture into this style. The presentation to this keynote can be found here [x].

Dr. Damian Borth Gives A Talk At The Quantified Planet Data Summit In Stockholm, Sweden

‘How can Machine Learning help the Planet?’ this was the question Dr. Damian Borth discussed in his talk at the Quantified Planet Data Summit. The QP itself is a non-profit and global community that collects real time sensor data. The goal is to build an open platform for innovation in health and environment data for a better world and to achieve the Sustainable Development Goals.

MADM presents at the press event covering Googles engagement with the DFKI

The DFKI announced that Google became a shareholder of the DFKI. The MADM group presented their Eye Tracking technology and Visual Sentiment Analysis (SentiBank) with Deep Convolutional Neural Networks (CNN), which was names as one of the key technologies by Google. In this context, MADM was also covered in an TechCrunch article.

"Financial Data Science" a discipline combining Data Mining and Financial Statistics

A new discipline emerges - “Financial Data Science” - which combines approaches from data mining and financial statistics leveraging large-scale data processing, machine learning, statistics, and financial modeling.

Nowadays the Internet, as a major platform for communication and information exchange, provides a rich repository of people’s opinion and sentiment about a vast spectrum of topics. Such knowledge is embedded in multiple facets, such as comments, tags, browsing actions, as well as shared media objects. The analysis of such information either in the area of opinion mining, affective computing or sentiment analysis plays an important role in behavior sciences, which aims to understand and predict human decision making and enables applications such as brand monitoring, stock market prediction, or political voting forecasts. (read more)

"Multimedia Opinion Mining for Social Good" presented at UN PRI Event

In the context of the BMBF sponsored project “Multimedia Opinion Mining”, Dr. Damian Borth, gave a guest lecture at the ICMA Center of the Henley Business School about how tools developed at MADM can be used for social good. The lecture presented novel way of mining social multimedia to gain insights about critical issues around environmental, social, and governance issues.

Dr. Damian Borth part of assessment committee for the Investment Innovation Benchmark (IIB)

Dr. Damian Borth served as a member of the assessment committee at the Investment Innovation Benchmark (IIB) summit in London. Damian was appointed to the committee due to his background in machine learning and large-scale data processing and will serve as an expert to evaluate FinTech submissions. The assessment committee consists of member from academia and some of the largest pension fund worldwide.

"YFCC100M: The New Data in Multimedia Research" got accepted at CACM

Our paper describing the 100 million images and videos Flickr dataset: “YFCC100M: The New Data in Multimedia Research” got accepted at “Communication of the ACM”. The paper describes the up-to-date largest dataset in the computer vision and multimedia research community. This work was a collaboration between Yahoo Inc., ICSI Berkeley, and the Lawrence Livermore National Laboratory. A pre-print is available on Arxiv.org.

Interview with Hannoversche Allgemeine Zeitung about Computer Vision Research

Dr. Damian Borth gave an Interview to Hannoversche Allgemeine Zeitung about the success of deep learning in computer vision research, recent progress to recognize sentiment and emotions in visual content, and how artificial intelligence (AI) can be used for social good.

Dr. Damian Borth moderates a Technology Panel at Bloomberg, New York, USA

With the advent of FinTech, disciplines like machine learning, natural language processing, and large-scale data mining are moving into the finance and investment industry. To discuss the potential of FinTech, Dr. Damian Borth was invited to moderate a Technology Panel at FinTech Bloomberg event. The panel was including CEOs and CTOs of leading startups in the area such as: eRevalue, TruValue and Bloomberg Labs.

Dr. Damian Borth speaks at RI Europe about Financial Data Science, London, UK

Dr. Damian Borth speaks about Big Data and Environmental, Social and Governance research and introduces together with Prof. Andreas Hoepner the concept of “Financial Data Science” at the RI Europe - a major conference in the responsible investment community. Financial Data Science embraces the processing of large datasets with machine learning methods and welcomes approaches such as opinion mining from public available data on the web.

Call for paper for Multimedia COMMONS Workshop is out

The CfP to the Multimedia COMMONS workshop (MMCommons) is out! We invite the community to attend MMCommons and explore the possibilities for novel research, future data challenges, and new benchmarks offered by this large-scale open dataset (YFCC100m dataset). The MMCommons will be held at the ACM Multimedia 2015, Brisbane, Australia around October 26-30.

Dr. Damian Borth appointed new head of MADM group

By April 2015, Damian Borth became the new head of MADM. Before this position Damian was a postdoctoral research fellow at the International Computer Science Institute (ICSI) with Dr. Gerald Friedland and at UC Berkeley with Prof. Trevor Darrell where he was involved in various project sponsored by DARPA, IARPA, and the Lawrence Livermore National Laboratory. His research focuses concept detection from social multimedia including trending topic detection, visual sentiment analysis, and multimedia opinion mining.

Invited talk about "Data Science and Opinion Mining" at the London School of Economics, UK

The project Multimedia Opinion Mining (MOM) was presented in the context of Data Science during an UN PRI event at the London School of Economics. Dr. Damian Borth introduced data science and opinion mining and outlined how it can be used for social good. The event was later interrupted by an evacuation due to the Holborn Fire in London.

Dr. Damian Borth gives an invited talk at the University of Reading, UK

Dr. Damian Borth speaks about “Mining Insights from Public Data” at an event of the University of Reading and outlines concepts and goals of the Multimedia Opinion Mining (MOM) project to the audience.

Dr. Damian Borth invited to BMBF dinner with Dr. Georg Schütte in Berkeley, USA

During a visit of the Bundesministerium für Forschung und Entwicklung (BMBF) in the Bay Area, the BMBF invited DAAD scholarship researcher from ICSI to join an exclusive dinner with Deputy Secretary Dr. Georg Schütte. Dr. Damian Borth introduces the BMBF funded project Multimedia Opinion Mining (MOM) to the delegation during the dinner.

BMBF funded project Multimedia Opinion Mining (MOM) starts

Multimedia Opinion Mining (MOM), a 2.5 years long project funded by the BMBF, aims to address the challenge of opinion mining in social multimedia content. The main goals of MOM are: (1) analyze social media data to detect and track trending topics. (2) understand multimedia content with respect to its sentiment and opinion. (3) discover individual persons or groups able to influence propagation of topics in social networks. And finally, (4) forecasting the progression of identified topics into the near future. The project will be coordinated within MADM and lead by Dr. Damian Borth.



This project aims to address the challenge of opinion mining of multimedia content from the web. This comprises a large-scale multi-modal analysis of social media streams and their underlying network dynamics considering different media channels such as Twitter, YouTube, Flickr, Google, and Wikipedia. The main contribution of MOM is four-fold as illustrated below:

1. Trending Topics

social media data is analyzed utilizing information from structured data sources to detect and track trending topics.

2. Sentiment Analysis

large amounts of multimedia content are analyzed with respect to its sentiment and opinion. This follows a holistic approach i.e. the analysis of a rich set of different modalities such as textual, visual, and tempo-visual content.

3. Social Network Analysis

this information is further enriched by network analysis to grasp structural diffusion as well as global and local impact of information in social networks.

4. Forecasting

the possibility of forecasting the progression of identified opinion topics into the near future is investigated.


  • Large-Scale Social Multimedia Analysis
    Benjamin Bischke, Damian Borth, Andreas Dengel
    Book Chapter in Big Data Analytics for Large-Scale Multimedia Search, Chapter 4, Wiley & Sons, 2017 (accepted)

  • Image Captioning in the Wild: How people caption images on Flickr
    Philipp Blandfort, Tushar Karayil, Damian Borth, Andreas Dengel
    ACM Workshop on Multimodal Understanding of Social, Affective and Subjective Attributes, 2017 (accepted)

  • The Multimedia Satellite Task at MediaEval 2017
    Benjamin Bischke, Patrick Helber, Christian Schulze, Venkat Srinivasan and Damian Borth
    Mediaeval Benchmark 2017, 2017

  • Which Saliency Detection Method is the Best to Estimate the Human Attention for Adjective Noun Concepts?
    Syed Saqib Bukhari, Mohammad Al Naser, Saleh Mozafari, Damian Borth, Andreas Dengel
    Int. Conf. on Agents and Artificial Intelligence (ICAART), 2017 (accepted)

  • Predicting Human Associations with Graph Patterns Learned from Linked Data
    Jörn Hees, Rouven Bauer, Joachim Folz, Damian Borth, and Andreas Dengel
    Proc. of the ISWC 2017 Demo Track, 2017. (to appear)

  • Symbolic Association using Parallel Multilayer Perceptron
    Federico Alberto Raue, Sebastian Palacio, Thomas M. Breuel, Wonmin Byeon, Andreas Dengel, and Marcus Liwicki
    Proceedings of the 25th International Conference on Artificial Neural Networks, 2017

  • AudioSentibank: Large-scale Semantic Ontology of Acoustic Concepts for Audio Content Analysis
    Sebastian Sager, Damian Borth, Benjamin Elizalde, Christian Schulze, Bhiksha Raj, Ian Lane, Andreas Dengel
    arXiv preprint arXiv:1607.03766, 2016


  • Multimedia COMMONS Workshop 2016 (MMCommons’16) - Datasets, Evaluation, and Reproducibility
    Bart Thomee, Damian Borth, Julia Bernd
    ACM Multimedia Workshops, 2016


  • Generating Captions with Emotions using Concept and Syntax Transition Networks
    Philipp Blandfort, Tushar Karayil, Damian Borth, Andreas Dengel
    ACM Multimedia, Grand Challenge, 2016 (accepted)

    [challenge website]

  • Contextual Enrichment of Remote-Sensed Events with Social Media Streams
    Benjamin Bischke, Damian Borth, Christian Schulze, Andreas Dengel
    ACM Multimedia, Grand Challenge, 2016 (accepted)


  • Which languages do people speak on Flickr? A Language and Geo-Location Study of the YFCC100m Dataset
    Alireza Koochali, Sebastian Kalkowski, Damian Borth, Christan Schulze, Andreas Dengel
    ACM Multimeda Community-Organized Multimodal Mining: Opportunities for Novel Solutions (MMCOMMONS) Workshop, 2016 (accepted)

    [YFCC100m Browser]

  • Introducing Concept And Syntax Transition Networks for Image Captioning
    Philipp Blandfort, Tushar Karayil, Damian Borth, Andreas Dengel
    ACM Int. Conf. on Multimedia Retrieval (ICMR), 2016

  • An Evolutionary Algorithm to Learn SPARQL Queries: Finding Patterns for Human Associations in DBpedia
    Jörn Hees, Rouven Bauer, Joachim Folz, Damian Borth, Andreas Dengel
    arXiv preprint arXiv:1607.07249, 2016


  • Trending Topic Aggregation by News-based Context Modeling
    Sebastian Fuchs, Damian Borth and Adrian Ulges
    KI, 2016 (accepted)

  • YFCC100m: The New Data in Multimedia Research
    Bart Thomee, David Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Dough Poland, Damian Borth, Li-Jia Li
    Communication of the ACM, 2016, (YFCC100m Dataset Paper)

    [paper] [watch video] [YFCC100m Browser]

  • Top-k shortest paths in directed labeled multigraphs
    Sven Hertling, Markus Schröder, Christian Jilek, and Andreas Dengel
    Harald Sack, Stefan Dietze, Anna Tordai, and Christoph Lange, editors, The Semantic Web: ESWC 2016 Challenges. Extended Semantic Web Conference (ESWC-16), 13th Extended Semantic Web Conference, May 29 - June 2, Heraklion, Greece. Springer, 2016
  • Edinburgh associative thesaurus as rdf and dbpedia mapping
    Jörn Hees, Rouven Bauer, Joachim Folz, Damian Borth, and Andreas Dengel
    International Semantic Web Conference, pages 17–20. Springer International Publishing, 2016

  • Kickstarting the Commons: The YFCC100M and the YLI Corpora
    Julia Bernd, Damian Borth, Carmen Carrano, Jaeyoung Choi, Benjamin Elizalde, Gerald Friedland, Luke Gottlieb, Karl Ni, Roger Pearce, Doug Poland, Khalid Ashraf, David A. Shamma, and Bart Thomee
    ACM Multimeda Community-Organized Multimodal Mining: Opportunities for Novel Solutions (MMCOMMONS) Workshop, 2015

  • Real-time Analysis and Visualization of the YFCC100m Dataset
    Sebastian Kalkowski, Christian Schulze, Andreas Dengel, Damian Borth
    ACM Multimeda Community-Organized Multimodal Mining: Opportunities for Novel Solutions (MMCOMMONS) Workshop, 2015

    [YFCC100m Browser]

  • What makes a Beautiful Landscape beautiful: Adjective Noun Pairs Attention by Eye-Tracking and Gaze Analysis
    Mohammad Al-Naser, Seyyed Saleh Mozaffari Chanijani, Syed Saqib Bukhari, Damian Borth, Andreas Dengel
    ACM Workshop on Affect and Sentiment in Multimedia (ASM), 2015

  • Aesthetic Photo Enhancement using Machine Learning and Case-Based Reasoning
    Joachim Folz, Christian Schulze, Damian Borth, Andreas Dengel
    ACM Workshop on Affect and Sentiment in Multimedia (ASM), 2015

  • The New Data and New Challenges in Multimedia Research
    Bart Thomee, David Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Dough Poland, Damian Borth, Li-Jia Li
    arXiv, preprint arXiv:1503.01817, March, 2015


  • Large-Scale Deep Learning on the YFCC100M Dataset
    Karl Ni, Roger Pearce, Kofi Boakye, Brian Van Essen, Damian Borth, Barry Chen, Eric Wang
    arXiv, preprint arXiv:1502.03409, February, 2015


  • Mapping images to sentiment adjective noun pairs with factorized neural nets
    Takuya Narihira, Damian Borth, Stella X Yu, Karl Ni, and Trevor Darrell
    aarXiv preprint arXiv:1511.06838, 2015


  • Large-scale visual sentiment ontology and detectors using adjective noun pairs
    Damian Borth, Rongrong Ji, Tao Chen, Thomas Breuel, and Shih-Fu Chang
    Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval, pages 385–388. ACM, 2016


The following people contribute to the project.

Prof. Dr. Prof. h.c. Andreas Dengel

Head of Department

    Dr. Damian Borth

    Project Lead

      Dr. Christian Schulze


        Jörn Hees


          Sebastian Palacio


            Joachim Folz

            Junior Researcher

              Benjamin Bischke

              PhD student

                Philipp Blandfort

                PhD student

                  Andrey Guzhov

                  PhD student

                    Tushar Karayil

                    PhD student

                      Federico Raue

                      PhD student

                        Victor Campos


                          William Tong


                            Prakriti Bhardwaj


                              Carlos Gallego


                                Aman Gautam


                                  Sebastian Kalkowski


                                    Alireza Koochali


                                      Hans Schmitz


                                        Jan Niklas Böhm


                                          Jessica Chwalek


                                            Shailza Jolly


                                              Magnus Wiese


                                                The following people are former member of the project.

                                                Patrick Helber

                                                PhD student

                                                  Peter Burkert


                                                    Tobias Hamann


                                                      Tim Hertweck


                                                        Jalpa Punjabi


                                                          Sebastian Säger


                                                            Ahsan Naeem


                                                              Rajesh Chidambaram


                                                                Ayushman Dash


                                                                  Mihai Fieraru


                                                                    Monit Shah Singh


                                                                      Kshitij Kumar



                                                                        • Dr. Damian Borth
                                                                          • Trippstaedter Str. 122
                                                                            67663 Kaiserslautern
                                                                            • 20575-4184 (631) 49 +
                                                                              • damian.borth@dfki.de

                                                                              German Research Center for Articial Intelligence
                                                                              [Internal login]