MIC – Machine Learning Talk Series: "Multi-Modal Joint Embedding Space Learning for Cross-Modalit Retrieval" with Matthias Dorfer (JKU, Department of Computational Perception)

Abstract:

Cross-modality retrieval encompasses retrieval tasks where the fetched items are of a different type than the search query, e.g., retrieving pictures relevant to a given text query. The state-of-the-art approach to cross-modality retrieval relies on learning a joint embedding space of the two modalities, where items from either modality are retrieved using nearest-neighbor search. In my talk I will review two different learning paradigms -- Deep Canonical Correlation Analysis and Pairwise Ranking Losses — which both yield embedding spaces exhibiting properties beneficial for retrieval. I will present potential application scenarios as well as experimental retrieval results on two different modality pairs, namely text and images as well as audio and sheet-music.

Short CV:

Matthias Dorfer finished his Master studies of Medical Informatics at the Vienna University of Technology in March 2013. He worked in a research project for the early diagnosis of the Alzheimer's Disease using EEG signal analysis at the Austrian Institute of Technology from 2011 to 2012. He was member of the CIR group from 2012 to 2014, and is now Researcher at the Department of Computational Perception of the Johannes Kepler University Linz, Austria.

http://www.cp.jku.at/people/dorfer/