The creation and consumption of content, especially visual content, is ingrained into our modern world. This book contains a collection of texts centered on the evaluation of image retrieval systems. To enable reproducible evaluation we must create standardized benchmarks and evaluation methodologies. The individual chapters in this book highlight major issues and challenges in evaluating image retrieval systems and describe various initiatives that provide researchers with the necessary evaluation resources. In particular they describe activities within ImageCLEF, an initiative to evaluate cross-language image retrieval systems which has been running as part of the Cross Language Evaluation Forum (CLEF) since 2003.

To this end, the editors collected contributions from a range of people: those involved directly with ImageCLEF, such as the organizers of specific image retrieval or annotation tasks; participants who have developed techniques to tackle the challenges set forth by the organizers; and people from industry and academia involved with image retrieval and evaluation generally.

Mostly written for researchers in academia and industry, the book stresses the importance of combing textual and visual information - a multimodal approach - for effective retrieval. It provides the reader with clear ideas about information retrieval and its evaluation in contexts and domains such as healthcare, robot vision, press photography, and the Web

Autorentext

Henning Müller obtained his Masters degree in medical informatics from Heidelberg University, Germany in 1997 and his PhD on multimedia information retrieval from Geneva University, Switzerland in 2002. Since 2007 he has been a professor at the University of Applied Sciences Western Switzerland in Sierre. He has worked on visual information retrieval for over twelve years and on multimedia retrieval evaluation for almost the same amount of time. He has co-started the Benchathlon initiative on image retrieval evaluation and leads the ImageCLEF benchmark on multilingual and multimodal information retrieval. Henning Müller has published over 200 scientific articles and is currently in the editorial board of five journals. He has participated in several EU projects and has initiated several national projects.

Paul Clough is a lecturer in Information Systems in the Department of Information Studies, University of Sheffield (UK). He received his BEng from the University of York in Computer Science while also working for British Telecommunications Plc. as a software engineer. He received his PhD from the Department of Computer Science, University of Sheffield and has since worked as a researcher on a range of language engineering and information access projects. Clough is member of the Information Retrieval (IR) group, and his core research interests are information retrieval (geographical IR, multimedia IR and evaluation), computational text analysis (plagiarism detection, authorship attribution and creating corpora) and human-computer interaction. He has over 60 peer-reviewed publications in his research area and a US patent for an information management system.

Thomas Deselaers is a researcher at the Computer Vision Laboratory of ETH Zürich. He received his diploma and his PhD degree from RWTH Aachen University in Aachen, Germany in 2004 and 2008, respectively. From March 2004 to December 2008, he was a full-time researcher at the Human Language Technology and Pattern Recognition Group of the Computer Science Department of RWTH Aachen University, where he was the head of the image processing and understanding group from 2005-2008. In 2006 he was a research intern at Microsoft Research Cambridge, UK. His research interests are object classification and detection in complex scenes, content-based image retrieval, and pattern recognition.

Barbara Caputo is a senior research scientist (permanent position) at the Idiap Research Institute, Switzerland, since 2006 where she leads the Cognitive Visual Systems group. She received her PhD in Computer Science from the Royal Institute of Technology (KTH) in Stockholm, Sweden, in 2004. Her main research interest is in Vision for Robotics. This is a new emerging domain that aims at providing robots with visual capabilities similar to those of humans. This domain intersects the well-established research fields of computer vision, machine learning and robotics, where she has been active since 1999. As a result of her activities, she has edited two books, and she has published more than 40 papers in international journals and peer reviewed conferences. As of today, Barbara Caputo has been Principal Investigator and co-Principal Investigator of seven European and National projects.

Zusammenfassung

The creation and consumption of content, especially visual content, is ingrained into our modern world. This book contains a collection of texts centered on the evaluation of image retrieval systems. To enable reproducible evaluation we must create standardized benchmarks and evaluation methodologies. The individual chapters in this book highlight major issues and challenges in evaluating image retrieval systems and describe various initiatives that provide researchers with the necessary evaluation resources. In particular they describe activities within ImageCLEF, an initiative to evaluate cross-language image retrieval systems which has been running as part of the Cross Language Evaluation Forum (CLEF) since 2003.

To this end, the editors collected contributions from a range of people: those involved directly with ImageCLEF, such as the organizers of specific image retrieval or annotation tasks; participants who have developed techniques to tackle the challenges set forth by the organizers; and people from industry and academia involved with image retrieval and evaluation generally.

Mostly written for researchers in academia and industry, the book stresses the importance of combing textual and visual information – a multimodal approach – for effective retrieval. It provides the reader with clear ideas about information retrieval and its evaluation in contexts and domains such as healthcare, robot vision, press photography, and the Web

Inhalt
Seven Years of Image Retrieval Evaluation.- Data Sets Created in ImageCLEF.- Creating Realistic Topics for Image Retrieval Evaluation.- Relevance Judgments for Image Retrieval Evaluation.- Performance Measures Used in Image Information Retrieval.- Fusion Techniques for Combining Textual and Visual Information Retrieval.- Track Reports.- Interactive Image Retrieval.- Photographic Image Retrieval.- The Wikipedia Image Retrieval Task.- The Robot Vision Task.- Object and Concept Recognition for Image Retrieval.- The Medical Image Classification Task.- The Medical Image Retrieval Task.- Participant reports.- Expansion and Reranking Approaches for Multimodal Image Retrieval using Textbased Methods.- Revisiting Subtopic Retrieval in the ImageCLEF 2009 Photo Retrieval Task.- Knowledge Integration using Textual Information for Improving ImageCLEF Collections.- Leveraging Image, Text and Crossmedia Similarities for Diversityfocused Multimedia Retrieval.- University of Amsterdam at the Visual Concept Detection and Annotation Tasks.- Intermedia Conceptual Indexing.- Conceptual Indexing Contribution to ImageCLEF Medical Retrieval Tasks.- Improving Early Precision in the ImageCLEF Medical Retrieval Task.- Lung Nodule Detection.- Medical Image Classification at Tel Aviv and Bar Ilan Universities.- Idiap on Medical Image Classification.- External views.- Press Association Images Image Retrieval Challenges.- Image Retrieval in a Commercial Setting.- An Overview of Evaluation Campaigns in Multimedia Retrieval.