Evaluation of Cross-Language Information Retrieval Systems

The second evaluation campaign of the Cross Language Evaluation Forum (CLEF) for European languages was held from January to September 2001. This campaign proved a great success, and showed an increase in participation of around 70% com pared with CLEF 2000. It culminated in a two day workshop in Darmstadt, Germany, 3-4 September, in conjunction with the 5th European Conference on Digital Libraries (ECDL 2001). On the first day of the workshop, the results of the CLEF 2001 evalua tion campaign were reported and discussed in paper and poster sessions. The second day focused on the current needs of cross language systems and how evaluation cam paigns in the future can best be designed to stimulate progress. The workshop was attended by nearly 50 researchers and system developers from both academia and in dustry. It provided an important opportunity for researchers working in the same area to get together and exchange ideas and experiences. Copies of all the presentations are available on the CLEF web site at http://www. clef campaign. org. This volume con tains thoroughly revised and expanded versions of the papers presented at the work shop and provides an exhaustive record of the CLEF 2001 campaign. CLEF 2001 was conducted as an activity of the DELOS Network of Excellence for Digital Libraries, funded by the EC Information Society Technologies program to further research in digital library technologies. The activity was organized in collabo ration with the US National Institute of Standards and Technology (NIST).

Inhalt

System Evaluation Experiments at CLEF 2001.- CLEF 2001 - Overview of Results.- Mainly Cross-Language.- Report on CLEF-2001 Experiments: Effective Combined Query-Translation Approach.- Multilingual Information Retrieval Using English and Chinese Queries.- Exeter at CLEF 2001: Experiments with Machine Translation for Bilingual Retrieval.- TNO at CLEF-2001: Comparing Translation Resources.- ITC-irst at CLEF 2001: Monolingual and Bilingual Tracks.- Experiments with the Eurospider Retrieval System for CLEF 2001.- Using Co-occurrence, Augmented Restrictions, and C-E WordNet for Chinese-English Cross-Language Information Retrieval at CLEF 2001.- Utaclir @ CLEF 2001 - Effects of Compound Splitting and N-Gram Techniques.- Using Statistical Translation Models for Bilingual IR.- Cross-Lingual Pseudo-Relevance Feedback Using a Comparable Corpus.- Investigation on Disambiguation in CLIR: Aligned Corpus and Bi-directional Translation-Based Strategies.- Vector-Based Semantic Analysis Using Random Indexing for Cross-Lingual Query Expansion.- Query Expansion Techniques for the CLEF Bilingual Track.- Intelligent Information Access Systems (SINAI) at CLEF 2001: Calculating Translation Probabilities with SemCor.- JHU/APL Experiments at CLEF: Translation Resources and Score Normalization.- Dictionary-Based Thai CLIR: An Experimental Survey of Thai CLIR.- English-Dutch CLIR Using Query Translation Techniques.- Thomson Legal and Regulatory at CLEF 2001: Monolingual and Bilingual Experiments.- Working with Russian Queries for the GIRT, Bilingual, and Multilingual CLEF Tasks.- IR-n: A Passage Retrieval System at CLEF-2001.- Monolingual Experiments.- Spanish Monolingual Track: The Impact of Stemming on Retrieval.- Shallow Morphological Analysis in Monolingual Information Retrieval for Dutch, German, and Italian.- Stemming Evaluated in 6 Languages by Hummingbird SearchServer(TM) at CLEF 2001.- Minimalistic Test Runs of the Eidetica Indexer.- Across the Bridge: CLEF 2001 - Non-english Monolingual Retrieval. The French Task.- Mpro-IR in CLEF 2001.- Some Terms Are More Interchangeable than Others.- Interactive Track.- The CLEF 2001 Interactive Track.- Noun Phrase Translations for Cross-Language Document Selection.- iCLEF at Sheffield.- iCLEF 2001 at Maryland: Comparing Term-for-Term Gloss and MT.- Evaluation Issues and Results.- The Philosophy of Information Retrieval Evaluation.- CLIR System Evaluation at the Second NTCIR Workshop.- Multilingual Topic Generation within the CLEF 2001 Experiments.- CLEF Methodology and Metrics.