Evaluation and production of resources

Evaluation is very important to measure the effectiveness and progresses of information indexing and retrieval methods. We participate in many national and international evaluations campaigns such as TREC, TRECVID, CLEF, INEX, VOC and MediaEval. We also contribute to the organization of such campaign by providing ancillary data and tools, by participating to advisory committees (TRECVID) or by directly organizing them (TRECVid semantic indexing task and CLEF eHealth). These participations enable the positioning of the team among international research as previously mentioned.

Production of annotated resources is also very important both for system training and system evaluation. In the context of the Corpus project of the Quaero programme, we produced over 30 million of concept annotations in still images or in video shots. Most of them were publicly released for the TRECVid 2010-2014 semantic indexing tasks. Such annotations were produced using active learning and active cleaning approaches ensuring that each annotation made is both reliable and as useful as possible for system training.

Laboratoire d'Informatique de Grenoble
Bâtiment IMAG
700 avenue Centrale
CS 40700, 38058 Grenoble Cedex 9 - France
Phone: +33 4 57 42 15 48
Group Leader: Georges QUÉNOT

Paper accepted for EDBT 2020 "Fairness in Online Jobs: A Case Study on TaskRabbit and Google", with S. Amer-Yahia, S. Elbassuoni, A. Ghizzawi, R. M. Borromeo , E. Hoareau and P. Mulhem.

Anuvabh Dutt defended brilliantly his PhD thesis on December 17th 2019 :

Le 2 décembre 2019, un article publié sur le blog BInaire sur le testing algorithmique.

Kodicare (continuous evaluation of web search engines) ANR International Research Project acccepted for funding. Cooperation with RSA Vienna and Qwant. 3 years, from 11/2019.

The paper "Quelques pas vers l'Honnêteté et l'Explicabilité de moteurs de recherche sur le Web." Philippe Mulhem, Lydie du Bousquet, Sara Lakah, was awarded Best paper of the CORIA 2019 conference,

We organized the 40th European Conference in Information Retrieval (ECIR) in march 2018