IR and Smart Cities, Evaluation of IR systems

Smart Cities

In the context of smart cities, billions of “things” connected to the Internet constitute the symbiotic networks of communication devices, smart appliances, objects and networks of people (e.g., social networks). So, the concept of traditional networks is expanding and in future will go beyond it, including more entities and information. These networks and devices are constantly sensing, monitoring and generating a vast amount of data on all aspects of human life. One of the main challenges in this area is event discovery, information retrieval and optimization of knowledge dissemination.
We are conducting different research projects to address challenges related to the context of smartcities. More specifically, we are designing new tools for Knowledge extraction and managementbased on activity recognition for mobile users and developing applications in the domain of Intelligent Transport Systems (ITS) and crowd management (on going PhD work of Shuja Jamil). We are building: i) predictive models based on Smart Networked Objects in Transportation (on-going PhD work of Adil Sheikh Amjad), Internet Of Things (IOT) capability Platform dedicated to the design and the assessment of IOT based applications that will have applications in smart buildings (ongoing PhD work of Khalid Halba, part of a joint project between NIST and University of Grenoble on smart Campus), and real-time user profile identification and management using multi-source heterogeneous data (ongoing PhD work of Edouard Bahri).

Evaluation of IR systems

Evaluation of IR remains a very active domain in the team. Firstly, investigations around document relevance assessment are still ongoing within the EIRAP project. This project aims at questioning classical assessment methods by comparing them. Once the manual assessment will be achieved, an analysis of the impact of the assessment methods on the evaluation results will be carried out in collaboration with the
project partners. Secondly, the team has considered evaluation from an industry perspective: systems are currently mostly evaluating their performances based on user logs. As such methods have limitations, we are considering the integration of Cranfield-type evaluations in a continuous evaluation-cycle. A project has been submitted to the ANR in collaboration with Qwant and Research Studio Austria. Lastly, the team is still highly involved in the organization of evaluation challenges and the scientific questions arising from such challenges. The CLEF eHealth evaluation lab, co-chaired by Lorraine Goeuriot since 2014, includes a shared task on consumer health,search, aiming at evaluating IR methods for laymen searching for medical information. Georges Quénot will also continue to lead the video retrieval task at NIST/TRECVid, currently planned until 2021.
Laboratoire d'Informatique de Grenoble
Bâtiment IMAG
700 avenue Centrale
CS 40700, 38058 Grenoble Cedex 9 - France
Phone: +33 4 57 42 15 48
Group Leader: Georges QUÉNOT

Paper accepted for CIRCLE 2022 "Multi-element protocol for IR experiments stability. Application to the TREC-COVID test collection", with G. Nicole González Sáez, P. Mulhem, L. Goeuriot and P. Galuscakova.

Kodicare (continuous evaluation of web search engines) ANR International Research Project acccepted for funding. Cooperation with RSA Vienna and Qwant. 3 years, from 11/2019, finishing in september 2023.