{"id":36,"date":"2024-12-10T07:56:45","date_gmt":"2024-12-10T07:56:45","guid":{"rendered":"https:\/\/lig-mrim-new.imag.fr\/?page_id=36"},"modified":"2024-12-12T15:06:36","modified_gmt":"2024-12-12T14:06:36","slug":"evaluation-and-corpora","status":"publish","type":"page","link":"https:\/\/lig-mrim.imag.fr\/index.php\/evaluation-and-corpora\/","title":{"rendered":"Evaluation and Corpora"},"content":{"rendered":"\n<p>Here are the list of corpora built by the MRIM team that are accessible for the community.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">&#8211; LongEval (Longitudinal evaluation of Web search)<\/h2>\n\n\n\n<h4 class=\"wp-block-heading\">For the 2023 edition of LongEval, we provide access to 3 epoch (one for training and reference test) 2 for testing: <a href=\"https:\/\/clef-longeval-2023.github.io\/\">link<\/a>. More details about the collection has been published at SIGIR 2023: P. Galuscakova, R. Deveaud, G. Gonzalez-Saez, P. Mulhem, L. Goeuriot, F. Piroi, M. Popel: <a href=\"https:\/\/arxiv.org\/abs\/2303.03229\">LongEval-Retrieval: French-English Dynamic Test Collection for Continuous Web Search Evaluation<\/a>.<\/h4>\n\n\n\n<p>For the 2024 edition 2024, we provide one epoch for training and two for testing:  <a href=\"https:\/\/clef-longeval.github.io\/\">link<\/a>. If you use this data please cite the ECIR paper *REF*  or the CLEF overview paper *REF*.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">&#8211; CLEF eHEalth<\/h2>\n\n\n\n<h4 class=\"wp-block-heading\">These evaluation campaigns are dedicated to evaluate IR systems in the context of medical data (<a href=\"http:\/\/clefehealth.imag.fr\/clefehealth.imag.fr\/index.html\" target=\"_blank\" rel=\"noreferrer noopener\">link<\/a>).<\/h4>\n\n\n\n<h2 class=\"wp-block-heading\">&#8211; TRECVid<\/h2>\n\n\n\n<h4 class=\"wp-block-heading\">These evaluation campaigns focus on multimedia\/video+text retrieval. TRECVid Semantic Indexing and High Level Feature Extraction (<a href=\"https:\/\/trecvid.nist.gov\/trecvid.data.html\" target=\"_blank\" rel=\"noreferrer noopener\">link<\/a>)<\/h4>\n\n\n\n<p>(HLF and SIN task training annotations from 2007 to 2015).<\/p>\n\n\n\n<p>If you use the TRECVid SIN or HLF annotations, please cite :<\/p>\n\n\n\n<p>St\u00e9phane Ayache and Georges Qu\u00e9not, \u201cVideo Corpus Annotation using Active Learning\u201d, 30th European Conference on Information Retrieval (ECIR\u201908), Glasgow, Scotland, 30th March \u2013 3rd April, 2008.<\/p>\n\n\n\n<p>The pdf of the paper is <a href=\"http:\/\/mrim.imag.fr\/georges.quenot\/articles\/ecir08.pdf\">here<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">&#8211; Corpora built during the GUIMUTEIC project<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Clicide : collection if still images (paintings) form the Grenoble Museum of Art (3425 images, 473 objects, 177 queries (143 objects)).\n<ul class=\"wp-block-list\">\n<li>Corpus (<a href=\"http:\/\/mrim.imag.fr\/Clicide\/Corpus.tar.gz\">gzipped tar file<\/a>)<\/li>\n\n\n\n<li>Queries (<a href=\"http:\/\/mrim.imag.fr\/Clicide\/Queries.tar.gz\">gzipped tar file<\/a>)<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>GaRoFou : Cultural heritage collection (sculptures, steles, small artifacts) from the Museun of Fourvi\u00e8re in Lyon\n<ul class=\"wp-block-list\">\n<li>Image Corpus : 1252 images, 311 objects. (<a href=\"http:\/\/mrim.imag.fr\/GaRoFou\/GaRoFou_I\/Corpus.tar.gz\">gzipped tar file<\/a>)<\/li>\n\n\n\n<li>Image queries : 184 images from 166 objects. (<a href=\"http:\/\/mrim.imag.fr\/GaRoFou\/GaRoFou_I\/Queries.tar.gz\">gzipped tar file<\/a>)<\/li>\n\n\n\n<li>Video corpus : 3 hours of video, (96 minutes with objects).\n<ul class=\"wp-block-list\">\n<li>Video files : <a href=\"http:\/\/mrim.imag.fr\/GaRoFou\/GaRoFou_V\/Video\/u1_A.mp4\">u1_A.mp4<\/a>, <a href=\"http:\/\/mrim.imag.fr\/GaRoFou\/GaRoFou_V\/Video\/u1_A.mp4\">u1_A.mp4<\/a>, <a href=\"http:\/\/mrim.imag.fr\/GaRoFou\/GaRoFou_V\/Video\/u1_D.mp4\">u1_D.mp4<\/a>, <a href=\"http:\/\/mrim.imag.fr\/GaRoFou\/GaRoFou_V\/Video\/u2_A.mp4\">u2_A.mp4<\/a>, <a href=\"http:\/\/mrim.imag.fr\/GaRoFou\/GaRoFou_V\/Video\/u3_A.mp4\">u3_A.mp4<\/a>, <a href=\"http:\/\/mrim.imag.fr\/GaRoFou\/GaRoFou_V\/Video\/u3_B.mp4\">u3_B.mp4<\/a>, <a href=\"http:\/\/mrim.imag.fr\/GaRoFou\/GaRoFou_V\/Video\/u3_C.mp4\">u3_C.mp4<\/a>, <a href=\"http:\/\/mrim.imag.fr\/GaRoFou\/GaRoFou_V\/Video\/u3_D.mp4\">u3_D.mp4<\/a>, <a href=\"http:\/\/mrim.imag.fr\/GaRoFou\/GaRoFou_V\/Video\/u4_A.mp4\">u4_A.mp4<\/a>, <a href=\"http:\/\/mrim.imag.fr\/GaRoFou\/GaRoFou_V\/Video\/u5_B.mp4\">u5_B.mp4<\/a>, <a href=\"http:\/\/mrim.imag.fr\/GaRoFou\/GaRoFou_V\/Video\/u5_C.mp4\">u5_C.mp4<\/a>, <a href=\"http:\/\/mrim.imag.fr\/GaRoFou\/GaRoFou_V\/Video\/u5_D.mp4\">u5_D.mp4<\/a>.<\/li>\n\n\n\n<li>Images from videos : <a href=\"http:\/\/mrim.imag.fr\/GaRoFou\/GaRoFou_V\/extract_image_from_video.py\">Extraction program extract_image_from_video.py<\/a>, image corpus extracted (test\/train using cross validation) <a href=\"http:\/\/mrim.imag.fr\/GaRoFou\/GaRoFou_V\/ImageFromVideo.tar.gz\">ImageFromVideo.tar.gz<\/a><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<p>If you use the Clicide of GaRoFou collections, please cite this work as:<\/p>\n\n\n\n<p>Maxime Portaz, Johann Poignant, Mateusz Budnik, Philippe Mulhem, Jean-Pierre Chevallet, Lorraine Goeuriot, Construction et \u00e9valuation d\u2019un corpus pour la recherche d\u2019instances d\u2019images mus\u00e9ales. CORIA 2017 \u2013 Conf\u00e9rence en Recherche d\u2019Informations et Applications- 14th French Information Retrieval Conference. Marseille, France, March 29-31, 2017, pp. 17-34.<\/p>\n\n\n\n<p>The pdf of the paper is <a href=\"http:\/\/www.asso-aria.org\/coria\/2017\/5.pdf\" target=\"_blank\" rel=\"noreferrer noopener\">here<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Here are the list of corpora built by the MRIM team that are accessible for the community. &#8211; LongEval (Longitudinal evaluation of Web search) For the 2023 edition of LongEval, we provide access to 3 epoch (one for training and &hellip; <a href=\"https:\/\/lig-mrim.imag.fr\/index.php\/evaluation-and-corpora\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"parent":0,"menu_order":5,"comment_status":"closed","ping_status":"closed","template":"onecolumn-page.php","meta":{"footnotes":""},"class_list":["post-36","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/lig-mrim.imag.fr\/index.php\/wp-json\/wp\/v2\/pages\/36","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lig-mrim.imag.fr\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/lig-mrim.imag.fr\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/lig-mrim.imag.fr\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/lig-mrim.imag.fr\/index.php\/wp-json\/wp\/v2\/comments?post=36"}],"version-history":[{"count":17,"href":"https:\/\/lig-mrim.imag.fr\/index.php\/wp-json\/wp\/v2\/pages\/36\/revisions"}],"predecessor-version":[{"id":171,"href":"https:\/\/lig-mrim.imag.fr\/index.php\/wp-json\/wp\/v2\/pages\/36\/revisions\/171"}],"wp:attachment":[{"href":"https:\/\/lig-mrim.imag.fr\/index.php\/wp-json\/wp\/v2\/media?parent=36"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}