INEX 2005 Workshop on Element Retrieval Methodology 30 July 2005

INTRODUCTION

The INitiative for the Evaluation of XML Retrieval (INEX), and the University of Glasgow invite participation in the INEX Workshop on Element Retrieval Methodology. The workshop is to be held in Glasgow as part of an information retrieval research festival.

The annual INEX workshop examining element retrieval has shown improvements in retrieval precision, while at the same time raising new questions about element retrieval methodology. The annual evaluation workshop focuses on retrieval performance; this mid-year workshop will focus on methodology.

PARTICIPATION

Opinion papers discussing XML element retrieval are sought. Topics include but are not limited to Theory, Application, Measurement, Judgment, and Experience. Note that relevance ranking algorithms are excluded (they are the focus of the annual INEX workshop).

This workshop is open to all interested parties. Prior participation at INEX is not required. Submission of an opinion paper is not required (but is encouraged). Places are limited and preference will be given to authors of accepted papers. If you would like to attend but cannot submit a paper, please contact Mounia Lalmas or Andrew Trotman.

Theory

What is the perfect collection and what is the ideal way to interact with it? Is a heterogeneous collection diverse in DTDs or diverse in content? How should queries of this collection be specified in either a formal or natural language? When a user provides feedback are they providing information about the element (e.g. too large) or the content of the element?

Application

Is there an existing application of element retrieval either commercially or academically? Can user modeling of this application be used to identify unaddressed problems or formally define existing problems? How is element retrieval used, and what is it used for? What is a relevant element in (a given) context, and how can performance be measured within this context?

Measurement

What are current element retrieval metrics actually measuring? What would the ideal metric measure? There already exists a plethora of metrics so new metrics are not of interest, what is of interest is the identification of what should be measured.

Judgment

What can be determined from the existing INEX relevance judgments? Can an E3S3 element validly be the child of an E3S3 element, and if so then what does it mean? Is the 10 point relevance scale understood by the judges? How and when does a relevant element have no relevant descendants and what are the implications of this? How much information is needed for an element to become relevant (can a reference make an element relevant)? Note that the existing INEX online assessment software is not under discussion.

Experience

What can the experiments of TREC, CLEF, and NTCIR bring to element retrieval and INEX? How (if at all) is XML element identification like spoken document retrieval, video document retrieval, or passage retrieval? Is element retrieval a form of topic distillation, and how might experience in topic distillation be used for element retrieval?

Others

Any other topic related to XML element retrieval methodology is welcomed. Discussion of relevance ranking algorithms is not - they are the focus of the end of year INEX workshop.

SUBMISSION

Opinion papers discussing any of the above topics, or any other topic related to XML retrieval methodology (excluding relevance ranking) are sought. Contributions formatted according to the ACM SIG Proceedings guidelines and not exceeding 8 (eight) pages should be submitted by email to Mounia Lalmas and Andrew Trotman. Submissions should be formatted in PDF.

WORKSHOP FORMAT

On-topic contributions will be combined into a publicly-available proceedings. This will form a discussion document for the workshop (so please read it). From this the program committee will choose the most fiercely debated topics for discussion at the workshop.

The workshop will consist for 4 (four) sessions, each discussing one topic. Sessions will commence with a 30 minute discussion-raising presentation (by invitation) followed by a 90 (ninety) minute open discussion. Notes will be taken and made available.

SCHEDULE