CHIIR 2020 was cancelled, and with it also the tutorial on multi-method evaluation was cancelled. We are deeply sorry.

Abstract

The half-day tutorial follows the objective to raise awareness in the CHIIR community concerning the significance of using multiple methods in the evaluation of information retrieval and recommender. Employing a multi-method evaluation integrating a number of single methods (e.g., a combination of an offline prediction experiment with an open dataset and a survey with closed- and open-ended questions) allows for getting an integrated and rich picture of system quality in the context of use. The goals of this tutorial are to raise awareness of the existence and risks of “blind spots” in single-method evaluations, introduce to multi-method evaluation, show various approaches how multiple methods may be integrated, provide participants the opportunity to apply these approaches to their individual research endeavors, and receive feedback from their peers.

It is warmly welcome (but not required) that participants prepare a small task before the tutorial:

Task

Please, reflect on your research goal and the one (or maximum 2) research question(s) that you aim to answer.

  1. Write down your research goal in 3 sentences as concise as possible
  2. Formulate and write down he research question (or the 2 questions) that you aim to answer.

Try to be concise, targeted, and “to the point'”!