EyeVisionBot (2003-2004)

A gaze-tracking artificial intelligence installation for image retrieval

by Hans H. Diebner, Sebastian Fischer, Lasse Scherffig

Hans H. Diebner, Sebastian Fischer, Lasse Scherffig: EyeVisionBot.
Recordings from the city anniversary of Karlsruhe 2003 (top),
Screenhots and Museum installation in the ZKM Media Museum
Karlsruhe 2004 (bottom).

Objectives of the project are to optimize image retrieval from databases and the Internet with the aid of eye tracking and adaptive algorithms as well as to scrutinize the instrumentalizing effect those adaptive algorithms and IT technologies can have. As a start, "EyeVisionBot" presents 9 to 25 images (depending on the monitor size). The time that the gaze dwells on the individual image is registered via eye tracking. The cumulated viewing times are used to estimate the searched category. By-and-by a new set of images are requested based on keywords which are generated from the previous searching behavior. In addition, structural comparisons with the so far longest viewed images are performed.

Keywords and structural attributes can be used for the search. The efficiency of the keyword-based search in databases depends on the usefulness of the classification and the correct assignment of the images to the categories. In the world wide web the keywords address terms that appear on the web page in which the image is embedded or as image file names. The structural search is based on comparisons of the searched images to structurally resembling archetypes.

"EyeVisionBot" shall be understood as an experimental interface to capture and analyse preconscious perception, too. Therewith, the search and classification behavior is to be investigated eventually in order to optimize the adaptation to the needs of the user. This research is not restricted to the derivation and analysis of adaptive algorithms to estimate the desired categories but also comprises the optimization of classifications as well as the presentational interface. We expect new insights with respect to the classification of databases as well as innovations in the fields of adaptive and context sensitive methods.

Hans H. Diebner,Sebastian Fischer and Lasse Scherffig: EyeVisionBot.
Video documentation with excerpts from the ZKM Media Museum Karlsruhe 2004.

"EyeVisionBot" has been realized with the image database of "Media Art Net" and was part of the project «Dynamical cognitive systems, neural networks and perception» of the Institute for Basic Research at the Center for Art and Media (ZKM). It is now further developed at the Institute for New Media (INM), Frankfurt.

The project was funded by the Land Baden-Württemberg with funds from the "Zukunftsoffensive III für innovative Projekte."

Hard- and software: Commercialized eye tracking system with computer. "GIFT" server (open source) for the structure-based image retrieval by "Computer Vision and Multimedia Lab," University of Geneva. In-house produced software for the examination and optimization of the search process, graphical interface as well as interfaces for the linkage and control of hard- and software components.


  1. Suchender Blick: Presentation of EyeVisionBot within the scope of Karlsruhe's anniversary from 17-20 June 2004.
  2. Since September 2004 part of the collection "Meisterwerke der Medienkunst" in the ZKM Media Museum Karlsruhe.
  3. Enseigner/Produire. Lehren, Kommunizieren und Produzieren im Zeitalter des Consumer Generated Content. Exibition: 20.01.- 03.03.2006, ZKM, Karlsruhe.