EEMCS

Full text available as:

PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader 39892 Kb

Official URL: http://dx.doi.org/10.3990/1.9789036529730

Abstract

The aim of this work is to explore, from a perspective of human behavior, which
gestures are suited to control large display surfaces from a short distance away;
why that is so; and, equally important, how such an interface can be made a reality.
A well-known example of the type of interface that is the focus in this thesis is
portrayed in the science fiction movie ‘Minority Report’. The lead character of this
movie uses hand gestures such as pointing, picking-up and throwing-away to interact
with a wall-sized display in a believable way. Believable, because the gestures
are familiar from everyday life and because the interface responds predictably.
Although only fictional in this movie, such gesture-based interfaces can, when
realized, be applied in any environment that is equipped with large display surfaces.
For example, in a laboratory for analyzing and interpreting large data sets; in
interactive shopping windows to casually browse a product list; and in the operating
room to easily access a patient’s MRI scans. The common denominator is that the
user cannot or may not touch the display: the interaction occurs at arms-length and
larger distances.

Hand and arm movements are the gestures that computer systems interpret in
this thesis. The users can control the large display, and its contents, directly with
their hands through acts similar to those in ‘Minority Report’. The control is gained
through explicitly issuing commands to the system through gesturing. After defining
the elementary commands in such an interface (Chapter 2), we index existing
approaches to build gesture-based interfaces (Chapter 3) and, more precisely, the
gesture sets that have been used in these interfaces. Meticulous investigation of
which gestures are suited for issuing these elementary commands, and why, then
follows.

In a Wizard of Oz setting, we explore the gestures that otherwise uninstructed
users make when asked to issue a command through gesturing alone (Chapter 4).
By gesturing as they see fit, users pan and zoom a map of the local topology of our
university. Our observations show that users apply the same idiosyncratic gesture
for each command with a great deal of similarity between users. Also, gestures are
explicitly started and ended by changing the hand shape from rest to tensed and
back again. Users really believed that they were in actual control of the display;
immersed in the interaction that they found believable.

This consensus in the observed gestures is explored with an online questionnaire
(Chapter 5) filled out by a hundred users from multiple western countries. User
ratings of video prototyped interactions through gesturing show that there is significant
preference for certain gesture-command pairs. In addition, some gestures are
preferably reused in a different context or system state to improve understanding
and predicting of the system’s responses. These results are validated in another (partial)
Wizard of Oz setting (Chapter 6) where the users experience what it feels like
to issue commands with the proposed gestures. The ratings in each investigated
condition were similar, with minor differences that are mostly caused by physical
comfort, or lack thereof, while gesturing. Our findings were influenced profoundly
by both traditional WIMP-style interfaces and recent mainstream multi-touch interfaces
that swayed our participants’ preference towards some gestures.

To consolidate our previous findings, we designed, built and evaluated a gesture
interface with which the user can interact with 3D and 2D visualizations of biochemical
structures on a wall-sized display (Chapter 7). This prototype uses lasers for
pointing, one for each hand, and small buttons attached to the fingers for issuing
commands. The preferred gestures define the precise layout of these buttons on
the hand. Again, we found that our participants preferred to interact with the least
amount of effort and with the highest comfort possible. There was little variation
between users in the shape of the gestures that they preferred: tapping the thumb
on one of the other fingers was the prevalent gesture to indicate the beginning and
ending of a command: it mimicked pressing a button.

When taking a human perspective on gestures suited to issue commands to largedisplay
interfaces, it is possible to formulate a set of intuitive gestures that comes
naturally to its users. The gestures are learned and remembered with ease. In addition,
it is comfortable to perform these gestures, also when interacting for longer
periods of time. We observe in our line of research that technological developments
that reach mainstream distribution in the public domain influence the perception
of ‘intuitive’ and ‘natural’ in the end-users. The best example of this is perhaps
the influence of the indoctrination over the past four decades that the keyboardand-
mouse interface has had on the public’s notion of human-computer interaction.
More recent examples include the Nintendo Wii and the Apple iPhone. We, as the
interface designers of future intelligent environments, are very much dependent on
this notion. That is, if we wish to have gesture-based interfaces succeed in providing
easy to use, intuitive interaction with the pervasive large display surfaces in these
environments. The gestures that are described in this thesis are an important part
of those interfaces.

Export this item as:

• BibTeX
• XML

To correct this item please ask your editor

Repository Staff Only: edit this item