My interest in failures (as attested by my Lift09 speech) led my peruse "Anatomy of a Failure: How We Knew When Our Design Went Wrong, and What We Learned From It" (by William Gaver, John Bowers, Tobie Kerridge, Andy Boucher, Nadine Jarvis) with attention.

The article is a case study that examines the appropriation (or the wrong appropriation I should say) of an home health monitor device. The authors identify what they call ‘symptoms of failure’ that touches 4 themes: engagement, reference, accommodation, and surprise and insight. They discuss theses reasons of failures by looking at three different angles: (1) problems particular to the specific design hypothesis they had, (2) problems relevant for mapping input to output more generally, and (3) problems in the design process they employed in developing the system.

An interesting aspect in the paper is the must-have definition of what constitutes a failure:

"Approaches to evaluating interpretive systems such as the sort we describe here tend to focus on how to go about gathering suitable material for assessment, but avoid discussing how success or failure might be determined. For instance, Höök et al. based their evaluation of a system on analysing the conversations that groups of people had on encountering it. Others seek alternatives to verbalised judgements to capture more intuitive and sensual aesthetic and emotional responses. Finally, others advocate gathering multiple forms of evaluation from a variety of perspectives, including those of ‘cultural commentators’ such as journalists or filmmakers. Opening out evaluation to multiple voices and new forms of expression in these ways reflects the multiple interpretations afforded by the class of systems in which we are interested. On the other hand, these approaches can invite a kind of relativism from which it is difficult to draw firm conclusions. (...) we propose features of user engagement as being reliably symptomatic of success or failure, (...) we describe the symptoms of success and failure that emerge from a comparison of the unsatisfactory experiences observed in this field study with more rewarding deployments of other systems in the past."

The authors then go on with a description of their system (the "Home Health Monitor") with an account of early field trials that serve as a sort of baseline against which they compare the results from a field study. What is important to my research here is the description of how the system failed in conjunction with certain behavioral indicators they did not find:

Engagement: Beyond any explicit declaration of liking, we take as evidence such things as an enthusiasm about discussing the design and their experience with it; persistence in use and interpretation over time; suggestions for new enhancements that reflect our original design intentions, showing the prototype to friends.

Reference: the tendency for volunteers to discuss successful prototypes through reference to other technologies or experiences that they like.

Accommodation: the degree to which people accommodate successful designs to their existing domestic activities and rhythms

Surprise and Insight: successful systems are those which continue to occasion new surprises and new insights over the course of encounters with them. For instance, new content might appear, or unfamiliar, potentially rare, behaviours might be observed, and this might give rise to new perceptions of the system or the things it indicates. Equally, people may find new meanings for relatively rich but unchanging experiences. Of course, surprise and insight are neither properties of the system per se nor of the people who use it, but instead characterise the relationship between the two.

These were the symptoms of failures, which should no be confused with potential reasons of failure. The authors also contrast early trials results to the field study to get a grip on the causes that are quite specific to their design.

Why do I blog this? pursuing my work about failures here, gathering material about design issues with regards to failures for publication ideas. This piece is highly interesting as it shows how field research may help to uncover symptoms and causes of failures. Surely some good content to add to my lists.