Interesting part of Designing through exploration: using observational methods in ubiquitous technology research (by Brown, B. and A. Weilenmann) about the challenges to study ubiquitous computing uses.
In this paper, the authors explain the added value of evaluating ubiquitous computing products. They advocated for 'exploring' rather than 'evaluating'.They exemplify this claim with 3 systems.
I am strongly interested in their argumentation about the challenges to study Ubicomp:
- ubicomp is experimental then work on prototype: fragile, unreliable, sometimes physically larger than final product
- the real world (where the experiment is conducted) is very different from the lab settings
- ubicomp designed to be embedded into the environment often for long periods of time: difficult to study over time (longitudinal studies) -> difficult to create usage scenario, and it will be difficult to study them in lab context
- ubicomp often involved in chains of interaction between different device and individuals so it's difficult to take this into account in the usage scenario.
- ubicomp used in lots of different environments, different from classical one, and often multi.context, unpredictable use of mobile stuff.
- ubicomp use: short and sporadic
Those challenges are hard to solve in a conventional lab study. Besides, experiments outside the lab are difficult as well.