[Research] A meta-review of mobile computing evaluation techniques

in

Lately I read lots of paper by Jesper Kjledskov. The guy wrote interesting stuff about the evaluation of mobile computing devices. One of his best paper is a meta-review of the techniques employed to do so:Kjeldskov J. and Graham C. (2003) A Review of MobileHCI Research Methods. In Proceedings of the 5th International Mobile HCI 2003 conference, Udine, Italy. Lecture Notes in Computer Science, Berlin, Springer-Verlag, pp. 317-335.

  1. One researcher read and classified the 102 selected research papers in relation to their method and purpose
  2. A second researcher blindly read and classified 20 randomly selected papers
  3. Due to disparities, all papers were then discussed one by one in collaboration between the researchers, resulting in a final classification

This reveals a tendency towards environment independent research and artificial setting research… at the expense of natural setting research. This reveals a tendency towards building systems based on trial and error, evaluating systems in controlled environments (if at all) … at the expense of studying real use of systems.Little research addresses the question of what is useful and what is percieved problematic from a user perspective. It seems assumed that we already know what systems to build and what problems to overcome (given the youth of the research field, this can hardly be true/The prevalent focus on applied research makes it difficult to set aside this assumption/On the contrary, it could be stated that mobile HCI in particular requires research addressing this issue). It seems also assumed that teal world studies are not very important (building and evaluating systems on the basis of applied research and laboratory experiments results in very concrete conclusions about specific solutions/Such conclusions can be difficult to generalize). Finally, methodology matters very little

The author then proposes different research avenue:

  1. Field and case studies for informing engineering through exploration of real use contexts
  2. Survey research for studying use of mobile technology in the hands of a large segment
  3. Action Research for developing knowledge through practice and intervention
  4. Basic research for developing theoretical understanding of the studied phenomena

Personally, my research is oriented towards the last point: it's basic research about how peope use location awareness information for group coordination in joint activities.