[Research] Lab experiment to evaluate mobile computing

J.Kjeldskov and M.B.Skov (2003) Creating Realistic Laboratory Settings: Comparative Studies of Three Think-Aloud Usability Evaluations of a Mobile System, Proceedings of the 9th IFIP TC13 International Conference on Human Computer Interaction, Interact 2003.
The authors adresse the topic of mobile devices usability using 'realistic laboratory settings'. They advocate for the use of lab settings because of various constraints like "mobile systems for highly specialized use contexts such as e.g. safety-critical or hazardous environments may prohibit exploratory usability evaluations since errors involving risks for people and equipment cannot be tolerated". They exemplifiy this issue by presenting the work they carried out to evaluate a mobile device used for coordination on large container vessels. They used 3 different experimental settings: - in a standard usability laboratory (mockup...mirror/camera...) with non domain subjects - in a standard usability laboratory (mockup...mirror/camera...) with domain subjects - in an 'advanced lab' (room with ship simulator) with domain subjects

They identified different usability problems in those 3 settings: Even though it seems interesting (with regard to the number of problems that emerged), it is a bit crazy to notice that the authors evaluate a MOBILE device in a NON-MOBILE situation! They cite this limit at the end of the paper but I do think it's a real drawback that should not be left as a side critic!