In "How do Design and Evaluation Interrelate in HCI Research?", Wania et al. investigates the relationship between two communities in human-computer interaction: those who focus on designing for usability and the ones who evaluate usability. Their goal was to discover how design and evaluation are related through an analysis of the citations in the HCI literature over a fourteen-year period in a database of over ten million documents (and bibliographic cocitation analysis). The result of their analysis allowed them to get this co-citation map:
Some interesting remarks from the paper:
"There are authors who draw attention to the fact that design and evaluation go hand in hand. But even some of those who do draw connections between design and evaluation seem to spend most of the time talking about them separately and then spend a short time talking about both design and evaluation together. (...) two orthogonal dimensions and these were discussed above. One, vertically, shows high involvement with end users (at the top) and low involvement (at the bottom). The second dimension, running horizontally, shows a strong focus on theory development (on the left) and a strong focus on system building (on the right) (...) While this analysis is clear about who is at the center, we must speculate about why this particular set of authors is at the center. (...) We believe that the central theme that ties these five authors together is a focus on the context of use of systems. Suchman’s situated action and Hutchin’s cognition in the wild have a clear focus on context. Fischer’s seeding-evolution-reseeding model of system development bases design decisions on studies of systems in their context of use. (...) There are distinct clusters here, but there are not clusters consisting only of design methods and others consisting only of evaluation methods. Rather, each of the seven clusters contains examples of both design and evaluation methods. What, then, is the glue that holds each cluster together? (...) To some extent, there is some mapping of problems to approaches, but it seems that the ties are weak. (...) The next hot research area will be the one in the center of the HCI map. (...) We predict, therefore, that the next hot topic in HCI will be a focus on understanding design and evaluation in the context of use."
Why do I blog this? it's interesting to read this sort of article to get an overview of a field. Of course it's a pity this only concern academic work (and not other interaction design actors) but only the academic system would allow such analysis based on publications and co-citations.
Wania, C.E., Atwood, M.E. and McCain, K.W. (2006): How do design and evaluation interrelate in HCI research?. In: Proceedings of DIS06: Designing Interactive Systems: Processes, Practices, Methods, & Techniques 2006. pp. 90-98.