Here at Ubicomp 2006, there was a very pertinent discussion yesterday (instead of having a talk that everybody was awaiting: "No more SMS from Jesus? Ubicomp, religion and techno-spiritual practices" by Genevieve Bell) about methodologies to study mobile technologies (by Beki Grinter and Ken Anderson). First, one of the remark was about the inherent problems of ethnographic studies to study artifacts that go everywhere (and even more likely to be used in toilets, as opposed to food).

But the most interesting part (to me) was the discussion about the bridge between qualitative and quantitative methods. Yahoo/UC Berkeley's Marc Davis advocated for a new "computational social science" that would use mixed-methods (quali-quanti), aka "the new social science of the 21st century". His point was that we have access to an incredible quantity of data (ranging from interview to logged actions) that would allow us to gain information about different layers: from micro scale cognitive insights to large group processes (social groups, national issues...).

Unlike Anne which states that "quantitative methods are still being trotted out to save qualitative methods from their perceived inadequacies, a.k.a. "Real Science To The Rescue!"", I haven't felt that. Given the fact that the conversant were largely qualitative-data oriented, he tried to summarize the advantages of bridging both qualitative and quantitative data collection and analysis:

Large datasets can enable us to know who to talk to (who to interview, or with whom to deepen the study with ethnographic methods): who are representatives (or not) in the groups
It can allow to reveal unconscious behavior (that users cannot state)
A nice avenue of research they're pushing forward in his team is to compute visualizations (based on quantitative data) and then get back to the users to discuss with them. This is exactly what I am doing with CatchBob! visualizations of coordination as well as presenting the players a replay of their activity. This provides a basis for the discussion about "what they did" and "why they did it" (with of course some different "epistemological levels").
Qualitative analysis can also allow to redesign the sensors and the logged information that would be better suitable/more interesting.

He said that we're going "From what to why" to "From why to what"

Jeff Axup, as a follow-up, explained how different methods can be apply to different moment in the conception and that it's an iterative process: quanti can help provide focus for quali, then it allows to discover new dimensions in which quantitative methods could be applied.

(See also Joe McCarthy's thoughts)

Why do I blog this? because I am right in the middle of this discussion. In my study of how people use location-awareness in the pervasive game CatchBob! I encountered the same issues. What is curious however, is that the discussion stayed at the data level and did not address the theoretical assumptions from all those methods.