Comparison between augmented and virtual reality

Tang, A., Biocca, F., and Lim, L. (2004). Comparing Differences in Presence during Social Interaction in Augmented Reality versus Virtual Reality Environments: An Exploratory Study In Proceedings of PRESENCE 2004, 7th Annual International Workshop on Presence, October 13–15, 2004, Valencia, Spain.This paper offers an interesting conclusion with regard to the differences between augmented reality and virtual reality: "the absence of representations of the user’s body in VR environment may lessen sense of spatial presence comparing with AR environment".

I like what this lab does, namely experiments about cognition in AR.

Catchbob analysis update

An update about how I am analyzing the data, from a temporal perspective:- divide each game in 3 parts: 'foraging the campus', 'joining the others', 'forming the triangle'. The end of the first part is when a player warns the other that he found the zone. the end of the second part if when all the players are within a certain area and begin to form the triangle. - for each part, calculate the number of messages (which type), the number of refresh, of disconnections, graphically represent the different category of path taken by the players. Chi-squared this to check the repartition of all those indexes! - for each part, ask the question: at that point is there a control of the environement (does the environment affords a behavior) or of the partners (explicit messages). - how does the leadership evolve during all those parts? (+ the strategy planning phase)

Other things: - calculate the number of pauses (no messages, no movements) - use the number of disconnections: could imply strategy modifications! - how players obeyed to orders!!!!!! - for each part, define the roles (caller, explorer...), look at the backtracking and overlap for each roles.

In the analyis, use: - Sperber and Wilson: participants wrote information they estimated to be relevant for them and their partners. - Kirsh + Kirsh and Maglio (Tetris): low-level information matters

IEEE Pervasive Computing Journal of Smart Phone

The last issue of IEEE Pervasive Computing is devoted to smart phones.

The mobile or smart phone is ushering in the real age of ubiquitous computing, and we shouldn’t undervalue its importance. This issue highlights work that presents specific smart phone applications as well as programming infrastructure for further development and studies of emergent uses.

Papers deal with various aspects ranging from social serendipity (the one by Rusell Beale) to the use of camera phones. The interesting point here is that the topic is addressed through various angles (not just the technical point of view). However, it's curious to see that location-based services does not appear that much in this issue. More later, I need to some time to read those papers ;)

Last week trip in Paris summary

Last week I had the chance to visit few labs in paris, France. I gave few talks presenting CatchBob! and the results we are getting. I received interesting feedbacks from the persons I met both from academics labs and private companies (mostly a telecom company, a video game editor and an electricity company). Thanks all :) My slides are here.

Few random notes: - Saadi: my stuff is about augmenting the representation instead of augmenting the environment. "what is amazing is that the augmentation should help users but it's not the case": NOT EVERY AUGMENTATION IS USEFUL. It would be great to formalize it, great for? - jim hollan: might be interesting to investigate the ambiguity of the positioning (what we thought about having an awareness tool with a lower accuracy) + he advised me to look at what they did about Active Campus. - IPSI: they wondered about the use of very accurate positioning but it turned out that it was tremendously difficult for ALL the users to have the positioning feature working. So users stopped using the tool. That is why they designed something simpler and that take into account the possibility of not having everybody with the location awareness tool.

phd advisor meeting

After watching 2 videos of CatchBob replay: - Check where people wrote (index of position) - mutual modeling act: "I saw that you were going in that direction, then I did not communicate that much": the person assume that his partner writes where he is. - players evaluates the plausibility of information: "I saw that Sandra was not moving, but I know her, she always move and she 's not lazy, so she was moving": players questioned the tool accuracy.

- when confronted to a discrepancy (due to the system), the are 3 reactions: believing the system, saying that the system is wrong or not understanding. Two discrepancies in catchbob: one's position (automatic) and others' position (automatic or manual). "I saw that it was indicated that B was positioned here but he was not", "I saw that B moved on the screen but I know he did not". Toward a discrepancy, people react with regard to information + expectations (the strategy decided or implicit information like knowing the partner)

- about the group confrontation: is the person who first get the signal who it the "narrative leader" during the replay? and is he the one send the larger number of messages/strategy messages? How does he influence the others?

- the word "reconfiguration" is not good, when talking about strategy

- Is the strategy negotiated (during the game) or is it just a personal strategy?

- importance of the weather?

- count the number of face2face meetings during the game (logfiles + replay)

- beware of players' personality, some players do not communicate before being sure of something. THEN, TRY TO KNOW SINCE WHEN THEY communicate their proximity to Bob.

- CHECK, when one player reached a high signal strength, if the keep noting the others signal strength (below) COMMUNICATION ECONOMY

- Strategy investigation: work on the two parts: foraging + triangle forming (for the latter part, check Morris maze strategies).

- 2 types of inferences: about the others, about the environment/network, about the environment/topology

- Do people write on a map as on a sheet of paper, are there different patterns? A lot make little annotation, some write big sentences

pierre was also impressed by the quality of people's narration, when confronted to their paths: they remind pretty well both their activity + the activity of their partners.

Conclusion: the awareness tool make people not discussing the strategy (there is no needs or inhibition due to to the awareness tool) OR since they do not discuss they don't talk about the strategy.

Concerning the mutual modeling, things are very simple in CatchBob!, players have to model their partners position and direction. Agreeing on the strategy is a cognitive prothesis for mutual modeling (once people decide a strategy they do not discuss it)

A nice independent variable: "no awareness tool" for a first BOB, then a second one with the tool.

Turning CatchBob! into a Sheep-Dog Trial

Well, CatchBob! task could be a bit more complex if add new dimensions like in the sheepdog trial:

Sheepdog trial (or simply dog trial) is a competitive dog sport in which herding dog breeds move sheep around a field, fences, gates, or enclosures as directed by their handlers. Such events are particularly associated with hill farming areas, where sheep range widely on largely unfenced land. These trials are popular in the United Kingdom, Ireland, Canada, the USA, Australia, New Zealand and other farming nations.

Different scenarios are possible:

There are several events, but the key element is the control of three to six sheep by one or two highly trained dogs under the control of a single shepherd. Both time and obedience play a part, as competitors are penalised if a sheep strays from the prescribed course.

One event consists of having the dog send three sheep up a steep hill through three or more gates. The shepherd must stand at the bottom of the hill and direct the dog by whistling. The huntaway dog barks loudly to push the sheep ahead up the hill.

Another popular event involves having the dog split six sheep into two groups of three and conducting each group in turn to small pens through a defined course by heading dogs. The group not being led is guarded by one of the two dogs, an eye-dog (from its ability to keep the sheep still by head movement alone). This is more difficult than it sounds, as the sheep invariably try to stay together.

PELOTE: Helping human and robot firefighters work as a team

PELOTE is an interesting EU research project "focused on how human firefighters and their robot counterparts would make use of a personal navigation and localisation system which could guide their movements, at the same time informing the external command centre of the exact location of each team member.(...) Researchers developed a backpack for the firefighter which uses inertial guidance systems, rather than GPS, to provide the location as shown on a personal display screen, as well as that of the command centre. “The idea is that team members would download a map of the interior before entering the building, and with the start point being calibrated at, say, the entrance, this personal map would enable them to know where they are – no matter how bad the visibility.”" The project website is here.

CatchBob! strategy analysis

In our location-based game CatchBob!, there seems to be 3 different strategies, when participants spread over the campus: the first one is the most common, the last one is 2 exceptions:

Then there are 2 possibilities with regard to how the strategy evolves:

As I said yesterday, players without an automatic display of others position are more reconfiguring their strategy. There are 3 reasons to reconfigure or not the strategy:

1. How easy is to go from one point to the other (mostly from the position of player A when player B asks the other to join him): the campus structure and its topology matters here: it depends upon the distance AND the easiness to move (if there are stairs, going outside...). In this case, the environment might be an important factor for the task. This factor relies on the way people figure out the distance or the effort.
2. When the player who calls the others (because he sees that he is close to Bob thorugh his proximity sensor) has 4, 5, 6 as a signal strength, he communicates it to the others, so they knows the zone where Bob might be located. The others then infer that it could be efficient to check other areas and not joining him by taking the same path. Here it's a mutual modeling act.
3. How the player who calls the other communicate it: "come here, I know it's there" or "it's in this area". Here it's an explicit act of communication.

How to move forward: check those 3 categories, use a chi-square to see if there are differences among the 2 groups:

• no reconfiguration
• just one player reconfigure his/her strategy
• 2 players reconfigure his/her strategy

And check how players reconfigure their strategies:

• no messages
• messages like "I join you"
• messages like "I join you by going in the upper area"

Different kinds of spatial exploration strategies

A good reference that summarizes the different kinds of spatial exploration strategies: Kallai, J., Makany, T., Karadi, K., & Jacobs, W. J. (2004). Spatial orientation strategies in Morris-type virtual water task for humans Behavioural Brain Research.Some excerpts I found relevant for my purpose:

The concept of spatial strategy varies in the research literature. The term “strategy” causes this confusion, as it refers to sets of strategies applied to specific behavioural situations. The most classic way in which psychology investigates the structure of behaviour is to observe performance across many situations and attempt to determine the possible commonalities of performance. Analysing the trajectories (search strategies) of rats during the completion of spatial tasks, for example, and describing the most common of these strategies is a simple and effective way to uncover invariance in exploration. (...) every goal-directed spatial action might be interpreted as spatial strategy Gaunet and Thinus-Blanc described two types of exploratory patterns: a Cyclic pattern and a Back and Forth pattern. (...) Hill et al. identified another set of search strategies. The first strategy involves the boundaries of the surrounding space. When this strategy is used, exploration is minimal, as the explorer stays close to the wall to maintain relative safety in a novel and frightening environment. The second strategy is a network-type exploration. The third strategy, an object-to-object strategy, involves random wandering until the first cue or landmark is found. A mixture of the first and third strategy also occurs; when the organism uses the boundary of the space as a reference point; nearby objects will be explored. A fifth strategy, which the authors identified as a special case, occurs when the organism uses a salient landmark as base reference and carries out all exploratory activity in relation to this point.

The authors also have their own categories, related to Morris' virtual maze:

Thigmotaxis represents a circular part of the path that is passed along close to the arena wall (...) We defined “circling” as an arc shaped search path, which occurred somewhere inside the arena but not close to the wall and with the same curvature as the arena wall (...) visual scan occurred when a subject remains in a fixed position and turns (...) Enfilading is composed of relatively small position corrections and non-strategic motions. During this search strategy, it seems that the subject performs a rapid search, small direction changes and some straight lines of walk on a limited area of the virtual space.

Though it's not directly usable for CatchBob! I have here an interesting account of various spatial strategies.

Studying people's movements in space

I am currently trying to find a relevant method to explicit CatchBob's players strategies (i.e. their movement on the campus, how they spread and explore various places). From my cognitive sciences course, I don't have so much about it apart from experiment with rats in watermaze. For that matter Morris' maze is interesting: a subject is tracked while it attempts to escape onto a platform in a swimming pool (Morris, 1981). In my experiment, I have 3 subjects and they are not in a watermaze, but a campus. I found this interesting software that helps researchers to analyse their data. They get results like:

I also like a lot the representation with all the paths, so that we can compare:

My interest is to move forward from this kind of representation to something more descriptive, with just relevant points. Like for instance, there are two kind of strategies: exploring vertically/horizontally. It's then amatter of finding constant design patterns into groups' strategies.

Studying people\'s movements in space

I am currently trying to find a relevant method to explicit CatchBob's players strategies (i.e. their movement on the campus, how they spread and explore various places). From my cognitive sciences course, I don't have so much about it apart from experiment with rats in watermaze. For that matter Morris' maze is interesting: a subject is tracked while it attempts to escape onto a platform in a swimming pool (Morris, 1981). In my experiment, I have 3 subjects and they are not in a watermaze, but a campus. I found this interesting software that helps researchers to analyse their data. They get results like:

I also like a lot the representation with all the paths, so that we can compare:

My interest is to move forward from this kind of representation to something more descriptive, with just relevant points. Like for instance, there are two kind of strategies: exploring vertically/horizontally. It's then amatter of finding constant design patterns into groups' strategies.

Catchbob results follow up

I just run few chi squared analysis on my CatchBob! data. It seems that people who are not provided with an automatic display of their partners' positions reconfigure more their strategies (= campus exploration) over time. It's another interesting argument! Besides the task division is a bit different in the two conditions: people without the location awareness tool explored less zones. This is an another important results in terms of performance: groups in both conditions approximately have the same performance but the ones without the tool explored a bit less.

Trip in Paris

I'll be in Paris till Friday, meeting some people, trying some mobile applications and giving some talks at R&D labs.

Paper in Psychnology about socio-cognitive functions of space

My literature review about the socio-cognitive functions of space has been accepted for a special issue of Psychnology about Space, Place and Technology. The paper is entitled "A Review of How Space Affords Socio-Cognitive Processes during Collaboration". Here is the abstract:

This paper reviews the literature about social and cognitive functions of spatial features used when collaborating in both physical and virtual settings. Those concepts come from various fields like social, cognitive as well as environmental psychology or CSCW (Computer Supported Collaborative Work). We briefly summarize the social and cognitive affordances of spatial features like distance, proxemics, co-presence, visibility or activity in the context of physical and virtual space. This review aims at grounding in an explicit framework the way human beings use space to support social interactions. This review can be used as a starting point to design efficient applications that take spatial context into account.

It will be online in few weeks I guess.

Poster for HCI 2005

I'll be at HCI 2005 In Las Vegas to present a poster abou my PhD thesis. The short paper might be downloaded here. It's mostly the rough description of my first experiments' results.

A Mobile Game to Explore the Use of Location Awareness on Collaboration by Nicolas Nova, Fabien Girardin and Pierre Dillenbourg
This contribution presents an ongoing study focused on how location awareness feature modifies collaboration in the context of mobile computing. First it describes the environment we designed and implemented in the form of a mobile game called CatchBob!. This application running on TabletPCs engages groups of three participants in a collaborative treasure hunt over our campus. The game is used as a platform to run field experiments to get empirical results about how information concerning partners’ whereabouts impact collaborative processes. We are interested in processes such as division of labor, the inferences made by participants about others activities and the building of a shared understanding of the situation. Players can communicate by drawing information on the TabletPC that displays a campus map. Those drawings are broadcasted to each participant. Finding the object was achieved through a proximity sensor that indicates how close the user was from the virtual object. Collaboration among the peers lies in the fact that they had to surround the object with a triangle formed by their positions. We tested two experimental conditions. In one condition, users could see their partners’ positions. In the other condition, participants were not given location-awareness. This poster presents the game, how it enables us achieving our goals and specifies which kinds of data we are able to extract. We then report the results of a study we conducted. According to our ongoing experiment, there seems to be no differences between the two conditions with regard to the task performance. However, players without the location awareness indications have a better representation of their partners’ paths. This is due to the fact that they annotated more the shared map: positions indications (to compensate the absence of location-awareness) but also directions and strategy messages.

Meeting with my PhD advisor

- keep doing the analysis: sequential analysis + nasa tlx + overlaps/backchannel/dispersion/covariate with bob's position + regression/multilevel modelling...- when looking at means differences, we have to be careful about the differences. For instance, a difference of 1 units (i.e. between 4 and 5) even significant must be stated carefully. The amplitude is not that big. - there might be an inertia caused by knowing where are the partners. - the performance measure in this task is the group path length. HOWEVER we did not really design the game to have a clear performance measure (this was just meant to find winners), the REAL performance if the accuracy/quality of participants' model of their partners' (intents, goals, activity). - the (planned) strategy, if well established imply less coordination on the field. AND coordination implies awareness of others OR messages. Now there might be a relation between the (planned) strategy and awareness/messages. Awareness...Messages... Errors BUT there are also link between awareness and strategy, awareness and message, awareness and errors. - That is why we need to discriminate the different strategies groups had (what I sketched on Pierre's whiteboard: three different group path + the fact that some groups stick to their strategies and others reshape it + when people converge to Bob, some go directly and others still wander around). - Discriminating people's strategies lead us to find descriptors: if we wanted to replay the games, which paramaters would we need? A simulation of this would imply to play with these parameters so that the agents use the same strategies as the ones encountered (emergence?): dispersion at the beginning + a certain speed to converge to bob. - we want to access to groups' strategies! Which indexes: angles made by the 3 persons? dispersions index (then have a graph: y=mean distance between the 3 persons, x=time) + we can define the EPFL as a graph/network amde up of different PLACES connected (like CO -> Archi, Elec, Unil, Esplanade....). So: 1) represent the EPFL as a graph 2) Draw players' paths (groups) thanks to 2a) path logs 2b) autoconfrontations, 3) Define categories (2/3) like for instance if they explore this graph in width or height: that may define strategy descriptors. - FIND literature about how can we describe people's trajectors: look at rat experiments in psychology. How can we analysed this?

- does the presence of the awareness tool trigger other/different strategies? What I saw is that people with AT stick to their strategy and the others modify it, reshaping it by annotating the maps.

- a mobile representation: strategy depends on: A's position, A's signal, B's position, B's signal, C's position, C's signal. In the condition with awareness tool, people have everybody's positions + potentially access to everybody's signal. In the condition without the tool, people have just their position + potentially access to everybody's signal. Agents use those inputs to build their strategies. We'll have to use them to simulate the task

Awareness and cooperative work in a café-restaurant

This paper (in french even though the abstract is in english) might be of interest, I'll peruse it in the train tonight: Awareness and cooperative work in a café-restaurant by Béatrice Cahour and Barbara Pentimalli. In Activités Vol. 2 No 1.

In this paper we present a workplace study of the collaborative activity between cooks and waiters in a restaurant/coffee-house. We show how awareness is linked to the attention mechanisms of the participants and how their level of awareness is constantly varying. First of all, using video footage, we analyse the functions of awareness between the co-team members through their peripheral attention, over-hearing and kinaesthetic perceptions: how they need to use shortcuts in their displacements and rapid communication so as to gain time, how they need to avoid collisions in a narrow space, etc. Secondly, based on 'explicitation' interviews conducted with two waitresses, we then present a detailed analysis of the variations in their attention which are linked not only to the cognitive overload present in their current activity but also to the objects and persons which are pertinent for this same activity. KEYWORDS: Levels of awareness, foci of attention, cooperative activity, socio-cognitive dynamics, communication, pluri-sensoriality.

Augmented video as a mechanism for improving collaboration and decision making

I stumbled across this research project at University of San Diego: Passing the Bubble:

We study augmented video as a mechanism for improving collaboration and decision making. Our special focus is decision making that depends on decision-makers and information analysts sharing their understanding. The interaction between these two groups involves commanders passing their intent to their information analysts, then refining their plans and decisions on the basis of information gathered by their analysts, often using AUV’s. We examine how different ways of augmenting video differ in how cognitively efficient they are in creating shared understanding. Most people who have seen augmented video assume it to be a powerful mechanism for communicating complicated objectives and facts about situations. But little or no work has been done on:

1. determining which of the many ways of augmenting video are most effective, and
2. developing a cognitive theory that explains why these different methods differ in their potency and cognitive efficiency. Video, if properly annotated, promises to enrich and reshape collaborative exchanges. Our goal is to understand how to maximize the impact such videos will have on collaboration.

Why do I blog this? Might be interested for my thinking about to use replay tool in my research project.

My current phd bibles

CatchBob! analysis: division of labor

I wrote a script to parse the CatchBob! logfiles. It allowed me to get interesting indexes with regard to the collaborative behavior of the players. CatchBob is a treasure hunt; thus it's a spatial task in which participants have to collaborate to do the shortest path to the object (that's what they are required to do during the experiments we conducted). That means that the division of labor concerns the way they spread over the campus and how they explore it thanks to the proximity sensor of their tool. What I would like to express here is that an index of the division of labor among the group would be the number of "zones" explored by each player. I divided the campus in a certain nuber zones that correspond to squares (20meters since it's the accuracy of our positioning device). My script gives me this and other interesting stuff:

• the number of squares explored by each player
• the percentage of squares explored by each player (not so useful)
• the number of backchannel for each player: the number of square explored more than 1 time by a player
• the path overlap between A, B an C: the number of square explored by 2 or 3 players (for each player).