references about sketching interfaces

• freehand drawing interfaces explored in 60s (light pens ) but failed, the mouse won
• now tablets+ pda offer drawing tools: renewed interest towrds freehand drawing interfaces
• especially used in design, architecture,a bit for military operations
• support creative design but not so many work in coordination or field activities

general thoughts about sketching: V. Goel, Sketches of Thought, MIT Press, Cambridge MA, 1995. Robbins, E. (1994). Why Architects Draw. Cambridge, MA: MIT Press. Tversky, B. (2002). What do Sketches say about Thinking? AAAI Spring Symposium

historical dimension: I. Sutherland. Sketchpad - a Graphical Man-Machine Interface [Ph.D. Dissertation]. M.I.T., 1963.

sketching and groupware: Greenberg, S., Hayne, S., & Rada, R. (1995). Groupware for Real-Time Drawing. London: McGraw Hill.

systems used in a design context: 1996 Gross, M. D. "The Electronic Cocktail Napkin - computer support for working with diagrams," Design Studies. 17(1), 53-70.

1996 Gross, M.D. and E. Do. "Ambiguous Intentions: A paper-like interface for creative design", Proceedings ACM Conference on User Interface Software Technology (UIST) '96 Seattle, WA. 183-192 Landay, J. A., & Myers, B. A. (1995). "Interactive Sketching for the Early Stages of Interface Design, CHI '95 - Human Factors in Computing Systems", (pp. 43-50). Denver, Colorado: ACM Press.

system used by the US army Forbus, K., Usher, J., and Chapman, V. (2003). Sketching for military courses of action diagrams. In York, A. P. N., editor, Proceedings of the 8th international conference on Intelligent user interfaces, pages 61–68, Miami, Florida.

Relevance of using map annotations

Forbus, K., Usher, J., and Chapman, V. (2003). Sketching for military courses of action diagrams. In York, A. P. N., editor, Proceedings of the 8th international conference on Intelligent user interfaces, pages 61–68, Miami, Florida.

"A serious barrier to the digitalization of the US military is that commanders find traditional mouse/menu, CAD-style interfaces unnatural. Military commanders develop and communicate battle plans by sketching courses of action (COAs)."

"One task where sketching is used extensively is when military planners are formulating battle plans, called Courses of Action (COAs)"

"A COA consists of a sketch and a textual statement. The sketch conveys a number of crucial properties of the situation and the plan. First, it includes a depiction of what terrain features are considered important. (Sometimes COAs are drawn on acetate overlays on maps, sometimes the basic terrain description itself is simply sketched.) The results of analyzing terrain, such as possible paths for movement (mobility corridors, avenues of approach) and good locations for different kinds of operations are identified. The disposition of troops and equipment, both for friendly (Blue) forces and what is known about the enemy (Red) forces is shown by means of unit symbols, a vocabulary of graphical symbols defined as part of US military doctrine. This graphical vocabulary also includes symbols for tasks, such as destroy, defend, attack, and so on. The COA sketch indicates a commander's plan in terms of the tasks that their units are assigned to do."

"Today's speech systems have serious problems in noisy environments, especially when operators are under stress. "

Why do I blog this? I am gathering some usage of map annotation tools like the one we're using in CatchBob. It seems that military use of such systems are relevant. The system they present is for desktop computers. I am looking for ubicomp military applications (working od pda or tablet pcs).

Map Sketches Promotes Collaboration

Another reference that might help me to analyse the map drawn by CatchBob! participants:Heiser, J., Tversky, B. and Silverman, M. (2004). Sketches for and from collaboration. In J. S. Gero, B. Tversky, and T. Knight (Editors). Visual and spatial reasoning in design III. Pp. 69-78. Sydney: Key Centre for Design Research.

Pairs of collaborators worked side-by-side using a campus map to design and produce an optimal emergency rescue route. Copresent collaborators shared a map; remote partners were separated by a barrier and used separate maps. In the co-present condition, gestures on the maps, notably pointing and tracing, served to focus attention and to communicate solutions. A shared diagram increased the efficiency of the collaboration, the product of the collaboration, and the enjoyability of the collaboration.

Why do I blog this? We are trying to show (with CatchBob!) that the map annotation feature is very relevant and could support collaboration efficiently (better than just providing a location-awareness tool since it promotes strategy discussion on the map).

Sketching habits in GIS

A study of people''s sketching habits in GIS by Andreas Glaser.

Abstract: Sketching is traditionally associated withdoodling simple strokes on a piece of paper.Only few professionals outside of design andthe fine arts have recognized the expressivepower of this intuitive modality. However,sketching seems particularly well suited tocapture objects and situations in a spatialenvironment, such as geographic space. To learnmore about the techniques and strategies peopleuse when sketching, a survey of sketching wasconducted. The study showed that paper andpencil sketches contain mostly simple andabstract objects that are composed of only fewstrokes. The spatial configuration of a sceneis primarily expressed through the topologicalordering of objects relative to each other.Metric relationships are used to refine spatialconfigurations. These and other findingssuggest that sketching is an appropriatemodality to interact with a computer where onewants to describe and capture objectconfigurations in a spatial environment, suchas a geographic information system (GIS).

Keywords: freehand sketching - human computer interaction - human subject testing - multi-modal user interfaces - spatial querying - spatial information retrieval in GIS

Why do I blog this? Since I am working on the coding scheme to analyse the map annotations in the CatchBob experiments, this kind of paper is useful.

Modeling Collaborative Activities

I still have to find ideas and methods to include some computational aspects in my PhD (I'm in a computer science faculty geeeeeeez). That's why I am looking at some paper about modeling collaborative situations. I stumbled across this one after meeting the author: Analyse et modélisationdes activités coopératives situées: évolutions d'un questionnement et apports à la conception which is in french. It deals with some epistemological concerns abouzt why using such model and how it could be fruitful for CSCW.

This paper summarizes the different steps of a research program which focused on the analysis, modelling and equipement of cooperative activities. We attempt to make explicit the various succesive theorical underpinnings and to show how they have influenced modelling and design issues.

How to develop a collaborative mobile game

My EPFL co-worker Fabien wrote a postmortem report (.pdf) about our CatchBob! project in which he describes the whole development process. He presents how we designed CatchBob on two platforms (namely iPAQ and TabletPC) and how he implemented it. The report goes throught the whole thing, dealing with the game architecture, the user interface, the positioning feature as well as the communication tool (that allows to annotate the map). Here are the key points for people in a rush:

• CatchBob! uses WiFi-enabled TabletPC as clients They use the Wireless Network sniffing capabilities of Place Lab to locate themselves by listening for radio beacons.
• The positioning technique is based on a propagation model using the degradation of the signal strength of a radio wave over distance in space. We implemented a very simple triangulation using a centroid algorithm. It positions the user at the center of the scanned nearby access points by computing an average of their x, y location and taking the signal strength as a weight. We hence achieved to get a rough positioning accuracy (10-20 meters) which was sufficient for our needs.
• CatchBob! is built on a client-server communication model. Every 30 seconds, the clients broadcast their positions, commands and annotations via a centralized server. The communication is done over SOAP. Clients are in a pull mode in order to retrieve and synchronize the data.
• The map, annotation, and awareness rendering on CatchBob! interface is generated in Java2D on a static background displaying the campus.

The end of the report aims at enlarging the scope, discussing interesting issues such as the design of engaging technology (for the user's point of view), the relevance of 3D positioning and the huge potential of map annotation.

This document might be seen a good summary of how Fabien achieved the mobile game development. Besides, it's absolutely human-readables. Non-tech-savvy people should not be afraid to have a glance if they are interested in how to develop a mobile game.

Catchbob Map analysis

Thanks Fabien for your help! We know have a php script that automatically generates a map of CatchBob! users' path. This map is going to be of interest since I want to calculate the number of error an individual made while drawing his own path like this one (here it is correct). In addition, I will also check if the partners drew this path correctly; on the following drawing, there is a mistake (in red): The main problem here to calculate the number of error is the low accuracy of the positioning tool; I'll just compute a rough error index. Here is the modethodology:

In the post-game questionnaire, we ask players to draw 3 maps: the path they took on the campus plus the map they thought their partner 1 and 2 did. Thanks to the client logs server, we can generate a map of each paths. It allows us to calculate different number of errors that are spatial modelling indexes:

• comparison the path player A drawn about where he went to his/her real path. It represents the quality of A’s representation of the physical environment.
• comparison between the path player A drawn about B or C to B or C’s paths. It represents the quality of A’s representation of B and C’s behavior in space. For example, on figure 3, we can compare the way A drew his path (in green on figure 2 aboce) to how B drew it (in red on figure 3 above).

Calculating the number of errors is made through the use of layers on printed material. An error is both a place where the player have not been or a place where he went but he forgot to draw it. Three criteria were defined to describe what is an error: distance (if the line is longer than the maximum size of our campus corridor), presence of an obstacle (door/wall/glass), walking back is not perceived as an error.

ecscw paper

just submitted a paper for but it does not worth it. well maybe a good way to get some feedback...

The future of location-based experiences

Steve Benford (University of Nottingham'sMixed Reality Laboratory) recently wrote a JISC Technology and Standards Watch report called Future location-based experiences which is of great interest for people into the locative media area.

This Technology Watch report considers the relevance of location-based experiences to education, discussing potential applications, reviewing the underlying technologies and identifying key challenges for the future.

The research community has already demonstrated a variety of location-based experiences that are of relevance to education including: information services and tour-guides in which information is delivered in situ; educational games in which a combination of mobile and online users learn together; support for field trips in which the technology provides access to learning materials during a visit to a site of special historical or scientific interest; and support for field science in which learners actively gather data from the environment for subsequent analysis back at base. (...) This report concludes that location-based experiences could indeed introduce significant benefits for education in schools, colleges and universities, especially when they connect location-specific data delivery and capture to subsequent reflection and abstraction back in the ‘classroom’. However, some serious challenges need to be met first. Technical challenges include dealing with the uncertainty of positioning and connectivity and also supporting interoperability between the very diverse set of technologies involved, in particular by developing flexible middleware support. Organisational challenges involve addressing serious privacy concerns, integration with current e-learning services and dealing with the potential culture clash involved in encouraging widespread use of mobile devices in educational environments. Recommendations are for the educational community to conduct a series of pilot experiments on different wireless test-beds while consulting closely with users and their representatives over privacy issues and with mobile operators about future service provision.

The report covers pretty well the are of location-based services, offering interesting insights related to potential challenges. It a good compendium to the last Communication of the ACM issue I mentionned last week.

Sketch maps as an indicator of internal cognitive maps

A very informative paper by Mark Billinghurst and Suzanne Weghorst about the use of sketch maps to provide a measure of internal cognitive maps of virtual environment. The Use of Sketch Maps to Measure Cognitive Maps of Virtual Environments.

Cognitive maps are mental models of the relative locations and attributes of phenomena in spatial environments. Understanding how people form cognitive maps of virtual environments is vital to effective virtual world design. Unfortunately, such an understanding is hampered by the difficulty of cognitive map measurement. The present study tests the validity of using sketch maps to examine aspects of virtual world cognitive maps. We predict that subjects who report feeling oriented within the virtual world will produce better sketch maps and so sketch map accuracy can be used as an external measure of subject orientation and world knowledge. Results show a high positive correlation between subjective ratings of orientation, world knowledge and sketch map accuracy, supporting our hypothesis that sketch maps provide a valid measure of internal cognitive maps of virtual environments. Results across different worlds also suggest that sketch maps can be used to find an absolute measure for goodness of world design.

Why do I blog this? I am wondering if there are connected studies in the real world. Is a sketch map also a good indicator of the internal cognitive map of the physical environment?

Mobile services and physical closeness

It's 3 years old but still very up-to-date from my point of view: Marko Athisaari's talk at Doors of Perception 7: here are few quotes I found relevant:

My topic is proximity. By proximity, I mean, quite simply, physical closeness; and many among you will ask, “How close is close?” But I mean the stuff that happens between the one-to-ten centimetres range and room-size interaction, and what people do, what kind of activities they undertake in that kind of space. (...) four kinds of services and product services value that people might get in physical closeness:

1. People in places: while we talk about devices being connected, and everything being connected in a technological sense, social interaction will be a prime driver in the future as well, even in technologically enhanced social interaction. By this, I don’t mean that we’ll supplant very good ways of communicating, like me speaking to you one-to-many now, but that other forms of communication overlay it – if there’s wireless coverage, you could be maybe discussing what I’m saying among yourselves. (...) he simplest and most important one, which has immediate implications and is quite near in the technological roadmap, is sharing different kinds of digital items, whether they’re personally created or otherwise, in physical closeness. We know from evidence in Japan where, for example, imaging phones have been around for quite a while, that across the table people won’t send the image across the cellular network, they prefer to show it on the device.
2. My things with me: We know that mobile portable products and appliances are very much linked to personal and intimate items, the kinds of things that one has an emotional relationship with. One can see that the relationship of people with their phone books is far from rational. (...) This includes not only items like a phone book or co-ordinates for other people in our social network, but also our personal and intimate media, like photographs. They need to be stored somewhere close by;
3. Enhanced Spaces: means content, digital content that’s local, related to this space, or the shopping environment. [the classical location based services postit like? nicolas]
4. Safe Consumption refers to the fact that in physical proximity, a lot of issues concerning trust and security are tacitly taken care of .

Why do I blog this? I guess it's a smart summary of proximity-based services, dealing with various dimensions (like safety or belongings).

Location systems are not yet ubiquitous

An interesting short paper in Communication of the ACM about location-based services by Gaetano Borriello, Matthew Chalmers, Anthony LaMarca, Paddy Nixon: Delivering real-world ubiquitous location systems :

Location systems are not yet ubiquitous, but are increasing their accuracy, coverage, and availability, and maintaining moderate cost. A number of candidate technologies are already in the marketplace, and new ones are continually being developed. Alongside the developments of such infrastructure, commercial and research-oriented applications are being developed in significant numbers. In the future, the development of new location systems is likely to be influenced by market trends in the hardware and software platforms of mobile devices—most particularly cell phones—as much as the technology options within location systems themselves. Privacy is clearly a major factor in location systems’ development and deployment. (...) Developers of applications will have to accept variation in the accuracy and availability of location data for some time. (...) we see significant potential in deliberately showing some of the seams in the infrastructure of ubiquitous computing systems, going beyond the simple displays of signal strength in phones and laptops, to offer people maps of availability of WiFi coverage and location services.

Why do I blog this? Well it summarizes pretty well the main concerns about location-based services. It's also very mattheau chalmers-oriented since it deals with what his lab does about seamless design; which is of great interest. I am trying to work on the very topic of accuracy, my current experiment is focused on comparing three different way of using this kind of mobile gear:

• without location awareness
• with a precise location awareness tool
• with an unnacurate location awareness tool

Results might be different than what the common sense leads us to expect... Besides, read the whole Communication of the ACM issue, it's pretty well done; it gives a good summary of what is happening lately in the field.

France Telecom\'s journal of social software

French multinational France Telecom now has a very relevant blog about social software and digital communities: affinity. Few posts at the moment but it sounds promising.

Affinity is a journal of research, innovation and ideas in the emerging field of social software. Jointly written by Wanadoo, Orange and France Telecom R&D, we intend to coalesce France Telecom's expertise in digital communities and social software. We believe that the recent vibrancy of innovation around digital communities, social software, social media and digital identity is transforming what it means to be a telecommunications provider in the 21st century.

For France Telecom's consumer businesses, Orange and Wanadoo, social software represents a nexus of mobility and broadband, entertainment and communication that will help us to transform telecommunications from standards, networks and protocols, to conversations, relationships and a essential mediating Life Services.

Why do I blog this? Even though I know few folks at France Telecom (in France mostly), it's always tough to know what they are up to. This blog is a nice footpint about their activities in the domain of social software. I hope they could expand to broader topic like location-based services or multimodal interactions (I know they do lots of stuff about it lately).

A virtual colloquium about spatial cognition

Barbara Tversky ( from Stanford University) is going to hold a virtual colloquium about spatial cognition on Friday, February 25, 2005, 1:00 - 2:00 p.m. EST. It is about 'Multiple Mental Spaces'

We engage in spatial cognition from the first moments of life: where to look, where to reach, where to go. Each of these acts entails a different space and different set of behaviors; the behaviors become fine-tuned through experience. In contrast, talk about space does not directly access this knowledge; instead, it seems to be mediated by cognitive constructions that are only schematically based on this knowledge and that may introduce systematic bias and error.

Why do I blog this? Barbara Tversky is one of the most important researcher with regard to spatial cognition and mental models of space. Most of her work is headed towards the space of the body (Morrison and Tversky, 2004), the space around the body (Franklin and Tversky, 1990; Bryant, Franklin, and Tversky, 1992; Tversky, Kim, and Cohen, 1999), the space of navigation (e.g., Tversky, 2000), the spaces created by people to augment their cognition (e.g., Tversky, 2001) like diagrams, language, gesture and other behaviors.

Discussion about location awareness\' definition

Mauro posted a definition of 'location awareness' on his blog last week. He defines it as "A continuous availability of the spatial positions of the agents involved in the interaction for the agents themselves.". Since I am dealing a lot with this fuzzy notion on awareness, I would like to comment a bit on that. Technically speaking, theories about 'awareness' are not so cognitive psychology oriented, it's more related to HCI or CSCW concerns (even though some cognitive sciences researchers now work on that notion). Dourish and Belloti (1992) have given one of the best-known definitions for awareness: “awareness is an understanding of the activities of others, which provides a context for your own activity”. Drawing on this, location awareness would be "the understanding of the others' position" which of course as Giles supposed (in a comment of Mauro's post) imply some cognitive processes to be activated.

Moreover, Gutwin and Greenberg (1999a) state that awareness:

• Is knowledge about a state of the work environment in a limited portion of time and space.
• Provides knowledge about changes in that environment.
• Is maintained by all the interactions between the team-mates and the environment.
• Is a part of an activity (completing a task, working on something…). Maintaining awareness is not the purpose of an activity. Awareness is used to complete a task.

The definition goes further here since they state that awareness is about having a constant flow of information (perceived and updated). Another interesting point is that awareness of others is not maintained for the sake of it but is a part of an activity (at least it's a presence indicator).

I would say that awareness it just limited to the perception plus the updated of information; about others' position if we talk about location awareness. Then other cognitive processes are implied like inferences ("A is here then I infer that..."), communication... Of course, location-positioning being technologies allow people to improve their location awareness. The problem is that we don't know yet how it impacts the way they collaborate, which is ultimately the topic of my phd thesis...

I tried to elaborate more on the location awareness dimensions in a previous post. Here is a figure about this (I know it's ugly):

Paper about ubiquitous computing examination in natural settings

Salembier, P., Kahn, J., Calvet, G., Zouinar, M. & Relieu, M.(2005). “Just follow me”. Examining the use of a multimodal mobile device in natural settings (.pdf). In Proceedings of the HCI International Conference, July 22-27, Las Vegas. Lawrence Erlbaum Associates.

This paper describes a method and a piece of equipment for capturing data about the use of a mobile multimodal device in natural settings . The equipment includes an instrument worn by the user to capture his or her perspective while moving, a means of capturing a wider view of the context and a multimodal data logger.

Why do I blog this? I already mentioned what this team does (here and here). Their methodology is pretty well thought, relying on various sources of information to capture what's happening on the field. The self-confrontation of the players to their interaction in various context is of relevant interest. I am looking forward to read more about the results. Besides, the tool they use to code the interaction looks promising.

Discussion with my phd advisor about data analysis

We talked about the CatchBob data analysis. We're going to use both qualitative and quantitative techniques. Note: we have to be careful to balance the number of Bob's location: there should be the same number of Bob at 'la coupole' as in 'faculte d'informatique'

1. about the tabletPC screenshots: their map annotations (from the 3 players): we came up with the following coding scheme to analyse the map annotation: - content: position/direction/signal strength(proximity sensor)/strategy(stay calm)/off-task notes/acknowledgement/content-free acknowledgement - mode: textual (+numbers)/graphical - position on the map: site-specific/non site specific - intent/pragmatics: announcement/request

Let's have a quantification of each messages: - number of messages per player/group - number of messages per category per player/group check if there are still position request in the condition "with location awareness tool". It if it's the case, it might be an indicator that this query and the answer are not only related to the position but also to something higher in terms of intent modeling...

2. Path drawings: I have the path of each participant (thanks to client+server logfiles) + their drawing (A drew is path and the path of B and C)

Calculate the number of errors about A's path: path drawn by A/real path by A - places where they have not been - place they were but forgot to draw PLUS: real path by A/A's path drawn by B and real path by A/A's path drawn by C The tricky thing here is to define what is an error, we need some criteria like: - distance (higher than the maximum size of epfl corridors) - no visibility between the two representations - door/wall/glass - walking back just a while is not an error

Then: we can compute the number of errors A did about B: E(A,B) as well the number of errors A did about C: E(A,C) Thanks to this errors evaluation we have an relevant evaluation: the quality of the spatial representation that A had about B and C: an indication of Spatial Mutual Modeling MM(A-BC)= [E(A,B)/E(A,A)] + [E(A,C)/E(A,A)] (we use the error about A own's position as a weight here that indicate the accuracy of his representation of space).

3. Self-confrontation to the replay tool look for explicit stuff like: “I understood that you/he/she/they wanted to…", “I did not get that you/he/she/they", “you/he/she/they did not understand that…": - critical incidents ('I thought you were doing this') - 'best practices' (good modeling)

then we can have a catalogue/typology of mutual modeling acts -> transcript just those moment. That will be good to illustrate + to find new variables we did not think about...

A crazy path taken in Catchbob

My experiment today went well, the players took crazy paths on the campus to find the virtual Bob. Their strategy at the beginning was way different than the others, which is interesting from the researcher's point of view. They took a very long time (nearly 29minutes). In fact, even though they played with a location of the others positions, it turned out that it did not help them that much.

Paul Baron\'s add on to my OPAN PLAN notes

Here is a small add on Paul Baron sent me, about my OPAN PLAN notes. Unfortunately, my spam guardian did not accept it... The point is that Paul wanted to share some of the experience that he had with Mogi. Thanks Paul!

About what steve benford mentioned (interweaving physical and digital interaction): 1- Mogi was also supposed to foster a collaboration between the online players who could see the whole country in high res at any time and the street players who might miss a rare object just cos they were a little far from it (more than 1km) and it would not appear on their screen. In practice, it only really happened to me once or twice, and it was the creator of the game pingging me to let me know about an object there.

About Mathew chalmers (uni glasgow): geolocation tech limit 2- In the Mogi game, you can at any time switch from celltower signal to real GPS. Players would take advantage of this in a few different ways. First of all, refreshing your position non stop in GPS when you are not moving, will usually find you at the exact same position. However, doing that with CellTower triangulation, would usually yield fluctuating results that might in some cases make you appear closer to an object that you initially were and therefore let you pick it up if you happened to be suddenly "moved" to less than 400m from it when originally the GPS was showing you at for example 450mtrs.

Meeting with Steffen P. Walz

Today we had a meeting with Steffen P. Walz from ETH Zurich. Steffen's research: looking at (pervasive) game design methods, ethnography... how that can be used in the computer aided architecture methods. Steffen: - he has 2 projects: a class called "ETH game": a location based learning/history game: depending on the location, history notes will appear + quizz + collaborative features (you need to answer to sites-specific question + proximity/range), wlan, 1000 people + how to optimize surveillance systems by using games (getting ahead of babydoc), a video tracking things - phd topic: using game design method to enrich architecture methods - "how do we deal with 5000 people fucking around with each other" - works on a book project about book and architecture (1. games, history, architecture 2. ubiquitous games... 3. how game design methods relates to/could enrich architectural methods). Might be interesting to write a chapter of it. - from an architectural point of view, it would be nice to have a topology of space/location that you can annotate, and why? - he points me to erick klopfer (MIT) - mad countdown: impressive scenario, event-based, site specific - raumtaktik: a board game based for urban planning (architectural game design:education) - - his phd is funded by MICS (Mobile Information and Communication Systems), mics is goind to an end, there's gonna be a mics2 (about 'sensor internet', sentient buildings...)... Prof. Karl Aberer ... steffen: using games for research about sentient building.

How to engage people into location-based annotation: - spatial-scale + number of people involved - critical mass of people with shared interest, how they can form a somehow 'community of practice': density of people - rewards (steffen), nicolas: especially true with mobile games that happens in the real world (social reward in community of practice)... steffen mentioned an interesting reward: a secret that you have to share but cannot... social statuts - ...