Tangible/Intangible

Aptative-use interface based on the tip of the nose

in

Beyond gestural interactions, this nose-based interface developed at Rensselaer Polytechnic Institute by Zane Van Dusen and Pauline Oliveros is curious:

"A computerized instrument that allows people to play music with the tip of their nose could give those who suffer from physical disabilities, such as cerebral palsy, the chance to experience music's positive effects. Not only could the interface allow for musical communication, it could also be adapted for speech, giving physically challenged patients the ability to form full sentences, rather than just providing yes or no responses. (...) Van Dusen's "adaptive-use musical instrument" overcomes these challenges with an inexpensive Web camera and specialized computer software that he wrote. The patient is placed in front of the computer, where they see live video of their face through a Web camera. Motion-tracking software places a red box on the tip of the person's nose and tracks the user's movement across an onscreen keyboard. The lowest notes are located to the left and the highest notes are located to the right. The outline of a rectangle around the person's face can be widened or narrowed in order to accommodate the patient's range of motion."

Why do I blog this? this is a low-cost user interface with some very interesting benefits. The very constrained and focused nature of this UI makes it appealing to some design challenges, especially in the context of kids.

My talk at Reboot9.0

in ,

Slides from my Reboot talk are here (pdf, 1.1Mb).

The presentation I made, entitled "Hybridization, fusing, melting, coalescence and salmagundi" was about hybridization. I basically gave an overview of what I find interesting in projects about hybridization of the digital and the physical, a sort of compendium of the consequences (from the cognitive to the architectural) and the implications. Take-aways of my talk are: - hybridization of the digital/physical are coming in a large variety of ways - leads to changes from the cognitive to the architectural levels - revisit false ideas: do not oppose the digital and the physical, less utilitarian future, digital takes room. - reality is complex, need to study situations (not just technologies)

Thanks Thomas for the invitation!

Robotic lamp

in

Via Fabien, AUR: a Robotic Desk Lamp by Guy Hoffman (MIT Medialab Robotic life group):

"AUR is a robotic desk lamp, a collaborative lighting assistant. It serves as a non-anthropomorphic robotic platform as part of my Ph.D thesis on human-robot fluency and nonverbal behavior. The lamp's design was conceived around an existing 5-DoF robotic arm, and is aimed to evoke a personal relationship with the human partner without resorting to human-like features. By retaining the lamp's "objectness", I hope to explore the relationship that can be maintained through abstract gestures and nonverbal behavior alone.

The lamp is animated using a custom pipeline enabling the dynamic control of behaviors authored in a 3d animation system, and will perform in a unique human-robot joint theater performance this spring."

Why do I blog this? looking at human-robot interaction for possible client work, this artifact is interesting to me because it reflects the convergence between robotics and ubiquitous computing. Non-anthropomorphic behavior seems IMO a very relevant approach, leading to projects about a new category of objects.

LEGO evolution

in ,

In the last issue of Escapist, there is a good piece about LEGO and games. It basically describes the different evolution and extension to the LEGO bricks. That part is interesting if you don't know what's up there but more relevant is the conclusion:

"To an extent, LEGO has always mirrored society. In the 1950s, the blocks were identical and interchangeable; in the '70s, you could buy mechanized kits to repurpose those blocks for many functions. Starting in the '90s, you could buy customized sets; now, there are online LEGO networks. We can imagine more innovation ahead, such as smart, networked, globally aware LEGOs with radio-frequency identification (RFID) tracking tags. (...) Inevitably, responding to the current zeitgeist, plastic building blocks will go open-source. The field of 3-D printers - "fabs" - is barreling along. In 10 years, maybe less, you'll have one on your desk, using Ldraw-based software to spit out LEGO-like knockoffs of your own design - thousands of them, for no more than the cost of the plastic.

Yet somehow The LEGO Group, given its high-tech savvy, will probably still make a fortune in brick-design licensing fees. Because LEGO has always mirrored society. Maybe once all those Mindstorms-trained robotics engineers grow up and get loose, it'll be the other way around."

Why do I blog this? the mirroring of the society is not very surprising (I guess marketing department take care of this) but it's intriguing to see how social and cultural changes are implemented in products such as toys. Besides, the 3d printing future seems curious and very well in line with LEGO's strategy.

Pong game played with a laser-tracking system

in

A colleague sent me this wallpong project (by Adrian Wong and Bhavin Rokad from Cornell University) this morning. It's a pong game played with a lase-tracking system:

"The Wall of Pong brings the arcade PONG game to the real world by projecting an image of the pong ball on any flat surface and allowing players to play the game with real paddles. The game also targets all three of our personal objectives for our ECE 476 final project: an interactive electromechanical system with fast operation. We wanted a project that involved interaction between the user and the system with very fast response times to user's actions to give a satisfying experience. We also wanted to stress a project with electromechanical components, as we wanted something tangible with our project instead of a project based entirely in software."

Why do I blog this? yet another good example to explain the fusing of the physical and the digital.

Augmented sensory room

in

Adam Montando just sent me a project he is working on: Visual Voice Pro is an intriguing new product:

"Visual Voice Pro will create an immersive, reactive digital playspace. The Visual Voice Pro installation comprises of a sensitive microphone, a computer with a data projector, and custom software written especially for the space. The microphone will listen to all the sounds in the room, from footsteps to laughter to singing or even banging a drum. The computer will then instantly process the sounds to create abstract, beautiful graphics that illuminate the wall via the projector. Different sounds will create different effects. Loud noises can create big effects, quiet noises can create subtle effects, high pitches or low pitches can create different effects."

Why do I blog this it's interesting to see that these interactive products are now sold, I'd love to see what kids do out of it.

Ultra-fast virtual air hockey

in

Among all the technologies designed by Spaceman Technologies, the one I found the most curious is this "Ultra-fast virtual air hockey":

"In 2001 Spaceman Technologies developed a large table-top virtual air-hockey platform for up to four players. Incorporating a proprietory ultra-fast velocity-sensitive multitouch touchscreen and a 500+ frames per second high-resolution LED display screen. The touchscreen is capable of tracking players' fast sweeping hand movements. A hand's velocity and point of contact with the virtual 'puck' can therefore be determined. This velocity and position data is factored into the physics of the simulation so that the puck responds in a realistic manner to contact with the players' hands. The high frame rate of the screen ensures that the puck image does not flicker even when moving at high speed across the table - giving an 'analogue' feel to the game."

Why? because it's a sort of hybridization of real and digital artifacts which seems playful. And it's spot on a talk I am working on for next week (need examples to show).

Brainwave detection in a Darth Vader outfit

in

The SFgate gives a quick overview of how Neurosky aims at building brain wave-reading toys and video games (use of EEG). Nothing really new compared to other announcement we read about this company but I was intrigued by the way the interface itself. The journalists describes it as a "crude biofeedback device cloaked in gimmicky garb":

"A convincing twin of Darth Vader stalks the beige cubicles of a Silicon Valley office, complete with ominous black mask, cape and light saber. But this is no chintzy Halloween costume. It's a prototype, years in the making, of a toy that incorporates brain wave-reading technology. Behind the mask is a sensor that touches the user's forehead and reads the brain's electrical signals, then sends them to a wireless receiver inside the saber, which lights up when the user is concentrating. The player maintains focus by channeling thoughts on any fixed mental image, or thinking specifically about keeping the light sword on. When the mind wanders, the wand goes dark. (...)"

(Associated Press photo by Paul Sakuma)

Why do I blog this? given the difficulties to use EEG (number of electrodes, calibration, use of gel), I found intriguing the use of this outfit. Besides, the convergence between toys/games industries with ubicomp technologies is fascinating.

Hacking the wiimote

in

The WSJ has an article about how people re-engineer the Nintendo Wii controller to do all sort of things. This is done through downloading free software on the Web and tweak the code to re-assign the control/movements to specific commands. See wiihacks.blogspot.com and WiiLi.org. Some excerpts I found interesting:

"what has most captivated hackers is a mechanism inside the Wii-mote called an accelerometer that can detect its speed and direction of motion. It is the accelerometer, made by Analog Devices Inc., in Norwood, Mass., that allows Wii players to use their remotes to act out whatever game they're playing, whether it's casting with a fishing rod or swinging a tennis racket. (...) Nintendo says it is surprised by efforts to reprogram the Wii-mote and discourages the practice. "The Wii Remote was created to play on the Wii system only," says Anka Dolecki, spokeswoman for Nintendo. But all the interest in the Wii-mote could have an upside for the company. The dozens of free games on the Web that incorporate the Wii-mote have helped add to the buzz surrounding the console. (...) Some companies see possible business applications with the Wii-mote. Rick Bullotta, vice-president of SAP Research, an arm of the German software giant SAP AG, is looking at ways to integrate the Wii-mote into their clients' manufacturing operations. He envisions factory and warehouse employees walking through facilities pointing and waving Wii-motes to monitor and control machines."

Why do I blog this? the wii hacks are more and more documented, it's interesting to see how this innovation from the gaming area can lead to change in other area.

Bioscope: materialize the sketches made by gestures

in

Bioscope is a project by intrepid friend Jean-Baptiste Labrune:

"a device that materialize the sketches made by gesture while talking to somebody or being in a creative or expressive process. The goal is to be build a creativity research tool that manifest visually the evolution of concepts and ideas. The idea is to bring another perspective on the creative process, like in the amazing movie of Henri George Clouzot called “le mystère Picasso”. In this film, Picasso’s sketches unfolds through time since to an innovative shutter camera system developped especially by Clouzot. The temporal aspect of the artefacts is the focus and not the spatial aspects. The concern about immateriality has been pushed to the point that the only artefact remaining of this period is the film, all the other productions by Picasso have been destroyed…"

" Traditionnally, the methods to do this involve either slow-motion photography, animation, film, video or compositing. I have decided to use a 3 axis IMU to capture gestures and then recompose it in a 3D scene. This picture shows 30 sec of movements while talking about a videogame concept. The accelerometer records position in space but is also used to edit video in a 3D environement according to the gestures of the user that act as metadata. Browsing and exploring the video footage is done by the 3 axis data manipulation."

Why do I blog this? the idea of visualizing the activities involved in a creative process and represent it in a way that can inform the design itself is very curious and pertinent. Besides, this is of interest to me because of some thoughts regarding the user experience of gestural interactions. Would it be possible to use such a device to help the design of gestures for certain game design for instance?

Spatial gestures challenges

in

Some quick elements about the challenges regarding 3D spatial gestures to control digital information are described in "Gameplay issues in the design of spatial 3D gestures for video games" (by Payne et al. 2006):

" The seemingly natural and intuitive ease with which gestures could replace command menu structures and cumbersome or intrusive controller mechanisms was seen to empower users with increased control over their intent. However the effective implementation of gestures is complicated. Issues related to human spatial motion awareness, user performance differences, cognitive/semiotic confusion and user feedback all complicate the implementation of spatial gestures in videogamessuch as a fireball or hurricane kick. (...) spatial gestures present their own problems in relation to: how to present 3D gesture feedback, user performance differences, how to instruct/learn user gestures, what are familiar semiotics for 3D gestures."

Why do I blog this? scanning the literature about this topic.

Wii Helmet

in

(via) Finally, there is a Wii Helmet, but it is an april fool:

"Problem is the average gaming geek is just not up to the strenuous task of vigorous arm movement for longer than 10 minutes. This makes those extended play sessions a thing of the past... and who wants to beef up at the gym just to play video games on your couch? We sure don't.

Fortunately for cream puffs everywhere the WiiHelm is now available at ThinkGeek. Simply lock your existing Wiimote into the stylish white helmet and free your hands for relaxation... or other important tasks. After extensive scientific study it was found that manipulating muscles in your neck for 10 minutes uses 64% less energy than waving your arms about like a lunatic. The WiiHelm works great with all of your favorite Wii games and the included foot pedal allows easy button presses using minimal toe effort. "

Why do I blog this? Although it is a joke, that was of the ideas I was playing with few months ago, thinking about a caving game in which you control the light using the wiimote duct-taped to one's head. Playing with the head can be troublesome though... What about yoga games?

A mysterious 3-handled tap

in

Sometimes, the environment encourages you to ask questions: The third tap

Three taps are generally uncommon, I've actually rarely ran across such an installation; that one has been spotted in Geneva, in a restaurant close to me block. So, three handles: one for the hot water, one of the cold water and the third one without any feedback when employed. Still, it's there, part of the design and definitely connected to the cold water valve. Besides, there's also a second hole.

Was it meant to be a strategy or a shortcut to get cold water more rapidly? Some complex feature to control the mixing of hot and cold water? Or was it just because the owner broke something and only had this to fill the gap between different parts.

Then, how this relates to design or ubiquitous computing? IMO, what is intriguing here is the relationship people have with technologies. What happened when people looks at this object? well, in the context of this interaction, I washed my hand and - of course - tried to open that handle to see what happened. The presence of this third handle (that I would qualify as an intru) definitely raises curiosity. Such an interface may interfere with the task at hands, not in this case because the interaction is very simple. Moreover, in such standardized setting, it clearly stands out as a focus of attention, there is a clear affordance to use it. But then nothing happens, how mysterious.

Interaction design for blind people

in ,

Two interesting examples of interaction design targeted at blind people: This virtual mapping project by greek researchers (lead by Konstantinos Moustakas) allows to "convert video into virtual, touchable maps for the blind. The three-dimensional maps use force fields to represent walls and roads so the visually impaired can better understand the layout of buildings and cities". The good thing is that this virtual touchable maps work with standard video cameras.

And Demor is a location based 3D audio shooting game that is equally enjoyable for both blind as sighted players (Via Mr. Sterling):

"Players can physically move through the auditory surroundings and hear the different sound objects in their relative positions around them. These sound objects include environmental elements like the sound of a creek or the songs of birds. As the player moves through the game these environmental sounds remain in their geographical place and can function as navigational aids. (...) To play the game, the player is equipped with a backpack containing a laptop, a GPS receiver, a head tracker, headphones and a modified joystick. By using both locative and directional sensors the software can continuously update de spatial audio feedback to accurately reflect the user's position and direction in the virtual space, creating a kind of auditory virtual reality. "

Why do I blog this? I am not into this kind of research but these technology seems to offer relevant opportunities to solve problem with a good innovative angle.

Talk at Mobile Monday Barcelona

in

Here are the annotated slides (pdf, 4.2Mb) of the talk I gave at Mobile Monday Barcelona. Thank you Fabien and Rudy for inviting me there.

The talk was entitled "Sinking one’s teeth into new metaphors for mobile gaming" and was basically a critique of the current state of mobile gaming. Instead of describing 3D on cell phones, location-based gaming or augmented reality, I encouraged mobile game designers to consider 3 other avenues: - rather than mobile gaming, what we have today is "immobile gaming" (as shown on the picture above, that I took last year in Geneva). So an opportunity is to benefit from motion/mobility instead of immobile situations. - remember that a phone is not only a small screen and little buttons: you can use other inputs (blow in the phone) and other outputs (tv-screen) - keep in mind that infrastructures (hardware/software/networks) are not always perfect and reliable so revealing the seams or taking advantage of them is a pertinent agenda (seamful design)

Down the road, all this stuff can be seen as metaphor or opportunities for mobile games of the near future.

Multitouch live performance

in

(via) French company JazzMutant, released a multitouch controller for live performance named "Lemur":

"The Lemur’s pioneering concept relies on JazzMutant’s unique patented multitouch technology. While conventional mice, touchscreens and tablets are limited to single contact points, the Lemur’s multi-touch capacity makes it possible to use all of your ten fingertips to accurately control multiple user-interface objects at once. The Lemur appeals to you straight away by its amazing responsiveness.

The continiously growing palette of configurable graphic objects enables you to design made-to-measure interfaces by using the free available JazzEditor."

Some videos here.

Why do I blog this? an interesting example of a concrete project with a well-targeted audience.

RFID gaming: La Fuga

in

La Fuga (by Négone) is a intriguing pervasive game in which players has to escape from Mazzinia, a futuristic high-security jail.

"Named La Fuga (The Breakout), the game opened this month at a former bank not far from the Real Madrid Stadium in northern Madrid. The facility can host up to 300 players at a time, each of whom tries to solve quizzes and pass through different obstacles in order to escape. Every player receives a console consisting of a specially designed PDA worn on the wrist. Between the PDA and its wrist strap is a passive RFID tag with a unique ID number used to locate and identify each player during the game. (...) "The game system activates the quizzes, the doors and the tricks in response to the detection of the tags. This allows the system to keep track of the gaming information of each player and generate each player's game individually," (...) RFID interrogators (readers) placed in doorways and in other areas of the game rooms enable the application to detect a player's location, and to use that information to drive the gamer's experience. For example, when the interrogator detects a person in a certain location, the system might display questions on the PDA screen that he or she must answer in order to progress. On the other hand, it might signal doors to open. "

(Picture courtesy RFID Journal)

Why do I blog this? yet another one on my list of pervasive game. The use of RFID tag/readers can be interesting to use in a location-based game; what is intriguing is the discovery of location on specific points such as doors and not on the whole field. That may lead to granularities that can be employed in specific gameplays.

Rebecca Allen's talk at LDM

in

Rebecca Allen's talk at LDM. Raw and messy back in the days "computers are too important to be left to engineers" so use of computer graphics, take human forms into the computer, in a natural way examples: the catherine wheel (1982) with choreographer Twyla Tharp, musique non stop with Kraftwerk (1986), mostly music clips then hired as a "3D visionary" by a game company to shift the way programmers were designing game space (2D sprites versus 3D, assembler versus C++)... then all companies wanted to do quake-like games, so she moved to UCLA, interested less by shoot'em ups but virtual worlds.

sense of loosing control when you're working as a designer with virtual worlds, artificial creatures set up aesthetic rules and let people interact, things emerge the world has a flavor of a game but there is no winning or loosing emergence: explore the role of human presence in a world of artificial life

At Media Lab Europe: notion of liminal devices: to explore the boundary between virtual and physical reality and between our inner and outer states of awareness (liminal = in between) to define subtle, intimate interface paradigm for mobile devices using biosignals and position to design simultaneous realities that allow us to see and sense more than the world in front of us in ways that enhance rather than overwhelm Project examples: - liminal identities: an interactive installation in the form of a wooden box, serves as a portal to the world of mixed reality. Through the holes in the box two people can mix their identities. - sleight of hands - body as interface: myophone (to get rid of heads-up display), project with Essilor (embed display in regular eye-glasses)

now: advisors on the "one laptop per child project"