Early Print Material can be seen here

Gone Gitmo – play it here

Gone Gitmo was chosen as part of an exhibition at the Moscow Museum of Modern Art and we rebuilt it in Unity. Play it here:

GONE GITMO IN UNITY

Make sure you get unity plugin to play it!

GG_titles3

Embodying a robot to do journalism

I had to share “MY” body with someone else.

Its not quite what it sounds like, so let me explain.

In a first-of-its-kind experiment held jointly between the Event Lab at ICREA-University of Barcelona and USC’s Institute for Creative Technologies, I donned an XSens motion capture suit and virtual reality goggles (an NVIS head mounted display), in order to “drive’ a robot more than 6000 miles away in Barcelona, Spain. The idea was to NOT simply experience what embodiment of the robot felt like, but to also complete a regular task in my work as a journalist: interviewing people for a story. In this case, I had two sets of interviews I wanted to do. The first was with Professor Javier Martinez-Picado, whose team reported an extraordinary breakthrough in identifying how HIV infection spreads by unlocking immune cells that then disperse the disease throughout the body. The second set of interviews was a discussion about the Catalonia independence movement, with three individuals from pro, anti and neutral positions prepared to discuss their rationale.

From start to finish, I was in the robot more than 3 hours and at some point during the experience, I began to adopt the robot body. Yes, there were constraints in movement, overcorrection from head turns or hand movement and a viewpoint that was much taller than my normal 5’ 3”. But the connection became so extensive that it wasn’t until more than 30 minutes after I took off the equipment that I realized I was only THEN re-entering my “normal” body.

And here’s where the body sharing business comes in: Bizarrely, when I looked at photographs of Professor Maria Sanchez-Vives operating the robot on one of the initial experiments, I unexpectedly shuddered. I felt like she was inside my body!

In fact, for days afterward, just thinking about the robot would involuntary cause my body to adopt the most comfortable position for matching the robot to my natural body stance. My arms would bend at the elbow, with my hands outstretched ready to wave, shake hands or gesture. My head would look upright and my back would stiffen in order to more readily walk forward, back or to swivel from left to right.

I can only describe the experience as trying to do a sit up for the first time – you have a concept of how to do it but no muscles to actually perform the task. My entire being had to work in a unified effort to occupy and embody a “second” self so I could conduct the type of interviews I have done over the past twenty years. Later, in another strange reaction, when I starting watching an interview with my robot-self in Barcelona, I found myself so upset about the viewpoint of the TV crew camera –it was at the wrong angle! – that I had to get up from my desk and walk away. I then had to force myself to sit back down to watch the whole video. (it is in both English and Spanish starting at 38 seconds): http://www.youtube.com/watch?v=FFaInCXi9Go&feature=share&list=UUU3lATTLOBgJeQbJom707eQ[/url]

In another post, I will detail the actual reporting, because the interviews were fascinating. In meantime, I want to thank everyone for their hard work in making this happen, especially Thai Phan, Xavi Navarro Muncunill, Raphael Carbonell and all of the incredibly helpful folks at the University of Southern California ICT Mxr Lab and the Universitat de Barcelona EVENT Lab for Neuroscience and Technology.

Hunger in Los Angeles – An Immersive Journalism Premiere

This machinima video shows Hunger in Los Angeles from the viewpoint of a witness/participant who is experiencing the recreation of a real event that unfolded on a hot August day. The situation transpired at the First Unitarian Church on 6th Street in Los Angeles, when the woman running a food bank line is overwhelmed. “There are too many people,” she pants. And then, much louder, shouting in frustration, “There are too many people!” Only minutes later a man falls to the ground in a diabetic coma. The line is so long that his blood sugar has dropped dangerously low while waiting for food.

Invisible to the average American, hunger is an insidious problem facing this country. More than one in five adults suffer with food issues and the rates are worse for children. The events of that day are mirrored circumstances around the country. Food banks simply can’t keep up.

While it is a story well worth reporting, coverage about the issue in the newspaper, on TV or on the web barely seem to resonate. In response, HUNGER IN LOS ANGELES, puts the public on the scene at the First Unitarian Church and makes them an eyewitness to the unfolding drama by providing a powerful embodied experience. By coupling the latest virtual reality goggles with compelling audio, which tricks the mind into feeling like one is actually there, HUNGER provides unprecedented access to the sights, sounds, feelings and emotions that accompany such a terrifying event. In fact, when this revolutionary brand of journalism was showcased at the 2012 Sundance Film Festival, the strong feelings of being present made audience members try to touch non-existent characters and many cried at the conclusion of the piece.

The 6.5 experience has been edited down in this video and the impact is much greater wearing head mounted display virtual reality googles, as did the audience at Sundance New Frontier, http://www.sundance.org/festival/film-events/hunger-in-los-angeles/ With so many folks coming out of the experience crying, or otherwise deeply disturbed, I began to realize that part of good journalism, of being a civic partner to my audience, is to offer them ways to act. I don’t consider this activism but rather an appropriate interaction. One great organization in Los Angeles is called Mend and another is the Los Angeles Regional Foodbank.

Also, there are two websites that have comprehensive listings of charities and rate them for you, so you can give according to what moves you personally.
Guidestar
Charity Navigator

“Hunger in Los Angeles” will premiere at Sundance Film Festival 2012!

http://www.sundance.org/festival/festival-program/hunger-in-los-angeles/Here’s the synopsis:

Former Newsweek correspondent and documentary filmmaker Nonny de la Peña is developing a groundbreaking brand of journalism that offers a fully immersive experience of the news report. Interested in calling attention to the growing issue of hunger in the United States, HUNGER IN LOS ANGELES recreates an eyewitness account of a crisis incident on a food bank line at the First Unitarian Church in Los Angeles. De la Peña combined the game development tool Unity 3D, a head mounted display with motion tracking, and live audio she collected from the incident, to construct a simulated world in which audiences can suit up, walk around, and interact with other characters on the scene. The project has been developed at the MxR Lab, a joint lab between USC’s Institute for Creative Technologies and The School of Cinematic Arts, with additional support from the Annenberg School of Communications & Journalism.

Here’s the link: http://www.sundance.org/festival/film-events/hunger-in-los-angeles/

add a thought

Using Motion Capture for a piece on Hunger and Overstrained Foodbanks

As part of a larger investigation on hunger in California out of USC’s Journalism School, I began collecting audio from food banks to see if we could create an immersive piece about the problem. Its been slow going without any budget but finally the project is taking shape. I’ll be able to embed a Unity version soon. In the meantime, we used motion capture to animate the characters who were on scene at the food bank. The audio is from the original scene and the animation will be soon married with the digital representation of a woman who was overwhelmed by the crowds at the food bank that day. I have been working with amazing artists Bradley Newman and John Brennan who have spent many late hours helping to pull this off.

Three Generations Newsgame

Three Generations, which I (Nonny de la Peña) designed alongside Simon Wiscombe was selected as a demo at Games For Change. Here’s some details

Three Generations
Developers / Designers Nonny de la Peña and Simon Wiscombe

Synopis Three Generations is a board game designed to introduce and ignite both an emotional reaction and an intellectual discourse about the Californian Eugenics movement that transpired in the early 1900s. The core design focuses on embracing its interactive nature in order to convey a primarily emotional experience as well as an informative one. Four players (known as Families within the game) are separated so that two Families become a team and play against the other two Families with the objective of “expanding the city of Sonoma.” Using resource-management, teams build up roads and buildings from either side of the board to gain the most ground and property. Each round of play is a “generation”, and players must place one more family member (Child) on the board (each building and road can house one child each). The money to buy these buildings and roads is gained by answering questions from the Stanford-Binet IQ test designed by Lewis Terman in the 1920’s. Answer correctly, and players are awarded with money to spend (the amount is modified positively by children in buildings and negatively by children on roads), otherwise the player receives one “unfit” token. Enough unfit tokens, and the player is declared feeble-minded, whereby they produce two children per turn and receive no monetary bonus when they answer questions. The other player on the team may make the decision to send their unfit teammate to be sterilized. Once sterilized, the player’s presence on the board slowly vanishes until they are out of the game completely. Play-testing has shown that the players, regardless of previous knowledge about the subject matter, come to a realization about the implications of their play and how it extends beyond the confines of the game’s boundaries. This elicits the desired goal: a discussion about not only the historical pretext, but their own emotional reactions to their personal involvement in a system that supports eugenics.

Intended Audience, Context, and Objectives Three Generations is a board game dealing with the inherent social biases of the Californian Eugenics movement. While the term “eugenics” originated with Francis Galton (cousin of Charles Darwin) in the late 1800s, it gained a very large following in the United States in the early 1900s. Although the eugenics philosophy employed later by National Socialists in Germany was based largely on the idea of superior genetics among races, the mindset that caught ground in California was largely due to money. Drunkards, criminals, “loose women,” the mentally handicapped: all were considered “feeble minded” and all depended on the states for monetary support. The logic was simple: prevent them from breeding and you prevent the passing of these “weak genes” from being passed along. Within three generations, eugenicists claimed, you could all but eliminate these perils from society. In 1909, the first law was passed in California allowing for forced sterilizations, and Sonoma State Hospital was quickly established as the lead authority when it came to asexualizations. There were several bills passed by the California State Legislature to expand the ability to sterilize through 1917, and a Supreme Court case in 1927 validated the states’ right to do so. It wasn’t until the atrocities of the Nazis were well-known that “asexualization” fell out of favor with the general public and was effectively forgotten.
It’s easy to see how human rights would be violated in this system, and there are a number of accounts of social and cultural biases as minorities were disproportionately sterilized. An example: feeble-mindedness in intelligence was determined by the Stanford-Binet IQ Tests developed by Lewis Terman, which have been well-documented as being largely discriminatory against those who had yet to become acculturated or lacked access to quality education at the time (e.g. poor and minorities).

From the beginning, there were two main goals we attempted to achieve in the game: 1) To inform the players of the events that occurred and led to the eugenics movement; and 2) To create both an emotional reaction and an intellectual discourse about eugenics. It was this second goal that draws a distinction between traditional news/journalism games. In order to facilitate their emergence, without it seeming forced, we had to recreate a world that exists within the magic circle of the game and let the players live within it. The information and historical facts would be used not only in this manner, but also to serve as a foundation for the gameplay and through-narrative. The goal was to completely shroud the game in the logic of the eugenicists and their movement. It should be noted that this game was, from the outset, intended to be a multiplayer board game experience; crafting the game depended heavily on this face-to-face experience.

MORE INFO

Immersive Journalism: Immersive Virtual Reality for the First-Person Experience of News

Here is our published paper on Immersive Journalism which outlines the fundamentals of immersive journalism. Immersive Journalism: Immersive Virtual Reality for the First-Person Experience of News

Teaching with Technology

There have been many advances in teaching with technology. Here is the latest project in which I used immersive experiences to try to consider how we could continue to conduct classes if campus were closed due to an earthquke.

Kinect Flappers – the palette for Immersive Journalism grows

I will be building a piece shortly using Kinect – but wanted to give you an idea of how physical the platform is: In this video, you must fly to pop floating bubbles and here’s what the new users look like.