Motion Gaming, Part 2: Project Natal

This generation of consoles has been an interesting one for sure, but it’s not over yet. In an effort to increase the console lifecycle, Sony and Microsoft have announced motion control devices to be launched this fall. Both companies are approaching motion gaming vastly differently, but both are using these new technologies as a way to re-launch their respective consoles. This is in no small way due to the huge success Nintendo has had with their answer to motion gaming. But how do these three devices compare? And which is more likely to control the free time of gamers everywhere?

This is the second part of the Motion Gaming series, you’ll find the first here. Part 1 focuses on Sony’s answer to Motion Gaming, the Playstation Move. Part 2, which you’ll find below; will take a closer look at Microsoft’s Project Natal. Part 3 will look to Nintendo and their contributions to the motion gaming genre, both past, present and future. And part 4 will look at all three in an effort to compare and predict the outcome of the motion wars, as well as try and imagine what motion gaming on a whole will look like next generation and beyond.

To the far left; the infrared projector. In the middle to the left, the monochrome CMOS sensor. In the middle to the right, the RGB sensor. Where is the microphone you ask? We have no idea.

A day ahead of Sony’s Playstation Move announcement, Microsoft shocked the world with their highly innovative Project Natal; a camera powered motion sensing device for the Xbox 360. Microsoft spent the better part of the last 20 minutes talking about this new and “revolutionizing” force in gaming. Despite the somewhat broken “bottom of shoe” presentation by Kudo Tsunodo, Natal was met with an open minded audience and high praise.

Unlike the Move or Wiimote, Project Natal consists of only one hardware device; the sensory bar. Natal consist of two active-pixel sensors or cameras for us regular people, and a crucial infrared light projector. And although details haven’t emerged on it, Natal is also supposed to be equipped with audio sensing capabilities. So far we don’t really know much about Project Natal, yet that didn’t stop Time Magazine giving it the fifth “Best Inventions of the Year” award in 2009.

Full Body Motion Sensing

Ricochet being shown on the E3 2009 stage for the first time.

There is a popular misconception going around that Natal uses its two cameras to detect depth via stereoscopic technology, this is not the case. If it were, they wouldn’t need the infrared light source. Natal’s two cameras actually have totally different jobs; one detects only monochrome light and is used to catch the infrared light bouncing back from the different objects in the room. So the infrared projector and the monochrome sensor provide the Xbox 360 with a crude picture of every object and person in the room, and how far away they are. For full body motion capture, this requires that the entire front of your body is visible to the camera.

Many have expressed concern because this fact makes coffee tables and cramped living rooms a problem. Luckily, as of right now it’s looking like only a small amount of games will require full body motion capture.

Facial Recognition

Milo recognizing the determination in the players voice and facial expression and reacting accordingly.

While the infrared light and the monochrome sensor are all fine and well for detecting the movement of big things, a finer touch is required to handle facial recognition. Natal has another camera. This one sees in the full red, green and blue color spectrum and represents the part of the Natal experience you’ll actually be aware of. The monochrome camera can only see, well, in monochrome. It cannot give life or meaning to the pictures. So what natal does is that it overlays the full colored image provided by the RBG sensor on top of the depth image provided by the monochrome sensor. This allows Natal to detect and understand a person’s face, creating a whole new level of interaction with the console. And allowing for other functions like video chat and augmented reality.

Voice Recognition

While Microsoft haven’t yet given any specifics about the microphone(s) the Natal sensor bar is equipped with, recently we have been informed of what the Natal technology is based on. Last month we got confirmation that the Israeli company PrimeSense are the masterminds behind this technology. Based on their description of the device, it seems likely that Natal will use a microphone array similar to that in the PSEye only with two microphones instead of three. Whether this matters or not depends on the technology behinds it, but even so, it could be debatable.

The PrimeSense technology diagram.

At E3 2009 Peter Molyneux of LionHead Studios showed the world the quality of Natal’s voice recognition with Milo, the user was able to have a relatively fast passed conversation with a character without any visible bugs or latency. Though the few people who got to try this themselves under less controlled conditions came back saying it “need’s a lot of work.”

Software

Project Natal lead Kudo Tsunoda showing of Burnout Paradise using Project Natal on Jimmy Fallon.

Supporting the Natal sensor bar is an elaborate and complicated algorithm based software solution on the Xbox 360. Initially it was reported that Natal would come with its own processing unit to support the massive amount of calculation required to make sense of all the data it received from the sensors. This was however scraped by Microsoft to bring down costs. Instead Microsoft decided to allow “10-15%” of the Xbox processor to handle the calculations, meaning that system resources will have to be allocated for this purpose. In effect this rules out the possibility of motion control becoming backwards compatible with older games.

But while older games can’t just be patched, we have seen demos of older games running Natal, games like Burnout Paradise. Games can in fact be redone to utilize Natal, but the change is so extensive that it wouldn’t be backwards compatible through patching. Instead the developer would have to re-release the game, effectively asking previous owners to buy it again. So while it is possible that you will someday get to play Halo 3 with Project Natal, it isn’t very likely.

Games

Not much can be said about the games announced so far. Primarily because there are so few of them and from what we’ve seen each is little more than a tech demo. The most substantial project would undoubtedly have to be Fable III, which will utilize Natal for some of it’s gameplay, though this does not by any means represent the bulk of it, nor is it required.

One can imagine an updated version where your hand is graphically represneted on the screen, enableing you to acutally reach into the water.

Another project by LionHead Studios is Milo, which as I mentioned above; lets you interact with a character in real time. This was very much a the tech demo stage last time it was shown, since it wasn’t clear how it might actually translate into a purchased product. Many have speculated that it could be used in the genera of the interactive movie, a genera that Quantum Dream popularized with Heavy Rain.
Then you’ve got Ricochet, the Breakout-like game, where the player is tasked with destroying blocks down range by punching or kicking back a bouncy red ball; all through full body motion capture. And Paint Party a painting game where you toss paint at a canvas, or pose for the camera to create backdrops and shapes. Both of these games are likely to be downloadable or included in some sort of a compilation, akin to Wii Sports or Sports Champion for the Move.

Despite the lack of information on games, many publishers have stated that several games for “all motion gaming platforms” are currently being developed. We just haven’t heard of them yet. One game was recently leaked by avid gamer and screen writer James Gunn; supposedly called River Rush. It’s a two player game based on controlling a river raft as is rushes downstream.

Conclusion

Much is still unknown about Project Natal. For example, a rumor just surfaced saying that the sensor bar would featured a motorized tilt mechanism. If this turns out to be true, then it would mean that the Natal sensor bar could readjust its angle to keep the subject in view. This would allow for more freedom of movement, and could result in some amusing minigames itself. But what stands out about Project Natal far more then its competitors is potential. Problems have to be overcome for it to work in traditional games, simple things like walking leave us with a headache trying to imagine how Natal will offer a solution. So while Natal has great potential, it is so far unrealized. In part 4 I’ll be looking more closely on this problem and make some predications on what attempts we can expect to see in solving it.