Project Natal: amazing Xbox 360 controller-free gaming at E3

Yesterday, Microsoft showed off its super-secret Project Natal motion controller at the 2009 E3 Expo. In IT Blogwatch, Richi Jennings watches bloggers watch Peter Molyneux introduce a whole-body, controller-free way to play Xbox 360 games. And, yes: there is an IT angle.

Not to mention Twitcaps...

Austin Modine spies an invisible controller:

Microsoft has revealed a motion-sensing gadget for the Xbox 360 that uses the player's full body to control video games. Dubbed "Project Natal," the new accessory is a horizontal bar placed near the console that allows players to control their games and Xbox media without touching hardware.

Rather than rely on a wand to detect motion like the ubiquitous Nintendo Wii, the device uses a 3-D depth camera and microphones to recognize players' voices and movements for controlling what's on screen. ... Natal can recognize a player's face and automatically log them into their Xbox profile. In addition, the console's menus can be navigated by hand gestures [like] Minority Report.

Matt Peckham thinks Natal isn't just about gaming:

Microsoft utterly wowed with "Project Natal." I mean really wowed. Yeah, it's kind of a dumb name, but it may turn out to be the most impressive show item any company's crowed about in years.
...
Now think beyond games for a moment, which is where trotting out a filmmaker of Steven Spielberg's stature factors more than superficially. ... It's part of more than just Microsoft's gaming lineup. Think about walking into a room to play a game that already knows precisely where your hands and feet are. A system that already knows whether you're grumpy or melancholy, smiling or frowning, how many fingers you're holding up, or how curled or extended each one is.

Ben Kuchera sees the Xbox 360 morphing into a social hub:

Microsoft brought Steven Spielberg onto the stage to demo the technology, and his avatar moved in real time as he controled the UI. While many called this technology a gimmick before E3, everyone seems completely impressed by what's being shown here today. This isn't waving your hands around like the Wii, this is a very immersive technology tied to a very powerful console.
...
The sense of play is very Nintendo, and that's a high compliment. ... Microsoft has set the bar amazingly high for everyone else. Natal looks incredible, although it will take some real-world experience before we can completely believe the claims made. ... How much of this will feel this real—and this good—once we get our hands on it?

But Karl Buckland's not so sure:

I would withhold judgement on Natal - Peter Molyneux over hypes everything. Black and white anyone?

Robert Winters thinks he knows how it works:

When I saw it, the first thing that came to mind was the research of Tokyo University engineer Tsuyoshi Horo. He interfaced his robot by gestures. What I found particularly interesting was the way he used and processed his input from the cameras. The output was based on Human Volume Reconstruction. This means the system perceived the user as a virtual object composed out of little virtual cubes.

Technically there was an array of cameras to detect body movement. This resulted in a real-time 3D volumetric model of himself that consisted out of these small virtual cubes. By analyzing these cubes as data, gestures could be extracted. From there on it's all math and probability calculations mapped on the model of a human body.

Seminal Wii hacker Johnny Chung Lee is also involved:

I don't deserve credit for anything that you saw at E3. A large team of very smart, very hard working people were involved in building the demos you saw on stage. The part I am working on has much more to do with making sure this can transition from the E3 stage to your living room - for which there is an even larger team of very smart, very hard working people involved.
...
Speaking as someone who has been working in interface and sensing technology for nearly 10 years, this is an astonishing combination of hardware and software. The few times I’ve been able to show researchers the underlying components, their jaws drop with amazement... and with good reason.
...
We would all love to one day have our own personal holodeck. This is a pretty measurable step in that direction.

So what's your take? Get involved and leave a comment.

And finally...

• Twitcaps.com -- weird and wonderful live stream of tweeted images
  [hat tip: Jonathan Griggs; some images may be NSFW]

Previously in IT Blogwatch:

• New iPhone 3 / 3G / 2009 leaks: autofocus, compass, MMS...
• Google Wave: who, what, where, why, when?
• Steve Jobs is healthy, says famous dancer Woz
• Android to dethrone iPhone?
• Skyfire: a better handheld browser?

Don't miss out on IT Blogwatch:

• Subscribe to the Computerworld Blogs newsletter
• Catch up with posts from the previous few days

Richi Jennings is an independent analyst/consultant, specializing in blogging, email, and spam. A 24 year, cross-functional IT veteran, he is also an analyst at Ferris Research. You can follow him on Twitter or FriendFeed, pretend to be Richi's friend on Facebook, or just use boring old email: contact Richi.