Kinect: taking control of computing

Stuart Andrews discovers how a pack of innovative hackers has transformed Microsoft's Kinect into much more than a plaything

With ten million sold since November, it’s the fastest-selling consumer electronics device in history. It’s also the most hyped console accessory of the past five years, and the only technology that can – in Microsoft’s words – make you the games controller.

Combining cameras and 3D depth sensors with voice and facial recognition, Kinect can capture motion from up to 48 joints in the human body, tell you apart from other members of your household, and allow you to control video games using speech, gestures and full-body motion. It’s made the Wii look like yesterday’s toy, and brought the Xbox 360 to a whole new market.

Kinect is fast becoming much more than a plaything

However, Kinect is fast becoming much more than a plaything. Working with open-source tools and unofficial drivers, a community of enthusiasts has hooked up Kinect to the PC.

These can-do coders have been creating home-grown gesture controls for Windows and developing some of the weirdest demo projects you’ve ever seen.

From virtual surgery, to gesture-controlled Windows applications and human body artworks being smashed into a million pieces, Kinect has virtually limitless potential.

That hasn’t gone unnoticed at Microsoft. Kinect’s creator has never denied that it has wider ambitions, and the release of a new official SDK should make it easier to ensure Kinect performs.

Suddenly, Kinect feels like the start of something big. In fact, senior figures at Microsoft see it as the beginning of a new era in computing.

Nurturing Natal

It wasn’t always this way. Kinect originally stems from the somewhat desperate situation that faced Microsoft’s entertainment division in mid-2007.

While the Xbox 360 was a hit with hard-core gamers, it wasn’t breaking the mainstream audience in the same way as Nintendo’s faster-selling, motion-controlled Wii.

Microsoft needed a new twist, and put a 31-year-old technology guru, Alex Kipman, in charge of a project that could push the console in a different direction. In June 2009, Kipman’s group was ready to demonstrate its wonder product, codenamed Project Natal.

Natal wasn’t all Microsoft’s own work. The basis of the hardware came from a Tel Aviv startup called PrimeSense, which demonstrated a revolutionary 3D full-body motion sensor in 2006.

Find out more

Top five Kinect hacks

PrimeSense’s unit contained a near-infrared depth sensor and a matching projector, along with a specialist system-on-a-chip (SoC) processor and an optional RGB video camera.

Using a system PrimeSense calls Light Coding, the projector floods the scene with near-infrared light, beyond the visible range of the human eye.

A regular CMOS sensor – the same kind found in many webcams – captures the light reflecting back and passes it on to the SoC. This uses complex algorithms to convert the light data into a full 3D-depth map, and this can then be combined with data from the RGB camera to give each 3D pixel a colour value.

Capture all this information, identify which parts are human, and you have all you need to operate full-body, motion-controlled 3D games.

Games were always the driving force behind PrimeSense’s technology.

However, it was Microsoft that recognised the sensor’s potential and made it the core technology in Natal.

Microsoft added a couple of extras to the reference design, including a more sophisticated, multi-array microphone and a motorised stand that would automatically adjust the position of the camera to keep players in the frame.

Read more

Analysis

Pages