News: The Revolution of the Hacked Kinect, Part 1: Teaching Robots & the Blind to See

The Revolution of the Hacked Kinect, Part 1: Teaching Robots & the Blind to See

In 2007, Nintendo introduced the world to motion control video games with the Wii. Microsoft and Sony built on Nintendo's phenomenal success and released their own motion control products for the XBox 360 and Playstation 3 late in 2010: the Kinect and the Move. The Move is basically an improved Wiimote that looks like a sci-fi Harry Potter wand, but the Kinect just might be the most important video game peripheral of all time.

In case you're not familiar with the Guinness World Record-holder for fastest selling consumer electronic device ever (8 million in 60 days), the Microsoft Kinect is a slim black box that plugs into the XBox 360, and sits below your television. From there, it uses two cameras, an infrared depth sensor, and a four-capsule microphone to detect, recognize, and follow the movements of people and objects in its field of view. This allows users to control specially-designed XBox 360 games with their entire bodies, without having to hold a controller.

Kinect hacking began in earnest after New York-based electronics shop Adafruit Industries offered a $3000 dollar bounty to the first programmer to develop open-source drivers for the Kinect. A 20-year old Spaniard collected the prize (which was fronted by former Microsoft employee and Kinect team member Johnny Lee) and ever since, enterprising DIY enthusiasts from all walks of life have worked on manipulating the Kinect to suit their myriad purposes. The results have been more wide-ranging and amazing than anyone at Microsoft could have possibly envisioned. Kudos to Microsoft for allowing these visionaries to do their work without fear of reprisal. Sony has long been harsh with hackers, but it seems Kinect may be forcing them to change their tune.

This week, we feature four segments on the revolution of the hacked Kinect, presenting video documentation of the best of thousands of hacks, broken down by field. The Kinect will assuredly change the way millions of people will do their jobs in years to come, so pay attention. Its innovation is extensive, and it just may be the most accidentally powerful piece of digital hardware ever released to the public. Today, innovations in the field of science.


Robotic surgery is becoming more and more widespread—leading inadvertently to surgical practice coming to resemble video game playing. The Kinect has great potential to aide in remotely operated robotic surgery. It can generate a live 3D model of the patient, which allows the doctor to perceive depth more accurately than solely with a camera. Some experiments have even enabled the Kinect to impart haptic feedback to the doctor, the lack of which has been a serious problem for robotic surgery. It has also shown promise as a portable means of guiding the blind via that same haptic feedback. Below, four astounding innovations in the field of medicine.

(1) Russians recently demonstrated the Kinect guiding a blindfolded man. (2) The Kinect is used with Da Vinci drawing software to perform surgical incisions at the Laboratory for Computational Sensing and Robotics (LCSR), at the Johns Hopkins University, Baltimore, MD. (3) Researchers apply Kinect to PACS imaging at the University of Forensic Medicine, Bern, Switzerland. (4) This University of Washington team uses the Kinect to add haptic feedback to remote surgery. (5) Master’s students Michael Zöllner and Stephan Huber of University of Konstanz in Germany created the NAVI, a Kinect-based portable unit that used haptic feedback to guide the blind.


The field of robotics is already on the cutting edge of science, so it's only natural that the Kinect has been so widely adopted by robotics engineers so quickly. It allows them to implement hands-free control of robots from remote locations. Furthermore, you can mount the Kinect on the robot itself, and allow it to "see" the world using a digital interface. Below, four videos demonstrating the use of the Kinect in robotics.

(1) Terrifying use of a NAO robot. Also supplemented by Wiimotes. (2) Controlling a PS2 industrial robot with Garratt Gallagher at MIT. (3) iRobot, makers of the famous Roomba robot vacuums, show a robot using Kinect sensors to navigate at CES in Las Vegas. (4) It's not technically a robot, but it IS controlling a flying machine from afar using the human body. By Tom Zickel of Zickel Engineering in Israel.

Next: The Revolution of the Hacked Kinect, Part 2: Even MBAs Gotta Have One

Just updated your iPhone? You'll find new features for Podcasts, News, Books, and TV, as well as important security improvements and fresh wallpapers. Find out what's new and changed on your iPhone with the iOS 17.5 update.

Be the First to Comment

Share Your Thoughts

  • Hot
  • Latest