Augmented Reality for Neurosurgeons

Presented for the first time in the middle of last year, Kinect Fusion is a software component that allows you to use the Kinect sensor to generate a full 3D environment. To take the idea of ​​3D elements in front of a regular Kinect sensor to another level, there were introduced concepts of augmentative reality (AR).

Starting from the sensor’s ability to interpret a three-dimensional environment in front of the camera, in a software application was loaded 3D model of a skull with a detailed brain. Because the virtual head had some specs that fitted a physical model out on the table, when the camera came in front of the model an association between the two models was fixed.

The whole system was demonstrated with a Kinect sensor taped to a tablet. Passing over a wide variety of an AR applications, the perfect scenario will be in an operating room. The fact that a 3D model of a head or any internal organ can be associated with a physical part of a patient means that the doctor can see in real time in 3D, what is inside the patient like peeking through a window.

Kinect Fusion, the most important part of the whole equation, will be included in the next version of Kinect for Windows SDK.