Kinect Basics: Joint Tracking

         This post is Kinect basics. I don't mean any of the stuff you could read else where, I mean the stuff only you read here. So assuming you know stuff about Kinect, this will make sense to you. But to those of you who are pretending to know something about Kinect when you know nothing then I banish you from this blog.

          Moving on, we are going to make an app (in WPF, if you use Winforms I banish you again, and don't even get me started on Silverlight) that takes a joint (your hand, for example) and maps an ellipse to it. This is very simple and I use it a lot. With the XAML just make an 2 images, 4 ellipses, and a canvas, like so:

 <Window x:Class="SkeletalTracking.MainWindow"  
   xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"  
   xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"  
   Title="MainWindow" Height="600" Width="800" Loaded="Window_Loaded"  
   Closing="Window_Closing" WindowState="Maximized">   <Canvas Name="MainCanvas">  
   <Image Canvas.Left="0" Canvas.Top="0" Width="640" Height="480" Name="imageviewer" />  
   <Ellipse Canvas.Left="0" Canvas.Top="0" Height="50" Name="leftEllipse" Width="50" Fill="#FF4D298D" Opacity="1" Stroke="White" />  
   <Ellipse Canvas.Left="100" Canvas.Top="0" Fill="#FF2CACE3" Height="50" Name="rightEllipse" Width="50" Opacity="1" Stroke="White" />  
   <Image Canvas.Left="66" Canvas.Top="90" Height="87" Name="headImage" Stretch="Fill" Width="84" Source="whateverimagefileyouwant.jpg" />  
   <Ellipse Canvas.Left="283" Canvas.Top="233" Height="23" Name="leftknee" Stroke="Black" Width="29" />  
   <Ellipse Canvas.Left="232" Canvas.Top="233" Height="23" Name="rightknee" Stroke="Black" Width="30" />  
 </Canvas>  
 </Window>  

           Now keep in mind that this is my first post with code so I don't have the whole Blogger setup figured out. I will find some way to put this stuff in code but it works for now. Now I am assuming if you know something about the Kinect SDK you will know how to display a color image, do that now with "imageviewer" Image. Now for the mapping, you will need these few methods:

 void GetCameraPoint(Skeleton first, AllFramesReadyEventArgs e)  
     {  
       using (DepthImageFrame depth = e.OpenDepthImageFrame())  
       {  
         if (depth == null ||  
           kinectSensorChooser1.Kinect == null)  
         {  
           return;  
         }  
         //Map a joint location to a point on the depth map  
         //head  
         DepthImagePoint headDepthPoint =  
           depth.MapFromSkeletonPoint(first.Joints[JointType.Head].Position);  
         //left hand  
         DepthImagePoint leftDepthPoint =  
           depth.MapFromSkeletonPoint(first.Joints[JointType.HandLeft].Position);  
         //right hand  
         DepthImagePoint rightDepthPoint =  
           depth.MapFromSkeletonPoint(first.Joints[JointType.HandRight].Position);  
         //Map a depth point to a point on the color image  
         //head  
         ColorImagePoint headColorPoint =  
           depth.MapToColorImagePoint(headDepthPoint.X, headDepthPoint.Y,  
           ColorImageFormat.RgbResolution640x480Fps30);  
         //left hand  
         ColorImagePoint leftColorPoint =  
           depth.MapToColorImagePoint(leftDepthPoint.X, leftDepthPoint.Y,  
           ColorImageFormat.RgbResolution640x480Fps30);  
         //right hand  
         ColorImagePoint rightColorPoint =  
           depth.MapToColorImagePoint(rightDepthPoint.X, rightDepthPoint.Y,  
           ColorImageFormat.RgbResolution640x480Fps30);  
         //Set location  
         CameraPosition(headImage, headColorPoint);  
         CameraPosition(leftEllipse, leftColorPoint);  
         CameraPosition(rightEllipse, rightColorPoint);  
       }  
     }  

 Skeleton GetFirstSkeleton(AllFramesReadyEventArgs e)  
     {  
       using (SkeletonFrame skeletonFrameData = e.OpenSkeletonFrame())  
       {  
         if (skeletonFrameData == null)  
         {  
           return null;  
         }  
         skeletonFrameData.CopySkeletonDataTo(allSkeletons);  
         //get the first tracked skeleton  
         Skeleton first = (from s in allSkeletons  
                  where s.TrackingState == SkeletonTrackingState.Tracked  
                  select s).FirstOrDefault();  
         return first;  
       }  
     }  

           Now all you need is to add in the AllFramesReadyEventArgs to tie it all together.

 void sensor_AllFramesReady(object sender, AllFramesReadyEventArgs e)  
     {  
       if (closing)  
       {  
         return;  
       }  
       //Get a skeleton  
       Skeleton first = GetFirstSkeleton(e);  
       if (first == null)  
       {  
         return;  
       }  
       //set scaled position  
       ScalePosition(headImage, first.Joints[JointType.Head]);  
       ScalePosition(leftEllipse, first.Joints[JointType.HandLeft]);  
       ScalePosition(rightEllipse, first.Joints[JointType.HandRight]);  
       ScalePosition(leftknee, first.Joints[JointType.KneeLeft]);  
       ScalePosition(rightknee, first.Joints[JointType.KneeRight]);  
       GetCameraPoint(first, e);  
     }  

           And that is all there is to it! Hope this helps. A large portion of this code was taken from http://channel9.msdn.com/Series/KinectQuickstart/Skeletal-Tracking-Fundamentals