Creating SL Animations using the Kinect

I have been experimenting of late with the Xbox Kinect as a cheap Mocap source to generate BioVision Hierarchy (BVH) files for upload into Second Life, and the results are pretty encouraging.

What you Need

Hardware
A standalone Kinect, with power supply (this comes as standard when purchased as a standalone), if you buy it as part of an Xbox bundle you will need to buy an additional cable sold separately to connect it to the PC. I bought my Kinect from a local gaming store for Euro139.

Kinect, showing its USB connector and power adaptor

Software
Brekel Kinect 3D Scanner
OpenNI
SensorKinect Drivers
NITE User Tracking Module
Bvhacker

Procedure
For my procedure I used a laptop running 32 bit Windows XP , although Brekel Kinect is confirmed to work also with win7 x86 & x64 as well as XP x64 & x86, but no Mac/Linux version of Brekel Kinect planned.

1. Once you have unpacked your Kinect you need to place it on a table or a tripod (which is what I did), at around 0.6m - 1.8m from the ground. You can plug it into a power socket at this stage, but do not connect the USB connector to your PC/laptop just yet.

Kinect mounted on an inexpensive camera tripod

2. Download and install the following software in this order:

Brekel Kinect 3D Scanner v0.36

OpenNI Alpha Build for Windows v1.0.0.23

PrimeSensor v5.0.0 (Modules for OpenNI)
Click the Downloads button and choose the zip file. After downloading extract the avin2-SensorKinect folder, and within this go to the bin subfolder, and run SensorKinect-Win32-5.0.0.exe

PrimeSensor v5.0.0 with correct options selected

PrimeSense NITE Beta Build for Windows v1.3.0.17
Use this key during installation: 0KOIk2JeIBYClPWVnMoRKn5cdY4=
(This is NOT a pirate key)

NITE Installation Wizard

Copy and paste the License Key from above

 Now download and install Bvhacker 


3. Once the software is installed you can  connect the Kinect USB lead. The Found New Hardware dialog should appear.

No need to connect to Windows Update

When it asks if it can connect to Windows Update, choose No, not his time. On the next dialog screen choose to Install the software automatically.

The hardware that will be detected are the Kinect Motor, Xbox NUI Audio and Kinect Camera. The drivers for the Xbox NUI Audio will not be found and will fail, but ignore this, they are not needed.

Device Manager showing the new device, PrimeSensor

When the drivers have been installed you can check that the hardware and its drivers are ready to be used by Windows by going to Device Manager (Start, Settings, Control Panel, System, Hardware, Device Manager) and you should now see a new entry, PrimeSensor, and under that will be the Kinect Camera and Motor.

4.You can now launch Brekel Kinect.

Brekel Kinect GUI

Brekel may launch with a Dark skin. You can change this by unchecking it under the Window menu item. I found I had to do this as some of the check boxes did not show too well with this almost black skin.

The only setting you now have to change is to enable NITE User Tracking at the centre bottom of the window.

The only setting to change, make sure NITE User Tracking is enabled

Once NITE User Tracking is enabled you should get a message in red on the central image that says: User1 [Looking for Psi pose]. Ensure that your full body is visible in the video windows, then adopt the Psi pose (like you are a trident, or surrendering). Once the pose is recognised it will take just a few seconds for tracking to commence (be patient, and KEEP STILL!), and the message should now go green.

Psi Pose

 As soon as you are being tracked (indicated by white dots covering your body) you can start recording animations. Start with simple ones like the ubiquitous salute, or hand clap. To start recording to BVH click the Start Capture BVH button (near the top right). If you are on your own you may find that tracking is lost if you go out of the field of view of the Kinect, in this case either try to have the PC/laptop within reach, or have a friend start the capture. There is a four second delay (useful if you are on your own) before recording starts. When done, click the Stop Capture to BVH button.

For full details on the capture procedure, and what can go wrong, go to the main menu, Help, How To - Capture motion to a BVH file. Additional resources may also be found on the main Brekel website.

Brekel can save BVH in two main formats, HumanIK (used by MotionBuilder and others) and Biped (used by 3DSMax). Neither of these can be imported directly into Second Life, as the naming convention for Root, joints and bones are slightly different, this is where Bvhacker comes in.

To prepare the BVH file for Second Life, start Bvhacker then File, Open, and load the BVH file. Once loaded you can play it to see what it looks like using the Start button at the bottom of the window. If all is well go back to the menu, Hack, and choose Attempt SL joint naming. This should go without a hitch, and you can now save it out ready for importing into Second life.

Have fun,

Rock