If you are considering streaming video into Second Life/Opensim (SLOS), either from a dynamic video source such as a webcam, or from pre-recorded video files (and for the purpose of this article I will concentrate on pre-recorded video files), then you have several choices. First the Prerequisites:

Prerequisites

• You must either be the owner or estate manager, or a member of a group that owns the property you plan to stream into (unless you are just streaming to others who are)
• Your video source must be in the QuickTime format, or in a format that QuickTime supports

Note: The two most common Quicktime formats are *.mov and *.mp4, but there are other supported formats and these are listed on the QuickTime wiki page. If the video you are attempting to stream does not play in QuickTime Player then it will not play in Opensim, so it is a good idea to download and install the QuickTime Player, and check that your video source plays OK. Details on the specific format used for SLOS are available here.

Choices
The  choices available to would-be streamers are as follows:

• Set up a video streaming server on a dedicated server or Virtual Private Server (VPS)
• Lease a video streaming service
• Do it the easy and zero-cost way

Setting up a video streaming server is not for the faint-hearted, and the streaming software can be quite expensive. There are open-source alternatives available, but can prove very daunting for the non-tech savvy, as they often need compiling from source and usually are command-line driven.

Leasing a video streaming service, such as Streamhoster or StreamUK, can likewise prove costly, depending on the number of users who simultaneously connect to the stream. There are some low-cost or free video streaming sites, such as Ustream, but where you have much less control, and compatibility with the QuickTime format is not always assured.

The whole point of streaming is that you do not need to wait for a video file to download completely before you can start viewing that video, you can start watching almost instantly. True streaming is one-to-many. If you are watching a stream and another user joins you, they will start watching the stream at the same point as yourself.

Youtube is the best known example of video streaming, but they use the Flash flv format which cannot be used in SLOS. It is possible to access an mp4 version of Youtube videos using any of a number of add-ons for browsers, particularly Firefox, that create a 'Download mp4' option on the Youtube page, and then rather than downloading the mp4 you could right-click and use the Copy Link Location option to get the URL link which you can paste directly into the About Land, Media tab, Media URL in SLOS. However, this often breaks as Google keeps changing the code for the mp4 streams on Youtube, and the URL links are now extremely long.

Do it the Easy Way
Until recently, I used the easiest method of all, and simply uploaded my mp4 files to a file hosting site, and linked directly to them. This works, but it is not streaming. I have to wait a minute or two for the video to download before it starts playing, and others who join me must do the same, so we are all watching the same video but at different points. If you leave the parcel while the video is playing, then return, the video will start again from the beginning, no matter how briefly you left the parcel.

The good news is that there is a middle way, and is easy, and has zero cost.

You can employ something called pseudo http streaming, but rather than one-to-many it is many one-to-ones. What this means is that I can watch a stream, but when another avatar joins me, the video will stream from the beginning for him, rather than at the same point as my video stream, but we both enjoy the advantages of streaming and do not need to wait for any files to completely download before watching the feed.

How do you set up pseudo http streaming?
Both mp4 and flv files are capable of being streamed using pseudo http streaming, and of course it is the  mp4 format that we are interested in. Servers do not stream mp4 files by default, and require a server side module to enable it. If you have your own server and wish to do this yourself, look here.

Fortunately, several free file hosting sites appear to be streaming module enabled. So I will explain the workflow for employing one of these sites.

Prepare your Files
You will need to first get your video files into the mp4 format. A good, free, and malware-free converter is Super (c) available from here (download link near bottom of page) and another is H.264, available here.

Super (c) Video Converter

As you can see, there are a number of parameters that can be set within the mp4 format. Some must be kept as standard, such as the Frame/sec of 29.97, and a sampling frequency of 44100 for the audio. The others can be experimented with. As I do not need excellent quality video within my Opensim I have gone for low settings: 320:240 video scale size; and video and audio bitrates of 160kbps and 32kbps respectively. This translates to a typical 3 minute music video file size of just under 20MB. This keeps well within the limits imposed by free file hosting sites, and ensures a fast start when played. Even better would be to select the typical 3G video scale size of 176:144, which gets the file size down to 10MB, and the difference in quality is almost imperceptible in SLOS.

Now, there is a trick to all this: many video studio software suites, and video encoders / converters, produce mp4 files with their metadata located at the end of each file, and most do not inform you about this, or give you any option to change it. This means that the entire file must be downloaded first before playback can begin. However, by moving the metadata to the front of the file the video can be streamed and watched almost instantly. A handy, free converter is available to do just this, simply called MetaData Mover. This great utility works on a folder of videos, rather than just on a single file. So create a new empty folder on your desktop and copy the files you want processing into it. Point MetaData Mover to the folder and it will process every file in the folder, moving the metadata from the back to the front of the file. One file typically takes 10 seconds to process.

You are now ready to upload your files.

File Hosting
The first file hosting site I tried did the job perfectly, so I am sure there are many more that are streaming module enabled. I used Fileden, which is a file hosting site, just requiring the registering of a free account. Once you have uploaded a file you are given the direct URL to the file. Not all file hosting sites provide this facility, so do shop around for one that suits you.

Once you have the URL to your mp4 file, just paste it into the media box of About land and the video should start playing almost instantly. If you are using some kind of video jukebox then just load the jukebox with the URLs of your uploaded video files. Needless to say, ensure that you avoid copyright infringement.

Playing your Video Streams
To set up a land parcel for playing video in SLOS, as mentioned previously,  requires that you either be a land owner, estate manager, or member of a group that has the privileges to set the Media for streaming video. This option can be accessed by either clicking the address in the title bar of the viewer, or by right clicking the ground and selecting the About Land option, then go to the Media Tab.

There are two main settings here for video. The first is the texture that will be replaced by the video when it plays, and the second is the URL for the video file. The texture can be any picture texture you like, but the same texture must be on the video screen that you intend to use. So don't use a common texture, like brick, or every house in your sim will turn into a video screen! You will also need to set up your Audio and Video settings in Preferences to enable streaming media. See Torley's video here on how to do all of this.


Have fun,

Rock

Following the previous article outlining my experiments with using the Kinect for creating animation files for Second Life and Opensim, I thought an update might be in order to summarise some of the lessons learnt, both by myself and other experimenters.

Since the last article on the 24th February, two of the key software applications used in the workflow, Brekel Kinect 3D Scanner, and Bvhacker have had further development. So what's new in the latest releases?

Brekel Kinect 3D Scanner
Two further releases have been issued since the last article:

v. 0.39
The most important changes in this release for SL and Opensim users is that now BVH positions are now OFF by default. Some experimenters were creating bvh files with Brekel, importing them in bvhacker, and in there they played OK, but when optimised by bvhacker, and then uploaded into Second Life, all kinds of problems were seen. As Deepgreenseas reported:

When I brought into SL, my avatar kind of floated oddly in the air, with arms straight out to the sides doing weird leg and abdomen contortions having nothing to do with what I saw in bvhacker. So, I'm at a total loss what I'm doing wrong. Any ideas?

 The problem was that in the BVH Capture section of the Brekel window, there is a check box for "Write
Positions." This check box was set ON by default, and so the resulting bvh file had both translation and rotation data recorded, while bvhacker and SL/Opensim require only rotation data. In v.039 this setting is now OFF by default, so only rotation data is present in the bvh files now.

Other improvements included easier installation, a more compact GUI,  a warning when running below 30fps (and thus dropping frames), more tooltips, and other improvements and the squashing of some bugs.

v.0.40
New features/improvements include:

• knees/feet should be better now when turning around

• Second Life format for bvh export option added *

• hand rotations are now supported using some custom calculations 

• added motor/LED support

• you can tilt the Kinect up and down (see slider in the CPU/FPS dock)

• the LED color will be set according to the skeleton calibration phase

• please see the How To under the Help menu on how to update the driver 

This last improvement, the ability to control the Kinect movement, is a big benefit, and will save me from all those fiddly adjustments to my tripod settings.

* I have not yet tried the Second Life option when saving bvh files. I would be interested if it now solves Twisted Pharoah's problem below.

Bvhacker
The latest release is now v.1.8, and the new features and changes are as follows:

• Added three new video tutorials to the help section
• Now possible to merge joints with their parent - useful for converting skeletons with roll bones
• Now possible to delete descendants - useful for removing multiple finger or toe joints
• Now possible to zero out descendants - see new video tutorial for use
• Added preferences dialog (Ctrl P)
• Now possible to set the default file view for open and save dialogs
• Now possbile to show or hide the head mesh display
• Added Zoom In and Zoom Out buttons to the view shortcuts
• Removed 'Show first frame' view option

There is also a note on the file properties pane saying that the joint count includes End Sites. This had confused me at first as earlier research had shown that SL/Opensim typically uses 24 joints, yet when I counted them in bvhacker I could only see 19. This is because the 5 End Sites (one for the head and two each for the hands and feet) should also be added. End Sites can also be manipulated, to make the feet point backwards, for example, but I will leave it to the more imaginative experimenters how they use these End Sites.

I will now provide some further info about joints/bones and some of the problems faced by differing naming conventions used in various applications.

Joints/Bones
Some sources refer to the constituent parts of a bvh file as being composed of bones. This also had me confused at first, but Dave (the author of bvhacker) explained it well:

Although the terms 'bone' and 'joint' can be used interchangeably, I
prefer 'joint', as the position, rotation and translation apply to the
point in space where the joint actually is. Mathematically, the 'bone'
is actually just the space in-between two joints, so I personally find
it easier to think in terms of joints instead of bones.

Unfortunately, SL/Opensim mainly uses the bones naming convention, so rForearm is the bone between the wrist and elbow in the right arrm.

The Nite tracking software for the Kinect and Brekel use joint names, so they do use RightWrist and RightElbow. The list of joints they use are:

Hips Chest Neck Head End Site

LeftCollar LeftShoulder LeftElbow LeftWrist End Site

RightCollar RightShoulder RightElbow RightWrist End Site

LeftHip LeftKnee LeftAnkle End Site

RightHIp RightKnee RightAnkle EndSite

The list of bones/joints used in SL/Opensim are:

hip abdomen chest neck head End Site

lCollar lShldr lForeArm lHand End Site

rCollar rShldr rForeArm rHand End Site

lThigh lShin lFoot End Site

rThigh rShin rFoot EndSite

As you can see, the NITE tracking software for the Kinect does not use both joints in the torso (chest and abdomen), just the chest, and explains the difference in the number of joints/bones, i.e. 23 and 24.

Bvhacker is designed to do the conversions between standard bvh files in joint-naming format and  the SL format using bone-naming format. However, it has been noted that the bone names in NITE/Brekel was causing some problems in Bvhacker, which renamed two joints to the same bone name. This was found by Twisted Pharaoh in the SLuniverse forum:

BVHacker renames 2 joints lShldr and 2 others rShldr.

What you should do is rename LeftShoulder into lCollar manually, RightShoulder in rCollar then do the Attempt SL Joint naming.

To manually rename joints in Bvhacker just load the bvh file, then in the joints pane on the left, select the joint and right click, and select Rename Joint.

Other Tips
Sarah Kline:

I discovered assuming a T pose before recording starts helped the shoulder rotation problems I had when bringing in to bvhacker.

 If you have any other Tips or suggestions for improving the current workflow, (if you find a cure for slight jitter, for example), then please add your comments to the article and I will provide a further update later.

Rock

I have been experimenting of late with the Xbox Kinect as a cheap Mocap source to generate BioVision Hierarchy (BVH) files for upload into Second Life, and the results are pretty encouraging.

What you Need

Hardware
A standalone Kinect, with power supply (this comes as standard when purchased as a standalone), if you buy it as part of an Xbox bundle you will need to buy an additional cable sold separately to connect it to the PC. I bought my Kinect from a local gaming store for Euro139.

Kinect, showing its USB connector and power adaptor

Software
Brekel Kinect 3D Scanner
OpenNI
SensorKinect Drivers
NITE User Tracking Module
Bvhacker

Procedure
For my procedure I used a laptop running 32 bit Windows XP , although Brekel Kinect is confirmed to work also with win7 x86 & x64 as well as XP x64 & x86, but no Mac/Linux version of Brekel Kinect planned.

1. Once you have unpacked your Kinect you need to place it on a table or a tripod (which is what I did), at around 0.6m - 1.8m from the ground. You can plug it into a power socket at this stage, but do not connect the USB connector to your PC/laptop just yet.

Kinect mounted on an inexpensive camera tripod

2. Download and install the following software in this order:

Brekel Kinect 3D Scanner v0.36

OpenNI Alpha Build for Windows v1.0.0.23

PrimeSensor v5.0.0 (Modules for OpenNI)
Click the Downloads button and choose the zip file. After downloading extract the avin2-SensorKinect folder, and within this go to the bin subfolder, and run SensorKinect-Win32-5.0.0.exe

PrimeSensor v5.0.0 with correct options selected

PrimeSense NITE Beta Build for Windows v1.3.0.17
Use this key during installation: 0KOIk2JeIBYClPWVnMoRKn5cdY4=
(This is NOT a pirate key)

NITE Installation Wizard

Copy and paste the License Key from above

 Now download and install Bvhacker 


3. Once the software is installed you can  connect the Kinect USB lead. The Found New Hardware dialog should appear.

No need to connect to Windows Update

When it asks if it can connect to Windows Update, choose No, not his time. On the next dialog screen choose to Install the software automatically.

The hardware that will be detected are the Kinect Motor, Xbox NUI Audio and Kinect Camera. The drivers for the Xbox NUI Audio will not be found and will fail, but ignore this, they are not needed.

Device Manager showing the new device, PrimeSensor

When the drivers have been installed you can check that the hardware and its drivers are ready to be used by Windows by going to Device Manager (Start, Settings, Control Panel, System, Hardware, Device Manager) and you should now see a new entry, PrimeSensor, and under that will be the Kinect Camera and Motor.

4.You can now launch Brekel Kinect.

Brekel Kinect GUI

Brekel may launch with a Dark skin. You can change this by unchecking it under the Window menu item. I found I had to do this as some of the check boxes did not show too well with this almost black skin.

The only setting you now have to change is to enable NITE User Tracking at the centre bottom of the window.

The only setting to change, make sure NITE User Tracking is enabled

Once NITE User Tracking is enabled you should get a message in red on the central image that says: User1 [Looking for Psi pose]. Ensure that your full body is visible in the video windows, then adopt the Psi pose (like you are a trident, or surrendering). Once the pose is recognised it will take just a few seconds for tracking to commence (be patient, and KEEP STILL!), and the message should now go green.

Psi Pose

 As soon as you are being tracked (indicated by white dots covering your body) you can start recording animations. Start with simple ones like the ubiquitous salute, or hand clap. To start recording to BVH click the Start Capture BVH button (near the top right). If you are on your own you may find that tracking is lost if you go out of the field of view of the Kinect, in this case either try to have the PC/laptop within reach, or have a friend start the capture. There is a four second delay (useful if you are on your own) before recording starts. When done, click the Stop Capture to BVH button.

For full details on the capture procedure, and what can go wrong, go to the main menu, Help, How To - Capture motion to a BVH file. Additional resources may also be found on the main Brekel website.

Brekel can save BVH in two main formats, HumanIK (used by MotionBuilder and others) and Biped (used by 3DSMax). Neither of these can be imported directly into Second Life, as the naming convention for Root, joints and bones are slightly different, this is where Bvhacker comes in.

To prepare the BVH file for Second Life, start Bvhacker then File, Open, and load the BVH file. Once loaded you can play it to see what it looks like using the Start button at the bottom of the window. If all is well go back to the menu, Hack, and choose Attempt SL joint naming. This should go without a hitch, and you can now save it out ready for importing into Second life.

Have fun,

Rock