Avateering (Kinect + Unity) | Vitruvius

Prerequisites

• Kinect for XBOX v2 sensor with an adapter (or Kinect for Windows v2 sensor)
• Kinect for Windows v2 SDK
• Windows 8.1 or higher
• Visual Studio 2013 or higher
• A dedicated USB 3 port

Source Code

The source code, along with the required assemblies, is included in the following versions of Vitruvius:

• Academic
• Premium
• Platinum

Step 1: Create a new Unity project

Launch Unity and create a new project, or open an existing Kinect project. Read our Getting Started Guide to see how easy it is.

You’ll also need to import Windows.Kinect.dll and LightBuzz.Vitruvius.dll. These are the required assemblies to use the complete power of Vitruvius.

Step 2: Import the 3D models

Open your scene file in the Editor and import the 3D models. Place them whenever you think it’s necessary for your game. I have imported the male and female 3D body models. They are named “male” and “female”, respectively.

This is how my editor looks like:

Step 3: Action!

It’s now time to write your Kinect script. You’ll need to include the following namespaces:

using Windows.Kinect;
using LightBuzz.Vitruvius;

After importing the assemblies, we’ll create the required members to open the sensor, acquire the Body data and update the avatars accordingly.

The separatedPlayers variable indicates whether a single player will control one avatar or multiple avatars.

Members

// Kinect
BodyFrameReader bodyReader;
Body[] users;

// 3D models
Model[] models;

public FBX female;
public FBX male;

// Determines whether
// one player will control all avatars, or
// whether each player will control one avatar
public bool seperatedPlayers;

Start

The Start method initializes Kinect and Vitruvius. It’s important to open the sensor and enable avateering functionality to your 3D models. The code is really simple and self-explanatory:

void Start()
{
    // 1. Initialize Kinect
    sensor = KinectSensor.GetDefault();

    if (sensor != null)
    {
        bodyReader = sensor.BodyFrameSource.OpenReader();

        sensor.Open();
    }

    // 2. Enable Avateering
    Avateering.Enable();

    // 3. Specify the 3D models to animate.
    models = new Model[]
    {
        female,
        male
    };

    // 4. Initialize each 3D model.
    for (int i = 0; i < models.Length; i++)
    {
        models[i].Initialize();
    }
}

Update

Now, it’s time for the cool part! To see your avatars animated, you simply have to loop through the tracked bodies and update the corresponding avatar. Vitruvius does this with just one line of code.

void Update()
{
    if (bodyReader != null)
    {
        using (var frame = bodyReader.AcquireLatestFrame())
        {
            if (frame != null)
            {
                users = frame.Bodies().
                        Where(b => b.IsTracked).ToArray();

                if (seperatedPlayers)
                {
                    // Each user controls a different avatar.
                    for (int index = 0; index < users.Length; index++)
                    {
                       Model model = models[index];
                       Body user = users[index];

                       // Yes, this line does ALL of the hard work for you.
                       Avateering.Update(model, user);
                    }
                 }
                else if (users.Length > 0)
                {
                    // A single user controls all of the avatars.
                    for (int index = 0; index < models.Length; index++)
                    {
                        Model model = models[index];
                        Body user = users[0];

                        // Yes, this line does ALL of the hard work for you.
                        Avateering.Update(model, user);
                    }
                }
            }
        }
    }
}

Dispose

Finally, do not forget to dispose Kinect and clean any resources.

public void Dispose()
{
    // 1. Dispose Kinect
    if (bodyReader != null)
    {
        bodyReader.Dispose();
        bodyReader = null;
    }

    if (sensor != null)
    {
        if (sensor.IsOpen)
        {
            sensor.Close();
        }

        sensor = null;
    }

    // 2. Dispose 3D models
    for (int i = 0; i < models.Length; i++)
    {
        models[i].Dispose();
    }

    // 3. Disable avateering
    Avateering.Disable();
}

This is it! You have just developed an avateering app using Kinect v2!

Bonus – Change textures

Vitruvius is bundled with five 3D models and multiple textures. Simply open the Samples/Models/FBX folder and pick the 3D models and textures of your choice. The following models are included:

• Female – a beautiful female model with various hair colors and clothes.
• Male – a male model with various clothes.
• Longsleeve – a shirt with long sleeves and various colors.
• Tshirt – a simple t-shirt model with various colors.
• Pants – a pair of trousers with various styles (jeans, black, formal).

This is a video of what our partners from Kinetisense did. They are using an animated 3D model to examine and measure the posture of patients. What will you do with Vitruvius? Let me know in the comments below!