Lens game "Tik Tok"

Hi, Laurent

Definitely this is possible, I've been playing with this concept some time ago. I can share some stuff with you 

So first I started with adding some helpers

I defined a Bone as a combination of three Skeletal Tracking Points - parent, start  and end point. Very similar to this forum post.

var Bone = function (parent, start, end) {
this.parent = { transform: parent.getSceneObject().getFirstComponent("ScreenTransform"), tracking: parent };
this.start = { transform: start.getSceneObject().getFirstComponent("ScreenTransform"), tracking: start };
this.end = { transform: end.getSceneObject().getFirstComponent("ScreenTransform"), tracking: end };
}

Bone.prototype.rotation = function () {
var parentDir = (new vec2(this.start.transform.position.x, this.start.transform.position.y).sub(new vec2(this.parent.transform.position.x, this.parent.transform.position.y))).normalize();
var dir = (new vec2(this.end.transform.position.x, this.end.transform.position.y).sub(new vec2(this.start.transform.position.x, this.start.transform.position.y))).normalize();
return new vec2(parentDir.dot(dir), parentDir.x * dir.y - parentDir.y * dir.x);
}

Bone.prototype.isTracking = function () {
return (this.parent.tracking.isTracking() && this.start.tracking.isTracking() && this.end.tracking.isTracking())
}

Then I represented a Skeleton class which is basically a list of bones

var Skeleton = function () {
this.bones = {};
}

it has a couple of methods for adding bones and debug

Skeleton.prototype.add = function (name, parent, start, end) {
if (start != null && end != null) {
this.bones[name] = new Bone(parent, start, end);
}
else {
throw ("Need 3 points");
}
}

Also it has a getPose method - that gives you all bone data in specific format

keleton.prototype.getPose = function () {
var pose = {};
for (var boneName in this.bones) {
var rotation = this.bones[boneName].rotation();
pose[boneName] = { x: rotation.x.toFixed(5), y: rotation.y.toFixed(5) };
}
return pose;
}

And also a method that checks if pose is matching with given threshold. 

Skeleton.prototype.isMatching = function (pose) {
for (var boneName in pose) {
if (this.bones[boneName] != undefined) {
if (!this.bones[boneName].isTracking())
return false;
if (pose[boneName].distance(this.bones[boneName].rotation()) >= threshold) {
return false;
}
}
}
return true;
}

Great! Next we have to create some poses.

In order to do this I created a bunch of photos of myself and loaded them to the preview panel of the Lens Studio

Also I created a helper, that outputs current pose in required format to the logger. 

script.createEvent("TapEvent").bind(function (eventData) {
    print(JSON.stringify(skeleton.getPose()));
});

It outputs something like this. The pose is output as a string version of a JSOn format. 

Then I collected like 10 of different poses, added them as a list of strings to the PoseManager script 

var poseJSONs = []
//1
poseJSONs.push('{"Upperarm.L":{"x":"0.71652","y":"-0.69756"},"Forearm.L":{"x":"0.08380","y":"-0.99648"},"Upperarm.R":{"x":"0.69033","y":"0.72349"},"Forearm.R":{"x":"0.28301","y":"0.95912"}}');
//2
poseJSONs.push('{"Upperarm.L":{"x":"0.99877","y":"0.04968"},"Forearm.L":{"x":"0.96434","y":"0.26467"},"Upperarm.R":{"x":"0.32683","y":"-0.94508"},"Forearm.R":{"x":"0.97679","y":"-0.21419"}}');
//3
poseJSONs.push('{"Upperarm.L":{"x":"0.99877","y":"0.04968"},"Forearm.L":{"x":"0.96434","y":"0.26467"},"Upperarm.R":{"x":"0.32683","y":"-0.94508"},"Forearm.R":{"x":"0.97679","y":"-0.21419"}}');
//4
poseJSONs.push('{"Upperarm.L":{"x":"0.99784","y":"0.06575"},"Forearm.L":{"x":"0.99707","y":"0.07650"},"Upperarm.R":{"x":"0.96594","y":"0.25878"},"Forearm.R":{"x":"0.97441","y":"-0.22479"}}');
//5

and created a textures for them to display (sorry for my drawing skills ;) )

I will not get deeper into details of my gameplay but basically if gives you a couple seconds to fit the given pose. It compares the directions of upper-body tracking points with the current pose and reacts on the result. 

I added behavior script that sends custom trigger to start the game when recording started. For debugging purposes in studio you can change event type to touch

You can find my project here, it is not optimal, just a draft. Please extend it to fulfill your needs

Let me know if there are any questions !

Best

Olha