Hand Tracking Orientation

  • The local orientation appears to change with left and right hands. If I were to hold my thumb up on my left hand, I'd get a Euler Z of 257. If I hold my right hand up I'd receive a Z of 77. Turning my left hand down I'd receive something around 77, and right hand down would get a 257. I've also tried getting world rotation coordinates instead of local. 

    Should I create a tracker for upper body? Would that help me determine if my left or right hand is up and I can rotate the coordinate system from there? 

    Comment actions Permalink
  • Do you think it has to do with what object your tracker object is parented to? i.e. do you see the same behavior if you check world rotation instead of local rotation? Since an image will orient to the hand as it's tracked, you should definitely be able to retrieve its rotation, but the value you receive from getLocalRotation() will of course depend on how your parent object is behaving. It might be worth a try to attach some image to the tracked object and check that image's world rotation...?

    Comment actions Permalink
  • Would be nice if someone has an answer. Im also curious how to do this :D




    Comment actions Permalink
  • Interesting point Michael, It doesn't look like the Hand Tracker is parented to the Upper Body tracker, but I did manage to come up with this method which I think works...

    var wristLeft = script.trackerWristLeft.getTransform().getWorldPosition();
    var wristRight = script.trackerWristRight.getTransform().getWorldPosition();
    var hand = script.trackerHand.getTransform().getWorldPosition(); // only identifies one hand

    var dist1 = wristLeft.distance(hand);
    var dist2 = wristRight.distance(hand);

    var minDist = Math.min(dist1, dist2);
    if (minDist === dist1) { // detected hand attached to Left
    } else if (minDist === dist2) { // detected hand attached to Right

    So for this example I create an Upper Body Tracker and pass that into the script as WristLeft and WristRight. Then I take the Hand Tracker position and since it only tracks one hand I measure the distance from the wrist. The minimum distance to the wrist would allow me to assume it's a Left or Right hand. 

    I'm also noticing it's taking a bit more CPU power to calculate this, so I'm going to try and delete some of the Upper Body trackers to reduce the load. 

    For a feature request it would be nice to detect both hands and populate them as an array. This would more closely replicate the Upper Body Left/Right detections. 

    Not sure if this is the best way to do it, but I'm open to suggestions!

    Comment actions Permalink
  • Oh nice one Brent. I finally understood the behavior you were explaining after trying some things in a project where I'm tracking an open hand. You're of course right that the thumb up/down doesn't offer an easy way to get orientation. One simple way that seems to work for me is to put two hand trackers in: one is set to track the hand center, and one is set to track the thumb joint. That way you can simply compare world y positions to know if the thumb is above the hand center or below it, giving you thumbs up/down, regardless of which hand (when combined with the gesture recognition of course).

    Not sure if the double object tracker would be considered good practice officially, or what performance implications might be, but it does seem to work. Obviously if your solution is doing it for you, this may not be so useful, but anyway:




    //@input SceneObject handCenterImage
    //@input SceneObject thumbJointImage

    function onUpdate() {
    var center = script.handCenterImage.getTransform().getWorldPosition();
    var thumb = script.thumbJointImage.getTransform().getWorldPosition();

    // very rudimentary check; assumes gesture reco already
    // confirmed thumb up gesture:
    if (thumb.y > center.y) {
    // thumbs up
    } else {
    //thumbs down
    Comment actions Permalink
  • Thanks Michael, that works much more consistently than my attempt. Appreciate the collaboration. 

    Comment actions Permalink
Have a comment?

Please sign in to leave a comment.

We're here to help! We just need a little bit of information...

What system are you using?

Have you downloaded the latest version of Lens Studio?

Please download the latest version of Lens Studio. If you still run into this issue, please come back and report it!

Is this issue causing Lens Studio to crash?

What system do you run Lens Studio on?



Follow the next steps to help us solve your issue:

  • Copy and paste this text into your TerminalCommand Window
    open ~/Library/Preferences/Snap/Lens\ Studio/ %LOCALAPPDATA%\Snap\Lens Studio Copy Text
  • Press ReturnEnter to run the command. The Lens Studio folder will automatically open
  • Prepare to upload your files: zip the "Log" Folder by right-clicking and choosing "compress."
    Locate the Log.txt file right above it.

    Attach a screenshot of the issue:



What is this most relevant to?

Please enter a brief description of your issue:

Thanks for submitting this issue.

Unfortunately, it's likely due to the operating system or hardware you're using – since they don't meet the system requirements for Lens Studio.

Still, we hear you loud and clear, and are logging the issue in case there's ever a workaround we can provide!

Minimum Requirements

Operating System: Windows 10 (64 bit); MacOS 10.11+

Hardware: Minimum of Intel Core i3 2.5Ghz or AMD Phenom II 2.6Ghz with 4 GB RAM; Intel HD Graphics 4000 / Nvidia GeForce 710 / AMD Radeon HD 6450 or better; screen resolution of 1280x768 or higher

We'll try to resolve this issue as soon as possible. Thanks for letting us know about it!

Keep an eye out for a followup email from us. We may have a couple more questions for you, or we might already have a solution to offer.

Happy creating!