Doggie Detective

Doggie Detective is an experience that leads players on a location-based scavenger hunt to solve a detective case of a missing dog.  This is a Northstar Project; it combines multiple experimental features to push the boundaries of our technology.  Due to the project’s complexity, it is best suited for creators who are familiar with Lens Studio and comfortable with coding.

To familiarize yourself with the building block features of this project, please look into the following: 

• Endurance Mode
• Location Trigger
• Scavenger Hunt
• Visual Scripting
• Behavior Script

Project Download

Please download the project here.

 

Use Cases

Doggie Detective demonstrates how you could create a 40-60 min experience that brings players to multiple real world locations.  We think this could be a great starting point for making some of the following: 

• Travel Guide
• Live Action Roleplay
• Detective Game

 

API

Compatibility

Make sure you have updated to the latest version of the Snapchat App on your phone, the Spectacles firmware, and Lens Studio.

• Snapchat v11.51.x and above
• Spectacles v4.67.x and above
• Lens Studio v4.4.1 and above

Enable Experimental API in Lens Studio

The experimental features in this API require that you update to Lens Studio 4.4.1 or later.  In your Lens Studio project, you must enable experimental features:

1. Open Lens Studio -> Preferences and check “Allow Experimental API.”  Now the “Allow Experimental API” option will show in Project Info.  
2. Open File -> Project Info and check “Allow Experimental API.”

Endurance Mode API

Please visit the Endurance Mode template to learn more about the Endurance API.  In the Doggie Detective project, Endurance API is handled in the “EnduranceHelper” script.  The naming conventions of the handling functions are slightly different, but the underlying API is the same.  Refer to the comments in the “Endurance Helper” script for guidance. 

Location Event API

Please visit the Location Trigger template to learn more about the Location API.  In the Doggie Detective project, Location API is handled in the “WaypointFinderHelper” script.

 

Project Walk-Through

Lens Controller

The “Lens Controller” script runs the project.  LensController holds references to all of the key scripts and vice versa.

The LensController onLensStarted() function also handles displaying a notification for the player to enable the Experimental Features Background and Endurance Mode, as described in section “Previewing Your Lens” below. 

Tip: In Lens Studio, the order scripts appear in the Objects panel is the script execution order.  Therefore, LensController is towards the bottom.  Because we’re sending behavior events in code, we need a behavior script at the top.  Same goes for the tween manager.

 

Notifications

LensController.onLensStarted() handles necessary notifications for the player:

• If they open the lens on a phone (i.e. endurance API is not available), the lens won’t work.
• If they open on Spectacles 2021, but don’t have the following experimental features enabled, the lens won’t work.
  • Toggle ON Background Location
  • Toggle ON Endurance mode
• Phone -> Snapchat -> Spectacles Home -> Experimental Features
• Note: we can’t know if the player actually toggled the experimental features.  Instead, we notify the player and ask them to tap the touchpad once they’ve toggled them.  If the player continues without the experimental features enabled, the lens will fail silently. 

In the future, these notifications should be coded into the Endurance and Location Event API.  But for now, you have to handle them in your Lens Studio project. 

 

Debugging

Due to the custom Endurance and Location Event API, Doggie Detective does not run in Lens Studio or on phone.  However, you can run one Waypoint at a time in Lens Studio, and this is useful for debugging.  To do so, drop the Waypoint stage you’d like to test into the first slot of the StageController.  In LensController, set “debug” Boolean to true. 

 

Camera

Anything for the Spectacles 2021 will be displayed in a 3D space, so always set the Camera Type to “Perspective Camera.”

Tip: Unless you are being careful about only enabling one camera at a time, only use one camera.

 

UI

To display text and images in screen space, they are parented to a “UI” transform, which is parented to the perspective camera at position of 600 cm in front of the camera. 

Tip: In Lens Studio, the camera faces backwards.  To place an object in front of the camera, parent it to the camera and set its z position to -x units.  Lens Studio units are cm.  So -100 units should be visible.  On the Spectacles 2021, anything < 10 cm in front of the camera is probably too close to the player’s face to be seen. 

Device Tracking: Making an Object “stick” to a Position in Space

To make an object “stick” to a position in space, we need to track the position of the camera (the Spectacles on the player’s head) in space.  To do this, add a “Device Tracking” component to the perspective camera and set its Tracking Mode to “World.”  Now, objects placed in the world (which are not parented to the camera) will stick to that position while the camera moves around them.  

 

Stage Controller

So long as we’re ready, the Lens Controller initializes the “Stage Controller.”  Scripts StageController and Stage form a state machine that progresses the project from one stage to another.  They were taken verbatim from the AR Scavenger Hunt example project.  Visit that link for more information. 

 

The stage controller works by disabling objects of the last stage and enabling objects of the next stage.  In Doggie Detective, each of our stages are either of the following: 

• Waypoint, which is controlled by script WaypointHelper
• Waypoint Finder, which is controlled by script WaypointFinderHelper

Waypoint Helper

Waypoints are narrative stages that show character animation and voiceover.  When a stage enables a Waypoint object, it also enables everything below that object.  

 

We use Visual Scripting to drive the unique narrative events of each Waypoint.  Visual scripting uses node-based script graphs.  Each WaypointHelper object holds a reference to a script graph, which is attached to the “Character” objects parented to the Waypoint.  The script graphs in turn hold references to the WaypointHelper. 

 

When the Waypoint is enabled, WapointHelper.onWaypointStarted() orients the Waypoint to appear some distance in front of the player and starts the script graph by calling: 

script.scriptGraph.api.playVo();

 

The “Show UI” boolean will turn on the 2D Waypoint UI and display the Waypoint Name, Instructions, and Image referenced in the Inspector.  This is used for the first and last Waypoints.

Tip: Use “Show UI” to prototype. 

 

Waypoints 1-4 use 3D art parented to the Waypoint instead of 2D UI. 

 

Set the “Progress to Endurance Mode” boolean to true if the next stage is a WaypointFinder stage. 

 

Set the “Show Arrow” boolean to true if you want to use the arrow to guide the player where to look until they look at the specified object (see Look Arrow section below).

 

When the script graph progresses through the various animations and sound.  At the end, the script graph calls “onWaypointEnded()” on its WaypointHelper.  

Then WaypointHelper.onWaypointEnded() fires the Behavior script message that tells the StageScript to progress to the next stage like so:

global.behaviorSystem.sendCustomTrigger(script.completeTrigger);

 

Waypoint Finder Helper

Trigger

Waypoint Finders are Endurance Mode stages that run while the player is walking from one Waypoint to the next.  The WaypointFinderHelper is set up to trigger the next stage in two ways:

1. Using time (useful for debugging)

2. Using location event (visit Location Trigger API to learn how to update this functionality with your local coordinates)

To enable you to test this lens at your desk, we have defaulted to enabling “Time Mode” and commenting out “Location Mode”.  In order to use Location Mode, update the coordinates to coordinates local to you, comment out the “Time Mode” block of code, and uncomment the “Location Mode” block of code. 

Hint: Waypoint Finders reference and set the Endurance Mode hint text and image.

 

Endurance Helper

Trigger

The Endurance Helper script controls entering and exiting Endurance Mode.  Study the Endurance Mode API to better understand this functionality.  

Note that Doggie Detective essentially exclusively uses the StageScript Trigger Type “Custom Trigger” to progress from one stage to the next, but you could experiment with many other triggers.  Visit Endurance Mode documentation -> Use Cases -> Triggers for a table on how we think various Behavior script triggers will work in Endurance Mode.

Hint

Note that during Endurance Mode, a touchpad tap/hold will trigger the display to briefly turn on without the usual “Entering/Exiting Endurance Mode” notifications for the purpose of showing the player a hint for what to do next.  If you have an active touchpad tap/hold event registered, this event will fire in parallel to the system level “Hint” functionality.  

 

Look Arrow

Doggie Detective uses the Look Arrow and Behavior script to ensure the player has looked at the character before the character starts talking.  The LookArrow script controls showing, hiding, and orienting the arrow. 

UI Visualization

The LookArrow script is attached to an arrow model parented to the UI object parented to the Camera so that it always appears in front of the player; only the rotation of the arrow changes. 

Behavior Script Trigger

The Look Arrow is shown and hidden by a Behavior Script parented to the Waypoint.  

Behavior scripts are very flexible, but unfortunately they only allow one response per trigger.  We need two things to occur when the player looks at the target: 

1. Progress the script graph
2. Hide the arrow

 

Therefore, we need two Behavior scripts.  The first Behavior script checks if the player is Looking At the Look Target within a given angle.  If so, it sends a Custom Trigger message (Trigger Name).  This message is received by both the script graph and the second Behavior script. 

The script graph uses the “Add Custom Trigger Response” node to set Boolean “breakFirstLoop” to true.  Everytime the character finishes their wave animation loop, they check this bool.  If it’s true, they progress to the next animation.  (For more information on using Visual Scripting for animation, see the “Animating With Visual Scripting” section below).

The second Behavior script hides the arrow by calling that function on LookArrow. 

Animating With Visual Scripting

There are two main ways to smoothly transition from one animation loop to another: 

1. Wait for the loop to complete before playing the next animation
2. Blend from within the loop to the next animation 

 

Wait For Loop To Complete

This option is a more cinematic technique that waits for the current loop to complete before transitioning to the next. With this technique, you can use custom transitions to go from one loop to another.  

 

Doggie Detective uses this technique at the beginning of the waypoint to loop the character waving and repeating “look over here” until the player looks at the character.  To study this approach, inspect the script graph “Test_TransitionOnLoopComplete.”  Note that the looping animation is not set to loop.  Instead, at the end of each loop, we check if a boolean has been set to progress to the next animation.  If so, we progress.  If not, we call the current loop to play again.  

 

For this technique, in the Animation Mixer, use Blend Type “Override.”

 

Customizing Waypoints

Replace our FBX files in the Resources folder with Right Click -> Relink to New Source.  Note: unfortunately, this will not update the names of the files in the Lens Studio Project - you’d have to do that manually.  

Another option is to do File -> Import -> and navigate to your FBX.  

 

Upon importing your FBX, you may or may not have individual clips set up.  To add and label clips, click on the AnimationMixer in the Objects panel.  In the Inspector panel’s AnimationMixer component, click “Add clip.”  Input your clip’s name, startFrame, endFrame, and any other details.  Now you can reference that layer (clip) by name. 

Swap out script graph animation and sound assets by dragging them into the editor fields on the script graph.  If you swap animation mixers, go to each “Play Mesh Animation” node and input your new animation layer (clip) name.

Previewing Your Lens

You’re now ready to preview your lens on Spectacles 2021.  To pair your lens with your Spectacles 2021, follow the Preview Your Lens On Device section on the Creating Lenses guide.

 

To enable Endurance Mode on your Spectacles 2021:

• Update Snapchat on your phone
• Go to Spectacles Home in Snapchat
• Tap “Experimental Features”  
• Enable “Endurance mode”

 

Each time you want to use location in your lens, you have to enable Location on your Spectacles 2021.  This feature will only last for one hour.  

• Update Snapchat on your phone
• Go to Spectacles Home in Snapchat.  
• Tap “Experimental Features”  
• Enable “Background Location”