In this example, we show how to make a shared AR experience using the Connected Lens Module.
People invite their friends to play together in realtime. Either as friends[a][b] playing in the same physical space using a Colocated Tracking, or as friends playing together in AR when they are each in different physical spaces.
Note: All players are required to be friends with each other on Snapchat to have a shared experience in a colocated space. However, for colocated experiences in remote location, friends of friends can join the same session. Learn more in the Connected Lenses Overview.
Features
In this example, players can place colourful AR blocks together in the world around them.
Developing a realtime multiplayer Lens (or any kind of experience or game) has a unique set of challenges. For each challenge there are multiple solutions, in this example we will explore some of these.
This Lens Studio project features:
- A multiplayer Lens working on device and within Lens Studio’s interactive preview window.
- A Lens which gives the players the option to play in Solo, Colocated or Remote session types and enable different functionality based on the session type using the ConnectedBehaviour.js script
- Error handling of typical error messages that could appear
- Handling multiplayer communications to allow:
- Players who join after the Lens experience starts, to request a list of all the placed blocks (i.e. the current game state when they join) Players to place a block in a shared AR space while triggering the other players’ devices to do the same action
- Players to perceive each other’s AR cursors (a transparent block representing where players are pointing their device)
- Avatars to make players sense the remote presence of each other, during in remote play
Avatar pointers to help players find each other (and sense their presence) when other players are located offscreen
- Players who join after the Lens experience starts, to request a list of all the placed blocks (i.e. the current game state when they join) Players to place a block in a shared AR space while triggering the other players’ devices to do the same action
Notes on sharing data between players
Creating a shared AR experience using multiplayer sessions
The magic of inhabiting a shared augmented reality version of the world with other happens by having each player share their actions with every other connected player.
For example, when one player places a block in their Lens, the Lens needs to tell the other Lens to place the block too. Therefore, the key to a Connect Lens experience is the ability to send messages to a group of players who are connected at the same time.
A group of players come together in realtime by having their Lenses connected to a single multiplayer session. It is this multiplayer session that the Lens uses to send messages to the other connected users/Lenses. A MultiplayerSession gets created by a Lens when a player opens the Lens from the Lens carousel. This Lens can then invite other Snapchat users to open the Lens and join the MultiplayerSession that it has created.
Message passing between players
As the Lens creator, you will want to consider how you pass messages between players to realize your share AR experience. Lens Studio has a ConnectedLensModule. This module gives the Lens access to a MultiplayerSession for sending and receiving messages (using the connectedLensModule’s sendMessage function and the connectedLensSessionOption’s onMessageReceived callback)
Passing different types of messages between players
Let’s imagine we are building a Lens with text chat capability.[d][e] (In reality, we wouldn’t do this as Connected Lens automatically allows players to chat with voice or text using the Snapchat Connect Lens interface). Players would send and receive their chat messages using the sendMessage function and onMessageReceived callback. But what if you wanted more realtime features than just chat messages? For example, if you wanted to send chat messages and place block messages? Now you would need to put something in each message to identify it as being a chat message or a place block message. In this situation, you could start the message with the word ‘chat’ or ‘block’.
Using JSON to send messages
An even better way to do this would be to use the JavaScript function JSON.stringify to turn a JavaScript object into a string for sending (as sendMessage sends strings). Then in the onMessage callback you can use the JSON.decode function to make it a JavaScript object. For example, you could have messages like {“type”:”chat”,”message”:”hello world”} or {“type”:”block”,”color”:”green”}.
Message routing and message type subscribing
In this example Lens, we take this idea one step further using a custom ScriptComponent we’ve created called Packets.js. It takes care of sending messages and allows other ScriptComponents to subscribe to a single type of messages or category of messages.
Message paths and path matching
Look at the Packet.js script to view its API. Using this script you can set up listeners using paths that look like URLs (if you have used Node.js or Ruby before these may be familiar). For example, you can subscribe to messages with a path of “/block/create/”. Or you can subscribe to messages matching a path pattern.
For example, “/block/:action/” (note, the :action) which will match “/block/create/”, “/block/delete/” and “/block/update/”. When you match a path by pattern, a params object containing the dynamic parts of the path will also be created. In our example, the params object for :action would be {action:”create”}, {action:”delete”} or {action:”update”}. This allows you to create a decoupled realtime feature or send and catch one of unique messages (like sending a projectile).
Modularity
In this example, you can see Packets.js being used for features like avatars, block placing, state sharing. As Packets.js takes care of the messaging between players, each feature doesn’t need to know about the other features needs. This allows each feature to be lifted independently to another Lens Studio project.
Simplifying data sharing with BroadcastValues and SyncValues
Packets.js exposes two data types called a BroadcastValue and a SyncValue. Using these data types, you don’t need to worry about how data is passed around as they give you a mechanism to ensure a value gets shared.
A BroadcastValue allows you to share and receive a value with all the other players for a specific path (handy for things that happen frequently like the positions of each player in 3D space).
A SyncValue is similar, but it also ensures anyone who joins a session after a value has been set will get that latest value from each player (handy if you want to share something that doesn’t happen frequently like the type of avatar a player has selected).
