Invert face mesh?
I am trying to figure out how I can accomplish an effect that I have seen in previous Snap lenses. I want to create a face mask which only works outside of the users face.
I am trying to find an example of this and could only find these:
Basically, I want to make the same effect as the dollar bills in the above photo. I want to have a texture cover the entire lens besides the users face.
Can someone help me out?
That's called a billboard effect. The tutorial is right here https://lensstudio.snapchat.com/guides/2d/billboard/
How do you program it so that it reacts with a face mask though?
In that case, just position the object outside of the face. It doesn’t have to be attached to the face and you can trigger animations as normal. Maybe someone with more lens coding knowledge may have an easier code to have billboards react to face gestures.
Cyrene is correct, we can use billboard to accomplish this. Positioning the image in our billboard to not cover the face will also help.
Another way you do this is to use Opacity Texture field on the material of your billboard. Opacity Texture will allow you to add an image to punch out your billboard image with. To enable Opacity Texture, we need to create a custom material for our billboard. To do this: select your billboard and click the Material field in the Sprite component.
Then in the popup window, select Add New > Unlit
Select the new Material in your resources panel. Once selected, in your Inspector Panel, press the Blend Mode field and use Normal. This will allow your billboard to use transparency.
Then, check the box next to Opacity Texture.
Here you can apply a texture with an approximate cut out of the face and shoulder and use it as your Opacity Texture. Your Opacity Texture should be either black on white, or black on transparent. Where your Opacity Texture is black will be the part of your billboard that will be cut out.
Check out the gif below for what this might look like on a purely green texture.
Using Head Binding to move Opacity Texture
For a more advanced technique, we can use Head Binding to move the Opacity Texture with the user's face. What we want to do is create a texture, where the black part of the Opacity Texture moves with the user's movement.
In your Objects panel, press Add New > Face Sprite
Then, set your Opacity Texture as your new sprite's texture.
Now in the camera, you should see your Opacity Texture attached to the head.
Select your Sprite Binding object, and set the Attach to Point to Face Center since we want our cut out to move with the face center.
Then, in your 2D scene panel, resize your face sprite so that the head matches with the preview head.
You will notice that our sprite is bigger and will go beyond the screen. This is Ok, since we want the user to have some leeway in moving around.
As we said earlier, your Opacity Texture should be black on transparent, or black on white. In your objects panel, select your Camera object and set its input to a white image.
Now in your preview window, you should be able to see a black and white image, where the black part follows your head movement.
Before we put this all together, we just need to set our Orthographic Camera's input to be directly from the Device Camera since we don't want to show this black and white image to the user.
Now, we can select the Billboard material we created earlier, and replace our Opacity Texture with "Camera Output"--which as we saw earlier contains the black and white image that moves with the user's face.
Your final result should be like below!
Let me know if I can clarify anything. Can't wait to see what you make!
WOW! You don't even need a green screen! I'm going to try this!
haha this is awesome, thank you Johnathan, i had a crude version but now i see this is how its done, cheers.
Thank you so much Jonathan Solichin, this is a huge resource for me and it looks like a few others as well.
I am almost done with my creation... hopefully tomorrow. I might still have one more question for you, but it will have to wait until I get back to the project file on my work computer.
Thank you so much for the detailed response, it was awesome documentation.
OK! So I ran into a problem after I created the inverted background using the strategies you gave me from above.
I started with the 2D face object template, then I added the opacity billboard.
Now, my face tracked 2d objects are not showing... how can I fix this?
Thanks in advance!
Wow that looks awesome!
Ah, the reason that the object doesn't show up, is that in the instruction above, we used the Camera object to create the mask so it no longer will render the attachments.
So in order to show our head attachments, we need to create another camera for it.
Setting our mask apart
The first thing we need to is put our mask in a different layer so that the camera rendering our attachments won't render our mask.
Select your Camera object, and in its inspector, press the Layers drop down and press the "add new" button. Then uncheck the "Default Layer". This will tell our camera that we want it to only render things in the new layer (Layer 1).
Then, select the sprite which contains your mask and set its layer to Layer 1
Creating a camera for our attachments.
Now we can add our new camera which will hold our head attachments.
In the Objects panel: press Add New > Head Binding.
This time, we will set it to render only things in the Default Layer
We can put all our head attachments (in this case the sweatband, under this new head binding).
In other words, all our mask related object (camera and sprite) will render in Layer 1, and all our attachment related things (Perspective Camera, Face Occluder, etc.) will render in Default layer.
One other thing we might need to do is to make sure our Head Binding is still binding to the first head. To confirm this, in your Objects panel, select the Head Binding under the Perspective Camera, and set it's Face Index property to 0 in the Inspector panel.
Hooking it all up
Finally, lets hook up all our cameras together.
Since we want our Perspective Camera (which renders our attachment) to add on to our original camera feed, in its input, make sure it uses "Device Camera Texture"
In the Orthographic Camera, set its input to "Perspective Camera Output" since we want to show our background on top of our attachment.
Then, select the Orthographic Camera Output in your Resources Panel, and check the "Output Texture" box in its Inspector Panel so that we render our set up.
In other words, our set up is Device Camera > Perspective Camera > Orthographic Camera > Output.
Our Camera object is not in this list because it is being used by the Opacity Texture of our billboard (green background).
Let me know how this goes! It's looking great so far :)
Hey Jon! Do You happen to have the asset you used for the human body outline? If you can, please send it to me. Thanks!
Agree, can someone post the project file for download? It'll help. Thanks!
Hi Isaac, Cyrene,
Here's an example project file: https://storage.googleapis.com/snapchat-lens-assets/f1a09194-f02d-43ed-92b8-62e843179ff0/lensStudio/Forum-Helpers/invert-face-mesh-example.zip#asset:5404
In the example above I quickly made a throwaway silhouette. I probably just drew a circle and some boxes to make the body and then used gaussian blur. I recommend you make a better one :) . In the example project, I used a blurry square. But you should be able to use anything!
We'll put in a task for the team to make a more complete guide so be on the look out for that!
As always don't forget to share your results!
Thank you! Have a great day.
Hey there! I am following this exactly, but my background is an animated gif (with alpha) but its coming in blended into the background, not a purely opaque animation with transparency. What am I doing wrong?
Hi Topher, Can you give a screenshot of what you mean? I grabbed an animated texture from the Giphy importer to the example project file from above and it seems to work fine?
Here is the lens. https://www.snapchat.com/unlock/?type=SNAPCODE&uuid=a7dd3ff1415e4584992a0303730cef93&metadata=01 I went with a still image because I couldn't figure it out, but it seems like the blending mode is not set to Normal, more of a multiply or screen effect.
Thanks for sharing Topher! Good use case.
Can you give a screenshot of the issue you're describing?
By the way, if you haven't, try out the particle template. It would be pretty cool to have ballon particles so that it's dynamic. But it's great either way :)
I guess I didn't think of that... That might be better. Can you set 3 different balloons as separate particles, and have those emit, or do you need 3 particle systems? I tried this with the flipbook technique in a previous lens (you helped me with the poker chips) but I had to duplicate the particle system and have 3 different ones to get the different colors.
Does this question make sense?
The easiest way is to create three particle systems.
If you want gradient of colors, you can use the Premultiplied Color (and setting Color min and Color max) option, to get colors within these ranges.
Take a look at https://lensstudio.snapchat.com/guides/general/particles/ for more information.
Let me know if you need clarification :D
Does it also make sense to use an occluder to do this?
We actually just released a way easier way to recreate this segmentation effect. Download the latest update today here and tryout our Segmentation Template!
every time i make a 3d lenses why i cant get gold star lens creator please see my lenses
this is link my ptofile every my loenses in my profile