Invert face mesh?

  • That's called a billboard effect. The tutorial is right here https://lensstudio.snapchat.com/guides/2d/billboard/

    Comment actions Permalink
  • How do you program it so that it reacts with a face mask though?

    Comment actions Permalink
  • In that case, just position the object outside of the face. It doesn’t have to be attached to the face and you can trigger animations as normal. Maybe someone with more lens coding knowledge may have an easier code to have billboards react to face gestures.

    Comment actions Permalink
  • Hey Colin,

    Cyrene is correct, we can use billboard to accomplish this. Positioning the image in our billboard to not cover the face will also help. 

    Another way you do this is to use Opacity Texture field on the material of your billboard. Opacity Texture will allow you to add an image to punch out your billboard image with. To enable Opacity Texture, we need to create a custom material for our billboard. To do this: select your billboard and click the Material field in the Sprite component. 

    Then in the popup window, select Add New > Unlit 

    Select the new Material in your resources panel. Once selected, in your Inspector Panel, press the Blend Mode field and use Normal. This will allow your billboard to use transparency.

    Then, check the box next to Opacity Texture.

    Here you can apply a texture with an approximate cut out of the face and shoulder and use it as your Opacity Texture. Your Opacity Texture should be either black on white, or black on transparent. Where your Opacity Texture is black will be the part of your billboard that will be cut out. 

    Check out the gif below for what this might look like on a purely green texture. 

     

    Using Head Binding to move Opacity Texture

    For a more advanced technique, we can use Head Binding to move the Opacity Texture with the user's face. What we want to do is create a texture, where the black part of the Opacity Texture moves with the user's movement. 

    In your Objects panel, press Add New > Face Sprite

    Then, set your Opacity Texture as your new sprite's texture. 

    Now in the camera, you should see your Opacity Texture attached to the head.

    Select your Sprite Binding object, and set the Attach to Point to Face Center since we want our cut out to move with the face center.  


    Then, in your 2D scene panel, resize your face sprite so that the head matches with the preview head. 

    You will notice that our sprite is bigger and will go beyond the screen. This is Ok, since we want the user to have some leeway in moving around. 

    As we said earlier, your Opacity Texture should be black on transparent, or black on white. In your objects panel, select your Camera object and set its input to a white image. 

    Now in your preview window, you should be able to see a black and white image, where the black part follows your head movement. 

     

    Before we put this all together, we just need to set our Orthographic Camera's input to be directly from the Device Camera since we don't want to show this black and white image to the user. 

    Now, we can select the Billboard material we created earlier, and replace our Opacity Texture with "Camera Output"--which as we saw earlier contains the black and white image that moves with the user's face.

    Your final result should be like below!

    Let me know if I can clarify anything. Can't wait to see what you make!

    Cheers,

    Jon

    Comment actions Permalink
  • WOW! You don't even need a green screen! I'm going to try this!

    Comment actions Permalink
  • haha this is awesome, thank you Johnathan, i had a crude version but now i see this is how its done, cheers.

    Comment actions Permalink
  • Thank you so much Jonathan Solichin, this is a huge resource for me and it looks like a few others as well. 

    I am almost done with my creation... hopefully tomorrow. I might still have one more question for you, but it will have to wait until I get back to the project file on my work computer.

    Thank you so much for the detailed response, it was awesome documentation. 

    Comment actions Permalink
  • OK! So I ran into a problem after I created the inverted background using the strategies you gave me from above. 

    I started with the 2D face object template, then I added the opacity billboard. 

    Now, my face tracked 2d objects are not showing... how can I fix this?


    Thanks in advance!

    Comment actions Permalink
  • Wow that looks awesome!

    Ah, the reason that the object doesn't show up, is that in the instruction above, we used the Camera object to create the mask so it no longer will render the attachments. 

    So in order to show our head attachments, we need to create another camera for it. 

     

    Setting our mask apart 

    The first thing we need to is put our mask in a different layer so that the camera rendering our attachments won't render our mask.

    Select your Camera object, and in its inspector, press the Layers drop down and press the "add new" button. Then uncheck the "Default Layer". This will tell our camera that we want it to only render things in the new layer (Layer 1). 


    Then, select the sprite which contains your mask and set its layer to Layer 1

     

    Creating a camera for our attachments.

    Now we can add our new camera which will hold our head attachments. 

    In the Objects panel: press Add New > Head Binding.

    This time, we will set it to render only things in the Default Layer

    We can put all our head attachments (in this case the sweatband, under this new head binding). 

    In other words, all our mask related object (camera and sprite) will render in Layer 1, and all our attachment related things (Perspective Camera, Face Occluder, etc.) will render in Default layer. 

    One other thing we might need to do is to make sure our Head Binding is still binding to the first head. To confirm this, in your Objects panel, select the Head Binding under the Perspective Camera, and set it's Face Index property to 0 in the Inspector panel. 

     

    Hooking it all up

    Finally, lets hook up all our cameras together.

    Since we want our Perspective Camera (which renders our attachment) to add on to our original camera feed, in its input, make sure it uses "Device Camera Texture"

    In the Orthographic Camera, set its input to "Perspective Camera Output" since we want to show our background on top of our attachment. 

    Then, select the Orthographic Camera Output in your Resources Panel, and check the "Output Texture" box in its Inspector Panel so that we render our set up. 

    In other words, our set up is Device Camera > Perspective Camera > Orthographic Camera > Output. 

    Our Camera object is not in this list because it is being used by the Opacity Texture of our billboard (green background). 

     

    Let me know how this goes! It's looking great so far :)

    Cheers,

    Jon

     

    Comment actions Permalink
  • Hey Jon! Do You happen to have the asset you used for the human body outline? If you can, please send it to me. Thanks!

    Comment actions Permalink
  • Agree, can someone post the project file for download? It'll help. Thanks!

    Comment actions Permalink
  • Hi Isaac, Cyrene,

    Here's an example project file: https://storage.googleapis.com/snapchat-lens-assets/f1a09194-f02d-43ed-92b8-62e843179ff0/lensStudio/Forum-Helpers/invert-face-mesh-example.zip#asset:5404

    In the example above I quickly made a throwaway silhouette. I probably just drew a circle and some boxes to make the body and then used gaussian blur. I recommend you make a better one :) . In the example project, I used a blurry square. But you should be able to use anything!

    We'll put in a task for the team to make a more complete guide so be on the look out for that!

    As always don't forget to share your results!

    Cheers,

    Jon

    Comment actions Permalink
  • Thank you! Have a great day.

    Comment actions Permalink
  • Hey there!  I am following this exactly, but my background is an animated gif (with alpha) but its coming in blended into the background, not a purely opaque animation with transparency.  What am I doing wrong? 

     

    Comment actions Permalink
  • Hi Topher, Can you give a screenshot of what you mean? I grabbed an animated texture from the Giphy importer to the example project file from above and it seems to work fine?

    Comment actions Permalink
  • Here is the lens. https://www.snapchat.com/unlock/?type=SNAPCODE&uuid=a7dd3ff1415e4584992a0303730cef93&metadata=01 I went with a still image because I couldn't figure it out, but  it seems like the blending mode is not set to Normal, more of a multiply or screen effect.  

    Comment actions Permalink
  • Thanks for sharing Topher! Good use case.

    Can you give a screenshot of the issue you're describing?

    By the way, if you haven't, try out the particle template. It would be pretty cool to have ballon particles so that it's dynamic. But it's great either way :)

     

     

    Comment actions Permalink
  • I guess I didn't think of that... That might be better.  Can you set 3 different balloons as separate particles, and have those emit, or do you need 3 particle systems? I tried this with the flipbook technique in a previous lens (you helped me with the poker chips) but I had to duplicate the particle system and have 3 different ones to get the different colors.

    Does this question make sense? 

    Comment actions Permalink
  • The easiest way is to create three particle systems.

    If you want gradient of colors, you can use the Premultiplied Color (and setting Color min and Color max) option, to get colors within these ranges.

    Take a look at https://lensstudio.snapchat.com/guides/general/particles/ for more information.

    Let me know if you need clarification :D

     

    Comment actions Permalink
  • Does it also make sense to use an occluder to do this?

    Comment actions Permalink
  • Hey Andy,

    We actually just released a way easier way to recreate this segmentation effect. Download the latest update today here and tryout our Segmentation Template!

    Cheers,
    Kaitlyn

    Comment actions Permalink
  •  

    Jonathan Solichin

    every time i make a 3d lenses why i cant get gold star lens creator please see my lenses 

    this is link my ptofile every my loenses in my profile

    https://www.snapchat.com/add/xozi_yak

     

    Comment actions Permalink
Have a comment?

Please sign in to leave a comment.

We're here to help! We just need a little bit of information...

What system are you using?

Have you downloaded the latest version of Lens Studio?

Please download the latest version of Lens Studio. If you still run into this issue, please come back and report it!

Is this issue causing Lens Studio to crash?

What system do you run Lens Studio on?

Version

Graphics

Follow the next steps to help us solve your issue:

  • Copy and paste this text into your TerminalCommand Window
    open ~/Library/Preferences/Snap/Lens\ Studio/ %LOCALAPPDATA%\Snap\Lens Studio Copy Text
  • Press ReturnEnter to run the command. The Lens Studio folder will automatically open
  • Prepare to upload your files: zip the "Log" Folder by right-clicking and choosing "compress."
    Locate the Log.txt file right above it.

    Attach a screenshot of the issue:

Name:

Email:

What is this most relevant to?

Please enter a brief description of your issue:

Thanks for submitting this issue.

Unfortunately, it's likely due to the operating system or hardware you're using – since they don't meet the system requirements for Lens Studio.

Still, we hear you loud and clear, and are logging the issue in case there's ever a workaround we can provide!

Minimum Requirements

Operating System: Windows 10 (64 bit); MacOS 10.11+

Hardware: Minimum of Intel Core i3 2.5Ghz or AMD Phenom II 2.6Ghz with 4 GB RAM; Intel HD Graphics 4000 / Nvidia GeForce 710 / AMD Radeon HD 6450 or better; screen resolution of 1280x768 or higher

We'll try to resolve this issue as soon as possible. Thanks for letting us know about it!

Keep an eye out for a followup email from us. We may have a couple more questions for you, or we might already have a solution to offer.

Happy creating!