Erasing Things Using SnapML
Hey to whoever may see this, I have been trying to figure out the process as to how SnapML is used to erase something and replace it with a computer generated background in order to completely make it disappear from the frame. I understand that everyone says that the Style Transfer template should be used but I can't for the life of me figure out how or what Style Transfer has to do with it as I need a computer generated background based off what is in the current frame, not an image that then takes hours to use. An example of this would be the Eyebrow Erase template in the SnapML template section of the site and the bald effect in Lens Studio also uses this when creating a background for where the hair would normally be
I think its pretty fair to say that I'm missing something big here so any help would be appreciated!
To summarize, I'm asking how I can make parts of a body disappear from the frame completely.
Another question I have relating to this is how to use segmentation over things like the bald SnapML effect. An example of this would be if I wanted to use segmentation around the bald head effect and place it in front of an image. I am able to do this but can't figure out how to get the area where the hair used to be to not show as well.
Hi Max,
I believe the segmentation textures pull directly from the camera texture, so the hair will still be included in the segmentation textures.
With regards to painting things out, the training code may be based off the style transfer code, but it is not going to actually be style transfer. SnapML makes it relatively easy to import and use machine learning models, but you still need to go through model creation.
My advice is to read about different inpainting techniques, then before you try to train a final model, train it for one iteration and make sure you can import it into Lens Studio (both size-wise and compatibility). Not all layer types (the different parts of the machine learning models) are supported by Lens Studio.
Snap themselves have a whole body inpainting model, so at some point they might make it part of the Asset Library (I don't believe it currently is at the time I'm writing this).