[Guide] Style Transfer ML
Using the template that Snapchat provides, there are a few variables you can change when using the notebook. Thinking of the style transfer, you have the notebook, the style image, and the test image. As far as I know, changing the test image doesn't do anything, but the style image plays a huge factor in the outcome of your lens. Mainly Style Transfer is a long trial and error process because it takes hours to see the result. Here is what I know so far about Style Transfer ML:
Style Image: Getting an image from the internet or even taking one, holds information. When you bring that into a photo editing application, you can play around with the colors, brightness, shadows, and contrast. It's best to use these tools to your advantage before you plug your style image into the notebook. Usually, I brighten the shadows and lower the highlights a tad to make the image flat, but not too flat that the computer can't differentiate the objects in the image. After I'm done editing the photo, I export it as a PNG rather than saving, even if it says that it is already one online, just to double-check. You don't want to go through hours of waiting on your ONNX file, and it has an error right before export.
Training Steps: Training Steps can be found in the "Global Variables For Training" section- a quick way to find that section, is to go to the left panel right under the CO to find the Table of Contents- in light green numbers you'll see "10000" which is the default amount of steps. Using this, it typically takes 2.5hours more or less. Changing this number will make the duration of the training loop extend past 2.5 hours, but it makes the model look way better and more exact. It looks better because the longer it trains, the more time the computer has to recognize the details in the image. I'm not too sure, but I think the model caps at a certain point. In my 100k training steps one, I noticed that the loss kept on going back and forth and stopped reducing after ~70k. I typically do 40k-60k steps for most of my models. For reference, my 100k steps took 19hours to complete and failed once because my computer turned off ~17hours the first time.
- Training Steps Underlined In Red:
Weight Decay: I heard of this factor, but I haven't tried it out yet. So far, I know changing the weight decay marginally changes the output.
- Weight Decay Underlined In Red:
Knowing How Long It Takes Till It's Done: Underlined in red in the image below, is where it shows the estimated time left till your ONNX file should be completed and ready to download. Where it says " [03:36<07:10] ", usually the time leans more toward the second group of numbers, so "7:10" would mean 7 minutes and 10 seconds. Keep in mind that this is a Style Transfer run at 1k steps for example purposes, not 10k so it's going to finish sooner.
More Steps = More Exact Image = More Time Spend Training.
How To Download: When your Style Transfer Training is done, an ONNX file will pop up in your folder. Hover over that, click the three dots, hit download, and it'll save to your computer and be ready to import into your Lens Studio.
Adding your ONNX file in Lens Studio: When you're done downloading your ONNX file from the notebook, you can drag and drop it into the resources panel. From there, you click the "ML Component" in the Objects panel then drag your ONNX file into the Inspector panel in the section labeled "Model".
Blend Modes: With the Style Transfer, you aren't limited to how it looks when putting in the model straight away, there are different blend modes you can use to your advantage to get the most out of your ML and the many hours put into it! Going into the Objects Panel, under "Style Transfer Output" there's a tab underneath it called "Output Image". Clicking the Output Image and going into the Inspector panel, under images is the Blend Modes where you can choose from various blend modes to see what looks good to you. You may need to reset the lens in the Preview to see how it looks after changing it.
Things to keep in mind: You can't change the training steps when it's already started, you'll have to factory reset the runtime. Error messages tend to pop up a lot, especially when using the regular version of Google Collab. It's easier to make a new google account as a fresh slate for the ML, I ran into this problem in the beginning. I made a few accounts, one of which I got Collab Pro. On the Collab Pro account, things do run smoother and faster, and I've noticed fewer error messages; it is $10, but it's only a smart investment if you don't want to make sure the runtime doesn't go idle and delete your work. I keep my style images in separate folders in different drives so I can run multiple images at once and see which one I like most to maximize my time.
Some of my Style Transfer ML Lenses:
I hope this helped and will save you time later on! :)
This is amazing i came from ur story one easy tip if u get this problem cause i always did is to change ur batch number mine works on 11 bur u can always try. Thanks tho this helped alot.
no matter what settings i change the output onnx file is 53kb, why is that and where is that setting?
also, how do u get the auto download to work so i can leave it on while im at work ?
Hey Calvin, no matter what you do the output ONNX file will always be 53kb.
For auto-download, I'm not too sure about that, I just keep my computer on and look at the time it estimates it'll be done from when I start the training loop. You can search on stack overflow ( https://stackoverflow.com/ ) for an answer. If I find anything refer back to this and comment.
Thanks for the guide man, these new features are awesome. Congrats on getting the OLC title.
Looks great, thanks Pam
Thanks vape juice
Thanks for sharing, it's really helpful.
this is really amazing information. It helped me a lot,
Thanks for sharing this helpful information.
you will get some help from The Vlogging Tech about your question about Style Transfer ML
Thanks for sharing, it's really helpful. i will use it for lg-cordzero-vs-dyson-v11
I often say that no matter what settings i change the output onnx file is 53kb, why is that and where is that setting? And visit here to make this difference.