The forum on this site is closed for new comments and posts. Continue the conversation in our Snap AR Discord server.

Machine Learning - Position and Rotation, Masking etc

  • Hi Casey Berman,

    Once you have a ONNX model ready, you can import it using MLComponent in Lens Studio and our internal converter will convert ONNX models to dnn models if all operations are supported on mobile inference.

    Eric

    Comment actions Permalink
  • Hi Eric,

    Thanks for getting back to me. I thought I responded, but it looks like I didn't...

    To give a little clarity, here is a link to the Wanaby presentation, if you're not already familiar: https://lensstudio.snapchat.com/templates/ml/ml-templates-library/

    My confusion is that there is no .onnx in the Wanaby example - only a .dnn. When I drag that dnn into the ML Component in Lens, I don't get any errors (it's always possible that Lens is prepared for someone to make a mistake like that, but it seems odd to me). I drag my .onnx into the ML Component, and I get an "ML Component can only set inference mode before model loading" error.This error seems to recur every minute or so. Also, even if I remove the .onnx from the ML Component (or delete it from the project), the project ceases to work - I must re-download the original project and start again.

    Do you have any insight into this? My model works on pictures of stop signs in the editor, so I don't *think* it's something wrong with the model, but maybe I'm wrong.

    Lastly, there does not seem to be a .onnx file in the Wanaby project's directories, and there is no .dnn in my dummy project's directories? I'm working in Lens 3.0 , I know there's a 3.1 out now - do you think that would have anything to do with it?

    I can send you the model if that would help troubleshoot.

    Thanks,

    Casey

    Comment actions Permalink
  • Hi Casey,

    Instead of directly drag your .onnx model into existing MLComponent, could you please try importing your model by clicking the + button under objects panel at top left and search for MLComponent, too? I tried using Lens Studio 3.1 but couldn't reproduce the error you reported.

    Thanks,

    Eric

    Comment actions Permalink
  • Hey Eric,

    Thanks again for getting back to me. Correct, that does not result in an error. However, that doesn't solve my issue - I want to attach an object to the detected object in real 3d space.

    Anyway, let me try rephrasing my question slightly. If I want to attach a physical object (say, a cube) to my ML Component's detected instance in 3 dimensional space, how would I do that?

    I was trying to use the working example from Wanaby, change the model and adjust from there - but maybe its aims are too specific and too different from mine. If I wanted to put a sign onto or below a detected stop sign, for example... can you point me in a direction to do that?

    Thanks,

    Casey

    Comment actions Permalink
  • Hi Eric,

    I saw in the documentation that attaching a 3D object to a model detection instance IS possible, but it seems only with the built-in models (people, hands and cats and dogs). Can I do the same with a trained ML model? Or, is this something that will be a feature coming soon to the platform? Lastly, am I missing something in the documentation?

    Thanks,
    Casey

    Comment actions Permalink

We're here to help! We just need a little bit of information...

What system are you using?

Have you downloaded the latest version of Lens Studio?

Please download the latest version of Lens Studio. If you still run into this issue, please come back and report it!

Is this issue causing Lens Studio to crash?

What system do you run Lens Studio on?

Version

Graphics

Follow the next steps to help us solve your issue:

  • Copy and paste this text into your TerminalCommand Window
    open ~/Library/Preferences/Snap/Lens\ Studio/ %LOCALAPPDATA%\Snap\Lens Studio Copy Text
  • Press ReturnEnter to run the command. The Lens Studio folder will automatically open
  • Prepare to upload your files: zip the "Log" Folder by right-clicking and choosing "compress."
    Locate the Log.txt file right above it.

    Attach a screenshot of the issue:

Name:

Email:

What is this most relevant to?

Please enter a brief description of your issue:

Thanks for submitting this issue.

Unfortunately, it's likely due to the operating system or hardware you're using – since they don't meet the system requirements for Lens Studio.

Still, we hear you loud and clear, and are logging the issue in case there's ever a workaround we can provide!

Minimum Requirements

Operating System: Windows 10 (64 bit); MacOS 10.11+

Hardware: Minimum of Intel Core i3 2.5Ghz or AMD Phenom II 2.6Ghz with 4 GB RAM; Intel HD Graphics 4000 / Nvidia GeForce 710 / AMD Radeon HD 6450 or better; screen resolution of 1280x768 or higher

We'll try to resolve this issue as soon as possible. Thanks for letting us know about it!

Keep an eye out for a followup email from us. We may have a couple more questions for you, or we might already have a solution to offer.

Happy creating!