The forum on this site is closed for new comments and posts. Continue the conversation in our Snap AR Discord server.

string input on SnapML instead of array

Follow
Avatar
Marco Vanossi
Snap Lens Network Member Lens Studio Team

Hi,

 

 My model requires a string (byte array) as input. I tried everything to convert this model input to array but it's really hard. So my next approach would be to try SnapML scripting to convert the input texture to a string. Is that doable?

 

thanks

0
  • Avatar
    Eric Hu
    Snap Lens Network Member Lens Studio Team

    Hi Marco,

    One options is to use one-hot encoding for each letter you are trying to feed. However, unfortunately most text processing related features like LSTM and GRU are not supported for now. May I know what specific model you have?

    Eric

    Comment actions Permalink
  • Avatar
    Marco Vanossi
    Snap Lens Network Member Lens Studio Team

    Hi Eric,

     

     Thanks for the reply. I asked because I was trying to use AutoML Vision from Google Cloud to train a model but their input internally is string not float array. I tried really hard to convert to float but it's tough to understand what they're doing internally.

     

    thanks

    Comment actions Permalink
  • Avatar
    Marco Vanossi
    Snap Lens Network Member Lens Studio Team
    • Edited

    Here's the link to download the model if you would like to take a look. https://drive.google.com/file/d/1NK68P91Tujv336hMxNLx7g2E41luxw8y/view?usp=sharing

     

    I understand that some operations on AutoML  Vision also wouldn't be supported but for now the main issue is the input is string.

     I belive AutoML uses string input even for images to save space using quantization. Maybe SnapML could benefit from this in the future.

     

    thanks

    Comment actions Permalink
  • Avatar
    Eric Hu
    Snap Lens Network Member Lens Studio Team

    Hi Marco,

    Thanks for uploading the model. We will take a look and get back to you shortly.

    Eric

    Comment actions Permalink
  • Avatar
    Marco Vanossi
    Snap Lens Network Member Lens Studio Team
    • Edited

    Thanks, Eric.

     

    Google AutoML Vision Edge support would be very interesting since they export the model to be mobile optimized.

    Comment actions Permalink
  • Avatar
    Eric Hu
    Snap Lens Network Member Lens Studio Team

    Hi Marco, 

    I've looked into the model you exported from Google AutoML. Looks like it is reading image binaries and converts them to floats via tensorflow::ops::DecodeJpeg inside its graph. Unfortunately our internal inference framework does not support these ops. We will update our Tensorflow export guide to provide code example on how to transform graphs from AutoML for Lens Studio import.

    Eric

    Comment actions Permalink

We're here to help! We just need a little bit of information...

What system are you using?

Have you downloaded the latest version of Lens Studio?

Please download the latest version of Lens Studio. If you still run into this issue, please come back and report it!

Is this issue causing Lens Studio to crash?

What system do you run Lens Studio on?

Version

Graphics

Follow the next steps to help us solve your issue:

  • Copy and paste this text into your TerminalCommand Window
    open ~/Library/Preferences/Snap/Lens\ Studio/ %LOCALAPPDATA%\Snap\Lens Studio Copy Text
  • Press ReturnEnter to run the command. The Lens Studio folder will automatically open
  • Prepare to upload your files: zip the "Log" Folder by right-clicking and choosing "compress."
    Locate the Log.txt file right above it.

    Attach a screenshot of the issue:

Name:

Email:

What is this most relevant to?

Please enter a brief description of your issue:

Thanks for submitting this issue.

Unfortunately, it's likely due to the operating system or hardware you're using – since they don't meet the system requirements for Lens Studio.

Still, we hear you loud and clear, and are logging the issue in case there's ever a workaround we can provide!

Minimum Requirements

Operating System: Windows 10 (64 bit); MacOS 10.11+

Hardware: Minimum of Intel Core i3 2.5Ghz or AMD Phenom II 2.6Ghz with 4 GB RAM; Intel HD Graphics 4000 / Nvidia GeForce 710 / AMD Radeon HD 6450 or better; screen resolution of 1280x768 or higher

We'll try to resolve this issue as soon as possible. Thanks for letting us know about it!

Keep an eye out for a followup email from us. We may have a couple more questions for you, or we might already have a solution to offer.

Happy creating!

Back
Next Next Next Submit Issue Got It