string input on SnapML instead of array

Follow

Marco Vanossi

Snap Lens Network Member Lens Studio Team

• June 13, 2020 04:42

Hi,

 

 My model requires a string (byte array) as input. I tried everything to convert this model input to array but it's really hard. So my next approach would be to try SnapML scripting to convert the input texture to a string. Is that doable?

 

thanks

0

• 

  Eric Hu

  Snap Lens Network Member Lens Studio Team

  • June 15, 2020 06:21

  Hi Marco,

  One options is to use one-hot encoding for each letter you are trying to feed. However, unfortunately most text processing related features like LSTM and GRU are not supported for now. May I know what specific model you have?

  Eric

  Comment actions Permalink
• 

  Marco Vanossi

  Snap Lens Network Member Lens Studio Team

  • June 15, 2020 06:24

  Hi Eric,

   

   Thanks for the reply. I asked because I was trying to use AutoML Vision from Google Cloud to train a model but their input internally is string not float array. I tried really hard to convert to float but it's tough to understand what they're doing internally.

   

  thanks

  Comment actions Permalink
• 

  Marco Vanossi

  Snap Lens Network Member Lens Studio Team

  • June 15, 2020 09:07
  • Edited

  Here's the link to download the model if you would like to take a look. https://drive.google.com/file/d/1NK68P91Tujv336hMxNLx7g2E41luxw8y/view?usp=sharing

   

  I understand that some operations on AutoML  Vision also wouldn't be supported but for now the main issue is the input is string.

   I belive AutoML uses string input even for images to save space using quantization. Maybe SnapML could benefit from this in the future.

   

  thanks

  Comment actions Permalink
• 

  Eric Hu

  Snap Lens Network Member Lens Studio Team

  • June 15, 2020 17:50

  Hi Marco,

  Thanks for uploading the model. We will take a look and get back to you shortly.

  Eric

  Comment actions Permalink
• 

  Marco Vanossi

  Snap Lens Network Member Lens Studio Team

  • June 15, 2020 18:52
  • Edited

  Thanks, Eric.

   

  Google AutoML Vision Edge support would be very interesting since they export the model to be mobile optimized.

  Comment actions Permalink
• 

  Eric Hu

  Snap Lens Network Member Lens Studio Team

  • June 16, 2020 18:44

  Hi Marco, 

  I've looked into the model you exported from Google AutoML. Looks like it is reading image binaries and converts them to floats via tensorflow::ops::DecodeJpeg inside its graph. Unfortunately our internal inference framework does not support these ops. We will update our Tensorflow export guide to provide code example on how to transform graphs from AutoML for Lens Studio import.

  Eric

  Comment actions Permalink

Please sign in to leave a comment.