react-native-vision-camera VS tfjs-models

Camera and Image Captioning: For taking pictures we will use react-native-vision-camera and for generating captions from the images, react-native-quick-tflite.

> One of the big "aha" moments I had was a couple years ago when I was trying out Stripe's Identity product, which lets you take a selfie and a picture of your ID for identity verification purposes. It's pretty amazing how Stripe was able to do the full image analysis (i.e. edge detection when you take a picture of your ID) solely with browser APIs.

Do you have any details on that Stripe app? I've found the browser camera handling lacking on mobile if you want to overlay it with a "position the subject here" rectangle, or display instructions alongside, or detect QR codes as well as other subjects. I'd like this to be an out-of-date view. I've been considering React Native for the next version, using https://github.com/mrousavy/react-native-vision-camera as it seems much more feature complete.

You can use react-native-vision-camera and a frame processor that uses some kind of OCR to get the number plates.

This is a demonstration of what VisionCamera V3 is capable of - I built a small plugin to load any TensorFlow Model (.tflite) from resources and run it on every Camera Frame. This is going to be a game-changer, since the developer no longer needs to write native code for such complex tasks - it only takes a few lines of JavaScript to spin up a Camera, do real-time processing with a TensorFlow model, and draw anything to the Frame. Magic!

it's https://github.com/mrousavy/react-native-vision-camera

I finally got react-native-vision-camera and vision-camera-face-detector working yesterday by using Windows Subsystem for Linux (WSL). On windows itself it just doesn't work for me (https://github.com/mrousavy/react-native-vision-camera/issues/1474)

I'm the author of react-native-vision-camera, a library that exposes platform native Camera features to React Native.

Here’s an example from react-native-vision https://github.com/mrousavy/react-native-vision-camera/blob/main/example/src/views/StatusBarBlurBackground.tsx

6. Vision Camera

I am wondering, will this also work with tensoflow.js or only tflite? I'd like to use this hand pose estimation from mediapipe: https://github.com/tensorflow/tfjs-models/tree/master/hand-pose-detection

I got the basic concept working using TensorFlow QnA model but the answers don't seem very accurate. Infact, unless you specifically ask the exact question, you dont get the right answer. Its not intelligent enough because the entire PDF content becomes a bag of words instead of having context to those words. for eg. When someone types "languages", it should search within a section named Languages.

I think you are beginning to connect the dots by now 😉 What we will do is to build a small proof-of-concept (POC) by writing a simple react app and hook up a pre-trained tensorflow.js model, The text toxicity model to "moderate" the user's text input and show a notification of what's wrong with it, a text toxicity meter if you will...

TensorFlow.js is a JavaScript library developers can use to run pre-trained machine-learning models in the browser. The library has a variety of models for tasks such as object identification and language processing. One of these models is the text toxicity detection model.

TensorFlow has a toxicity classification model available. I don't find it to work that well, but it does something. Not sure if it works in other languages than English. Tried the demo with some offensive Dutch and it passed it.

No clue if it will work on drawn images. It depends on what the model was trained on. The models github is a good place to get started on figuring that out: https://github.com/tensorflow/tfjs-models/tree/body-pix-v2.0.4/body-pix

You probably want to check out something like tensorflow https://github.com/tensorflow/tfjs-models where you can build and test your models.

I decided to start with the toxicity model OOTB demo code for simplicity. You can find this code in their Github repo.