I made a website that puts your face on your pet, using Cloud Vision and ML. The results are absurd as they are ridiculous

This page summarizes the projects mentioned and recommended in the original post on /r/webdev

Our great sponsors
  • SurveyJS - Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • cat-dataset

    Slightly improved cat-dataset for use in cat face landmark prediction models

    Have a go at petswitch.com if you wish... I made the original Petswitch almost ten years ago, and it's had mild success since then, including CNET writing an article about it and it receiving the prestigious honour of 'most useless website' in week 41 of 2018, as determined by theuselesswebindex.com. Aside from the obvious question of why I even made this, it was getting pretty creaky – I originally built it with PHP and ImageMagick, with the facial features being manually selected via jQuery UI. So I decided to rebuild the whole thing with a full face-to-pet ML pipeline, on static hosting. To get the human face features, the app renders the upload to a temporary img element. This is a handy way to orient the image correctly via the browser, and saves having to deal with EXIF data. It's then resized, rendered to a canvas element, converted to a base64 string, then sent via fetch to Google's Cloud Vision API, which returns landmark coordinates of the face. I use these coordinates to correct any tilt on the face, mask the eyes and mouth via a mask image, then store each masked element as an additional canvas. Detecting pet faces was trickier. Google, Amazon and Microsoft all offer object detection APIs via transfer learning, and the approach is largely the same: you supply a series of images with bounding boxes around the objects you want to detect, either added via a web interface or uploaded via their API. You train a model online from these supplied images, then the service will return the estimated coordinates of any detected objects in an uploaded image. I found a dataset of both cats and dogs that had been labelled with landmarks on their faces, then wrote a script to convert the landmarks into bounding boxes around their eyes and nose, the dimensions based on a simple formula around the distance between the eyes in each image. All in all it's been trained on about 17,000 images of cats and dogs, and the accuracy seems to be pretty good. I was pleased to discover it actually works pretty well on other pets too. I've also added some friendly pets to the Petswitch family for those that don't have a pet on hand. I decided not to use a framework for this, it's written from scratch using a series of ES6 modules – although I did use Konva to handle the manual selection of facial features if the API can't detect a face. I used ParcelJS as my task runner, and my detection APIs are hosted on Firebase Cloud Functions. Let me know if you have any questions, although I can offer no good explanation for why I created this monstrosity...

  • parcel

    The zero configuration build tool for the web. 📦🚀

    Have a go at petswitch.com if you wish... I made the original Petswitch almost ten years ago, and it's had mild success since then, including CNET writing an article about it and it receiving the prestigious honour of 'most useless website' in week 41 of 2018, as determined by theuselesswebindex.com. Aside from the obvious question of why I even made this, it was getting pretty creaky – I originally built it with PHP and ImageMagick, with the facial features being manually selected via jQuery UI. So I decided to rebuild the whole thing with a full face-to-pet ML pipeline, on static hosting. To get the human face features, the app renders the upload to a temporary img element. This is a handy way to orient the image correctly via the browser, and saves having to deal with EXIF data. It's then resized, rendered to a canvas element, converted to a base64 string, then sent via fetch to Google's Cloud Vision API, which returns landmark coordinates of the face. I use these coordinates to correct any tilt on the face, mask the eyes and mouth via a mask image, then store each masked element as an additional canvas. Detecting pet faces was trickier. Google, Amazon and Microsoft all offer object detection APIs via transfer learning, and the approach is largely the same: you supply a series of images with bounding boxes around the objects you want to detect, either added via a web interface or uploaded via their API. You train a model online from these supplied images, then the service will return the estimated coordinates of any detected objects in an uploaded image. I found a dataset of both cats and dogs that had been labelled with landmarks on their faces, then wrote a script to convert the landmarks into bounding boxes around their eyes and nose, the dimensions based on a simple formula around the distance between the eyes in each image. All in all it's been trained on about 17,000 images of cats and dogs, and the accuracy seems to be pretty good. I was pleased to discover it actually works pretty well on other pets too. I've also added some friendly pets to the Petswitch family for those that don't have a pet on hand. I decided not to use a framework for this, it's written from scratch using a series of ES6 modules – although I did use Konva to handle the manual selection of facial features if the API can't detect a face. I used ParcelJS as my task runner, and my detection APIs are hosted on Firebase Cloud Functions. Let me know if you have any questions, although I can offer no good explanation for why I created this monstrosity...

  • SurveyJS

    Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.

  • Konva

    Konva.js is an HTML5 Canvas JavaScript framework that extends the 2d context by enabling canvas interactivity for desktop and mobile applications.

    Have a go at petswitch.com if you wish... I made the original Petswitch almost ten years ago, and it's had mild success since then, including CNET writing an article about it and it receiving the prestigious honour of 'most useless website' in week 41 of 2018, as determined by theuselesswebindex.com. Aside from the obvious question of why I even made this, it was getting pretty creaky – I originally built it with PHP and ImageMagick, with the facial features being manually selected via jQuery UI. So I decided to rebuild the whole thing with a full face-to-pet ML pipeline, on static hosting. To get the human face features, the app renders the upload to a temporary img element. This is a handy way to orient the image correctly via the browser, and saves having to deal with EXIF data. It's then resized, rendered to a canvas element, converted to a base64 string, then sent via fetch to Google's Cloud Vision API, which returns landmark coordinates of the face. I use these coordinates to correct any tilt on the face, mask the eyes and mouth via a mask image, then store each masked element as an additional canvas. Detecting pet faces was trickier. Google, Amazon and Microsoft all offer object detection APIs via transfer learning, and the approach is largely the same: you supply a series of images with bounding boxes around the objects you want to detect, either added via a web interface or uploaded via their API. You train a model online from these supplied images, then the service will return the estimated coordinates of any detected objects in an uploaded image. I found a dataset of both cats and dogs that had been labelled with landmarks on their faces, then wrote a script to convert the landmarks into bounding boxes around their eyes and nose, the dimensions based on a simple formula around the distance between the eyes in each image. All in all it's been trained on about 17,000 images of cats and dogs, and the accuracy seems to be pretty good. I was pleased to discover it actually works pretty well on other pets too. I've also added some friendly pets to the Petswitch family for those that don't have a pet on hand. I decided not to use a framework for this, it's written from scratch using a series of ES6 modules – although I did use Konva to handle the manual selection of facial features if the API can't detect a face. I used ParcelJS as my task runner, and my detection APIs are hosted on Firebase Cloud Functions. Let me know if you have any questions, although I can offer no good explanation for why I created this monstrosity...

  • functions-samples

    Collection of sample apps showcasing popular use cases using Cloud Functions for Firebase

    Have a go at petswitch.com if you wish... I made the original Petswitch almost ten years ago, and it's had mild success since then, including CNET writing an article about it and it receiving the prestigious honour of 'most useless website' in week 41 of 2018, as determined by theuselesswebindex.com. Aside from the obvious question of why I even made this, it was getting pretty creaky – I originally built it with PHP and ImageMagick, with the facial features being manually selected via jQuery UI. So I decided to rebuild the whole thing with a full face-to-pet ML pipeline, on static hosting. To get the human face features, the app renders the upload to a temporary img element. This is a handy way to orient the image correctly via the browser, and saves having to deal with EXIF data. It's then resized, rendered to a canvas element, converted to a base64 string, then sent via fetch to Google's Cloud Vision API, which returns landmark coordinates of the face. I use these coordinates to correct any tilt on the face, mask the eyes and mouth via a mask image, then store each masked element as an additional canvas. Detecting pet faces was trickier. Google, Amazon and Microsoft all offer object detection APIs via transfer learning, and the approach is largely the same: you supply a series of images with bounding boxes around the objects you want to detect, either added via a web interface or uploaded via their API. You train a model online from these supplied images, then the service will return the estimated coordinates of any detected objects in an uploaded image. I found a dataset of both cats and dogs that had been labelled with landmarks on their faces, then wrote a script to convert the landmarks into bounding boxes around their eyes and nose, the dimensions based on a simple formula around the distance between the eyes in each image. All in all it's been trained on about 17,000 images of cats and dogs, and the accuracy seems to be pretty good. I was pleased to discover it actually works pretty well on other pets too. I've also added some friendly pets to the Petswitch family for those that don't have a pet on hand. I decided not to use a framework for this, it's written from scratch using a series of ES6 modules – although I did use Konva to handle the manual selection of facial features if the API can't detect a face. I used ParcelJS as my task runner, and my detection APIs are hosted on Firebase Cloud Functions. Let me know if you have any questions, although I can offer no good explanation for why I created this monstrosity...

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts