Photorealistic Monocular 3D Reconstruction of Humans Wearing Clothing

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
  • duckduckgo-locales

    Translation files for <a href="https://duckduckgo.com"> </a>

    You should ask Putin about wearing heels. He is extremely self conscious about his height and has shoe implants to make him look taller.

    Did you know he is somewhere between 5’1” (155 cm) and 5’3” (160 cm)?

    Anyways, there are pics online of his high heels that he wears with his business suit.

    https://duckduckgo.com/?q=putin+height&t=fpas&iax=images&ia=...

    Here is a pic of him with Serbian president Aleksandar Vučić who is 6’6” (198 cm). Vučić towers over Putin.

    http://static.kremlin.ru/media/events/photos/big2x/YtX08J6xA...

  • Meshroom

    3D Reconstruction Software

    I have been recently looking at several "3d scanning" from monocular images/videos solutions and although they make for impressive demos I'm afraid they are a few years away from robust results.

    On one hand you have traditional photogrammetry based on classical image descriptors (from the beginning of the 2000s). A nice open source solution is Meshroom [1]. I have very little experience as I have just tinkered a little but I would say they work OK with geometry of medium complexity and detailed textures. They fail horribly with untextured objects, like for example a candle. They are semi-automatic: you upload some photos and run the pipepiline but for best results it's easy to tweak as there are a lot of knobs to try.

    On the other hand you have these deep learning research papers. A lot of them have Colab notebooks you can try yourself (which is awesome, thank you). They can give you these very nice demos for some kind of objects (the ones we have a lot of training data, like human bodies), but they are not truly general and sometimes will generate artifacts. There is some opportunity here if they are integrated in a semi-supervised workflow, like this [2].

    Anyway, just curious and not an expert.

    [1] https://alicevision.org/

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  • MotioNet

    A deep neural network that directly reconstructs the motion of a 3D human skeleton from monocular video [ToG 2020]

    They could be a great application for niches, though. Prototyping 3d games or movies can be helped greatly by such techniques.

    If you can take a photo (effort required: seconds) and turn it into a textures low- to mid-poly model within seconds, that's a huge time saver right there.

    Combined with other techniques [1], [2] this could indeed present a viable business model - again for certain niches. It's the same old routine: identify potential, present solution, profit (if only for a limited time). On its own this might not look too useful/impressive, but I'm certain there's people out there who could already benefit greatly from this work.

    It's not always about "the next big thing" or "turning an entire industry on its head". I could see this work its way into various products without much fanfare and gazillion dollar valuations.

    [1] https://github.com/Shimingyi/MotioNet

    [2] https://deepai.org/publication/end-to-end-learning-for-3d-fa...

  • awesome-virtual-try-on

    A curated list of awesome research papers, projects, code, dataset, workshops etc. related to virtual try-on.

    It's called virtual try on, there are lot of research papers about it:

    https://github.com/minar09/awesome-virtual-try-on#Image-base...

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts