Ask HN: How to handle user file uploads?

1. Uppy

   1 31 29,567 9.6 TypeScript

   The next open source file uploader for web browsers :dog:

   

   Look at https://uppy.io/ open source and lot of integrations. You can keep moving to different levels of abstraction as required and see some good practices of how things are done.

2. Nutrient

   www.nutrient.io featured

   Nutrient – The #1 PDF SDK Library, trusted by 10K+ developers. Other PDF SDKs promise a lot - then break. Laggy scrolling, poor mobile UX, tons of bugs, and lack of support cost you endless frustrations. Nutrient’s SDK handles billion-page workloads - so you don’t have to debug PDFs. Used by ~1 billion end users in more than 150 different countries.

   

3. magika

   2 7 8,471 9.7 Rust

   Detect file content types with deep learning

   

4. Smithereen

   3 9 424 9.7 Java

   Federated, ActivityPub-compatible social network server with friends, walls, and groups.

   

   In my project[1], I convert all user-uploaded images to high-quality webp and store them like that. I discard the original files after the conversion. I use imgproxy[2] to further resize and convert them on the fly for actual display.

   I don't do videos yet, but I'm kinda terrified of the idea of putting user-uploaded files through ffmpeg if/when I'll support them.

   [1] https://github.com/grishka/Smithereen

   [2] https://github.com/imgproxy/imgproxy

5. imgproxy

   4 32 9,374 9.2 Go

   Fast and secure standalone server for resizing and converting remote images

   

   In my project[1], I convert all user-uploaded images to high-quality webp and store them like that. I discard the original files after the conversion. I use imgproxy[2] to further resize and convert them on the fly for actual display.

   I don't do videos yet, but I'm kinda terrified of the idea of putting user-uploaded files through ffmpeg if/when I'll support them.

   [1] https://github.com/grishka/Smithereen

   [2] https://github.com/imgproxy/imgproxy

6. libvips

   5 26 10,123 9.4 C

   A fast image processing library with low memory needs.

   

   Read through the comments and was surprised no one mentioned libvips - https://github.com/libvips/libvips. At my current small company we were trying to allow image uploads and started with imagemagick but certain images took too long to process and we were looking for faster alternatives. It's a great tool with minimum overhead. For video thumbnails, we use ffmpeg which is really heavy. We off-load video thumbnail generation to a queue. We've had great luck with these tools.

7. imagor

   6 56 3,585 7.0 Go

   Fast, secure image processing server and Go library, using libvips

   

   I have used https://github.com/cshum/imagor infront of S3 before and liked it, there is many (some commercial) offerings for this

8. tus-js-client

   7 3 2,270 8.3 JavaScript

   A pure JavaScript client for the tus resumable upload protocol

   

   We map the TUS[0] protocol to S3 multipart upload operations. This lets us obscure the S3 bucket from the client. The TUS operations are handled by a dedicated micro-service. It could be done in a Lambda or anything.

   Once the upload completes we kick off a workflow to virus scan, unzip, decrypt, and process the file depending on what it is.

   For virus scanning, we started with ClamAV[1], but eventually bought a Trend Micro product[2] for reasons that probably don't apply to you. It is serverless based on SQS, Lambda, and SNS. Works fine.

   Once scanned, we do a number of things. For images that you are going to serve back out, you for sure want to re-encode those and strip metadata. I haven't worked directly on that part.

   [0] https://tus.io/

9. CodeRabbit

   coderabbit.ai featured

   CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

   

10. SwiftVips

    8 1 5 2.3 Swift

    A lightning fast image processing and resizing library for Swift

    

    +1 to vips! It's amazingly fast and stable. I even wrote (some minimal) Swift bindings for it to be used with a Swift backend: https://github.com/gh123man/SwiftVips

11. photo_id_resizer

    9 7 13 2.6 Go

    Resize photo ID images using face recognition technology

    

    Slight OT...

    I created a program for profile pictures. It uses face recognition technology as to not deform faces when resizing photos. This may be useful to you.

    https://github.com/jftuga/photo_id_resizer

12. meson

    10 112 5,900 9.8 Python

    The Meson Build System

    

Suggest a related project