Our great sponsors
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
Since you've got lots of images, you could use GNU Parallel to spread the job across multiple CPUs.
IO speed is often a limiting factor, particularly with many small files. It might help to store the images in a zip file to avoid the overhead of opening and closing many files. It also improves throughput because in a zip file the small files will be stored contiguously while as regular files they'll each take up at least one disk block. When you store them in the zip file, disable compression because JPEGs are already compressed. Python has zip file access built-in, and in C you can use libzip or libarchive.