ESP32-W5100 VS w5100_esp32

Out of curiosity, are you using KaeLL's ESP32-W5100 component or something else? I'm currently using Arduino as a IDF component with a modified version of the Arduino W5500 library and getting decent results, but would love to strip away some of the overhead.

Since you're using the ESP32, here's an idea: see if you can optimize your drivers. It's an exploitative approach to this rather than explorative, but it requires you to increase the depth of your knowledge and you can put it to use right away, instead of acquiring shallow knowledge about multiple designs. I have an ESP32 based custom design with a Ethernet and a NFC chip. The NFC driver is largely just the manufacturer's plugged into the project, since rewriting it requires ISO14443 knowledge, making it quite a herculean task for a solo, and the Eth was almost completely rewritten due to it being really simple. There is still room for optimization given how the ESP32 differs from the MCUs the original drivers were meant to be used with. For instance, the Ethernet speed in my design is currently limited by the way SPI transactions are performed. My point is two-fold: you don't really have to rewrite everything insofar as you truly understand it; if you want to learn this, get out of your comfort zone;