lightningcss VS csv-parser

Would you consider using some libraries in your project? There are lots of good ones in the Rust ecosystem, and many of them are not part of any existing browsers.

For example:

- https://github.com/servo/html5ever (HTML parsing - note: this is used in Servo)

- https://github.com/parcel-bundler/lightningcss (CSS parsing)

- https://github.com/DioxusLabs/taffy (web layout)

- https://github.com/pop-os/cosmic-text (text layout and rendering)

Obviously you should be free to work on whatever you like, but just as a benchmark on the scope of your project: I spent ~6 months implementing just the CSS Grid algorithm in Taffy last year. An entire browser from literal scratch is probably a 10 year project for one person.

Libraries for a lot of this stuff exist (albeit in many cases not very mature yet):

- https://github.com/pop-os/cosmic-text does text layout (which Taffy explicitly considers out of scope)

- https://github.com/AccessKit/accesskit does accessibility

- https://github.com/servo/rust-cssparser does value-agnostic CSS parsing (it will parse the general syntax but leaves value parsing up to the user, meaning you can easily add support for whatever properties you what). Libraries like https://github.com/parcel-bundler/lightningcss implement parsing for the standard css properties.

- There are crates like https://github.com/BurntSushi/bstr and https://docs.rs/wtf8/latest/wtf8/ for working with non-unicode text

We are planning to add a C API to Taffy, but tbh I feel like C is not very good for this kind of modularised approach. You really want to be able to expose complex APIs with enforced type safety and this isn't possible with C.

Background: I encountered this problem using lightningcss.

You can do it - that's actually exactly what my project is doing. I have a single repository with a Rust project, that builds the .wasm file (+ .d.ts + .js) using wasm-pack, and a Node.js project, that uses this .wasm file. There's no problem in packing that and exposing as a npm package. See parcel-bundler/lightningcss for a full blown example (it's not using wasm-pack but builds the Rust project directly).

Initial commit, 9 Oct 2021. That is pretty new.

Inside the API route, convert the CSV file to JSON with csv-parser

https://github.com/mafintosh/csv-parser/pull/121 https://github.com/mafintosh/csv-parser/pull/151 https://github.com/mafintosh/csv-parser/issues/218

The author of the library probably has learned, the hard way many many lessons (and probably also decided to prioritize some of the requested issues / feature requests along the way).

The above is not meant as a ding on the project itself and I am sure it is used successfully by many people. The point here is that your claim that you can easily write a csv parser in 200 lines of code does not hold water. It's anything but easy and you should use a battle tested library and not reinvent the wheel.

If this were me, I'd probably add a getStaticProps function to the page, and then use the csv-parser library within this function to convert the CSV file into JSON data, and then return that JSON data from the getStaticProps function so the page can access it as props.

I started with using the external library csv-parser. However, since it is basically a wrapper around the base Node.js technologies I listed above I has the same problems working with my data that I will list below. I eventually uninstalled it and wrote my own light-weight version.

I have found this CSV parser package and it almost works exactly like I need it. I've ran into 2 issues.

I was using the [csv-parser](https://github.com/mafintosh/csv-parser) library to handle csv parsing in node. The file can be huge ranging from 50,000 to 500,000 lines, maybe even larger. I had to perform some computations on the csv, after this is submitted to the server for that I was thinking of dividing the csv into chunks which I could then provide to worker threads for performing the computation. The worker thread would get the number of lines to skip and then start reading the lines after that up to a particular limit. I create a read stream and pass the csv-parser in with the option of number of lines to skip. I tried to perform some benchmarks on it but could find no visible benefits between skipping lines and not skipping lines. Even if I read the whole file it was sometimes faster than reading the ending 30,000 lines.