csv-parser
BrowserBoxPro
Our great sponsors
csv-parser | BrowserBoxPro | |
---|---|---|
9 | 24 | |
1,399 | 2,602 | |
- | - | |
0.0 | 6.9 | |
3 months ago | 7 months ago | |
JavaScript | JavaScript | |
MIT License | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
csv-parser
-
Basic analytics with Vercel Postgres, Drizzle & Astro
Inside the API route, convert the CSV file to JSON with csv-parser
-
Why my favourite API is a zipfile on the European Central Bank's website
https://github.com/mafintosh/csv-parser/pull/121 https://github.com/mafintosh/csv-parser/pull/151 https://github.com/mafintosh/csv-parser/issues/218
The author of the library probably has learned, the hard way many many lessons (and probably also decided to prioritize some of the requested issues / feature requests along the way).
The above is not meant as a ding on the project itself and I am sure it is used successfully by many people. The point here is that your claim that you can easily write a csv parser in 200 lines of code does not hold water. It's anything but easy and you should use a battle tested library and not reinvent the wheel.
-
I have created a nextjs app which has multiple card column, i want to import data from csv file to this card. can somebody guide me? for reference i have attached some image what i want
If this were me, I'd probably add a getStaticProps function to the page, and then use the csv-parser library within this function to convert the CSV file into JSON data, and then return that JSON data from the getStaticProps function so the page can access it as props.
-
Node.js - Streams and Promises
I started with using the external library csv-parser. However, since it is basically a wrapper around the base Node.js technologies I listed above I has the same problems working with my data that I will list below. I eventually uninstalled it and wrote my own light-weight version.
-
Working with CSVs in Node.JS
I have found this CSV parser package and it almost works exactly like I need it. I've ran into 2 issues.
- Parsing large datasets into memory or database
-
Effecient way for creating partition in large file to feed in to a node worker thread
I was using the [csv-parser](https://github.com/mafintosh/csv-parser) library to handle csv parsing in node. The file can be huge ranging from 50,000 to 500,000 lines, maybe even larger. I had to perform some computations on the csv, after this is submitted to the server for that I was thinking of dividing the csv into chunks which I could then provide to worker threads for performing the computation. The worker thread would get the number of lines to skip and then start reading the lines after that up to a particular limit. I create a read stream and pass the csv-parser in with the option of number of lines to skip. I tried to perform some benchmarks on it but could find no visible benefits between skipping lines and not skipping lines. Even if I read the whole file it was sometimes faster than reading the ending 30,000 lines.
- Can this problem be fixed with JS?
BrowserBoxPro
-
Why my favourite API is a zipfile on the European Central Bank's website
I'm using this type of simple approach to build a SaaS right now. We need to spin up many VPS and provision them, and the fastest way to do that is with rsync and ssh.
But we didn't stop there: this SaaS for our open source browser product is entirely built like this^0: behind the scenes it's a collection of bash scripts that implement and execute the business operation of the SaaS.
So basically, it's a command-line interface to the SaaS. Think of it this way, say I didn't have a website, with login, and "click a button to open a browser", but instead people would write me letters, send me cheques, or call me on the phone. Then I can serve their requests manually, at the command line.
The reason I made it like this was:
- clear separation between thin web front-end and actual business logic
- nice command-line interface (options, usage, help, clear error messages) to business logic for maintenance and support to jump on and fix things
- inheritance of operating system permissions and user process isolation
- highly testable implementation
Maybe this is dumb, but I really like it. To me it's an architecture and approach that makes sense.
I'm sure this is not new, and I think a lot of good quality operations must be built via this way. I highly align with the author's stance of the composition of a few simple command line tools to get the job done.
Perhaps we can call this "unix driven development", or "unix-philosophy backend engineering"
0: https://github.com/dosyago/BrowserBoxPro (saas coming soonish)
-
PF bug in macOS Sonoma release candidate
If you're concerned about these kinds of bugs on your local OS platform you may consider "abstracting away" your local connection point via a remote browser. This way, whatever your local machine and OS, you can have a dedicated server that you run your browsing through. Granted it doesn't enclose your entire network connection: only your browsing, but what it does there is change your IP address, mask your location, and add protection from browser 0 days.
We're constantly adding new features add BrowserBox to respect and protect privacy and improve the overall experience. It's open source so you can change it how you want too. If you don't like AGPL-3.0 you can get a commercial license. Come take us for a spin: https://github.com/dosyago/BrowserBoxPro
If you don't want something open source, but prefer the joy of a large company I think Mullvad also has their Mullvad Browser which does something similar!
-
CVE-2023-4863: Heap buffer overflow in WebP (Chrome)
Agree. This is one of the reasons it's better to go with older and more reliable JPEG for viewport streaming. An exploit chain would need to penetrate screen capture images to pass to the client. Browser zero days do occur and this is why it's important to have protection. For added protection consider browser isolation. Check out open source Zero Trust browser isolation at BrowserBox using JPEG (now WebP) now: https://github.com/dosyago/BrowserBoxPro
Technically, we did try using WebP due to its significant bandwidth gains. However, the compute overhead for encoding versus JPEG introduced unacceptable latency into our streaming pipeline, so for now, we're still against it. Security is an additional mark against the newer standard, as good as it is!
-
An Ode to Renice
Notes
renice: https://stackdiary.com/linux-docs/renice/
audio server code: https://github.com/dosyago/BrowserBoxPro/blob/boss/src/servi...
audio client code: https://github.com/dosyago/BrowserBoxPro/blob/boss/src/publi...
-------
FAQ
What real time prio did you renice to? We tried a few around -16 but settled on -15.
Did you try WebRTC for the audio? Yes, we tried the channel for streaming chunks, but in this case WS was more reliable and faster (I guess it was because we were producing small chunks at a consistent rate). I'm interested in exploring WebRTC audio channel streaming if anyone knows, come and contribute: https://github.com/dosyago/BrowserBoxPro or get in touch at [email protected]
Why are you using WAV not MP3 for audio? Because we're chunking it. I didn't know how you can slice MP3 into tiny pieces for custom ACK-based streaming like we are doing, and I'm not even sure if it's possible: when I tried the MP3 became corrupted, but WAV worked fine, (I guess because it's linear not a compressed self-referential format like MP3). In tests the reduction in bandwidth due to MP3, was somewhat lost to the increase in latency / compute on the server to encode it. If anyone knows a better way to stream audio (or MP3) in this case come contribute: https://github.com/dosyago/BrowserBoxPro or get in touch at [email protected]
- Two GitHub bots not talking to each other on a PR
-
Show HN: BrowserBoxPro on GitHub Container Registry
People have requested this for years. I finally got around to it. This is a WIP but tested on MacOS and Linux it worked.
Pull the image and follow the run instructions: https://github.com/dosyago/BrowserBoxPro/pkgs/container/brow...
-
Show HN: I run BrowserBoxPro in a GitHub Action using ngrok
How to Use GitHub Actions and ngrok to Test a Remote Browser, BrowserBoxPro
It's worth noting the following method will work for any web application, it's just surprising to me that it worked with one which uses WebSockets, WebRTC and has a significant back-end component!
I've recently made an exciting breakthrough with my project, BrowserBox, a remote browser application that supports WebSockets and WebRTC. Amazingly, I was able to run and test it successfully in the context of a GitHub Action, even browsing the web as normal! If you're curious about running your own remote browser for testing purposes, here's how you can do it too:
How can you do this yourself?
1. Fork the repo
https://github.com/dosyago/BrowserBoxPro/fork
2. Add your ngrok auth token to your fork's repository secrets under NGROK_AUTH_TOKEN (you need to sign up for an ngrok free account if you don't have one)
3. Go to your fork's Actions page and run the CI action.
4. Wait a couple minutes for the setup to run and click on the URL produced by the "Print ngrok URL" step.
5. Play around with the remote browser! Just click on the big + to create a new tab and enter a search query or an address in the address bar and you're away!
IMPORTANT! I'm not sure if this violates the GitHub terms doing this (it may do! Any GitHub employees please email me at [email protected] and I will remove this Action if it does!), but it's logical that just using this to browse the web would be wasting resources from the Actions runners intended purpose, so don't overdo it! To try to help with this I've set the Action to only run the browser for 5 minutes.
If anyone wants to port this to GitLab or another CI platform, we would very much welcome your contribution!
Anyway, I was really happy and surprised to discover that we can use the generous free compute from Microsoft and GitHub, and the free tunnel from ngrok, to really do some useful things, and you can check up on those and integration test using ngrok and GitHub Actions!
-
Web Environment Integrity API
Many comments on here are about this protecting the ad business model, but I think it's actually about protecting against competitor browsers.
If official Google Chrome is the only browser that passes this attestation proposal, you can effectively own the market and prevent new competitors.^0
I'm sure the technical specifics will be slithered over with enough worm-tongue to make it vague and innocent enough to not trigger anti-competitive lawsuits or whatever, but that might still be an option.
From a legal view, isn't this worse than microsoft force bundling IE into its OS? Taken to its full realization, it seems attestation is Alphabet force bundling Chrome into the whole web (that ubiquitous, "global OS", used my almost everyone). It's not there yet, but is it impossible to go from current zero to very-scary one?
As a maker of a competing browser technology (that uses Chrome under the hood), I'm worried about this, but heartened by the fact that as we are also an open-source product, the solution (that I've talked about elsewhere in this thread), if it's possible, will be distributed and built by people.
It's plain that Alphabet faces a conundrum: how do they prevent their investment in the open-source product being used against them? How do they prevent competitors (like brave, and BrowserBox) benefiting from the code Alphabet pays its employees to write, essentially using Alphabet's money to gradually chip away at (or threaten chipping away at) Alphabet's Chrome market share.
I understand the paradox they face, but I don't think "DRM" level control of a global and ubiquitous "means of access" is the way to solve it. But as owner of an open-source company myself, I don't think the solution is one where Google can't capture any value from what they invest in creating.
In terms of the long term economics, I don't have a solution. But I don't think that matters. I think technically there will be solutions to this, and they'll be built in the open.
DOSYAGO is really not an activist company, nor do we seek to be. But some things are worth standing up for. Future of the web should be one, I think. If you'd like to get involved, come on over to BrowserBox and contribute!
https://github.com/dosyago/BrowserBoxPro
0: With a current "monopoly", these new competitors may seem theoretical, but I think internally their viewed as very real threats over the long term. Brave, etc. And the fact that anyone can use Chromium to build a new browser.
- Use chrome remotely
- BrowserBox Pro goes open-source
What are some alternatives?
xlsx-populate - Excel XLSX parser/generator written in JavaScript with Node.js and browser support, jQuery/d3-style method chaining, encryption, and a focus on keeping existing workbook features and styles in tact.
BrowserBox - 🌀 Browse the web from a browser you run on a server, rather than on your local device. Lightweight virtual browser. For security, privacy and more! By https://github.com/dosyago
PEG.js - PEG.js: Parser generator for JavaScript
neko - A self hosted virtual browser that runs in docker and uses WebRTC.
parse-json - Parse JSON with more helpful errors
Isotope - :revolving_hearts: Filter & sort magical layouts
google-libphonenumber - The up-to-date and reliable Google's libphonenumber package for node.js.
nativefier - Make any web page a desktop application
neat-csv - Fast CSV parser
BrowserBox - 📷 BrowserBoxPro - The internet. But unrestricted. And secure. Remote browser isolation product, available here and in Pro for purchase on our website. [Moved to: https://github.com/dosyago/BrowserBoxPro]
Jison - Bison in JavaScript.
encrypted-media - Encrypted Media Extensions