Ask HN: Is it hard to get into systems programming from back end engineer?

• arxiv-sanity-lite

  17 1,082 0.0 Python

  arxiv-sanity lite: tag arxiv papers of interest get recommendations of similar papers in a nice UI using SVMs over tfidf feature vectors based on paper abstracts.

Yes! But it depends what you want to get out of it. I'm a big proponent of project-based learning, so my question to you is: what would you want to build?

I'll lead with an example: I really like the idea of search indexes and search engines. I cobbled together a search engine on my own that indexes arXiv's metadata, very similar to Andrej Karpathy's Arxiv Sanity Preserver [1] in Python. I very quickly encountered that the "What can I rip off of medium blogs and StackOverflow" approach wasn't scaling well, so I had to dig into how each part of my tech stack worked to figure out where things were going slower, and how I could improve it.

You can "do systems engineering" in any language and just about any tech stack, but you'll often see it closely associated with compiled languages, because those are often chosen when we care about performance because extra abstractions often make it harder to reason about what's really going on in certain portions of your tech stack. It started in Python, but then dusted off my C skills, and then my C++ skills for different parts of the entire stack.

Before I get too carried away, here's the main point:

Try to find a problem where you have to deal with lots of data on a single computer/server/device. You'll quickly hit limitations on what your software can do. Figuring out why you hit those limits is gonna start you down a path to acquiring the "systems knowledge" that you seek.

It's actually quite difficult to hit performance limitations on your average server with just text. Video and 3d content are much more intensive mediums that very quickly let you hit those limits. It's one of the reasons why Twitch and YouTube don't have lots of competition, but we've hopped between link aggregator to link aggregator over the years.

1: https://arxiv-sanity-lite.com/

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project