db-benchmark
learning-notes
db-benchmark | learning-notes | |
---|---|---|
12 | 6 | |
124 | 4,889 | |
6.5% | - | |
8.0 | 0.0 | |
4 months ago | 4 months ago | |
R | SCSS | |
Mozilla Public License 2.0 | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
db-benchmark
- Database-Like Ops Benchmark
-
Polars
DuckDB maintains a benchmark of open source database-like tools, including Polars and Pandas
https://duckdblabs.github.io/db-benchmark/
- Planning a New Benchmarking for Comparing Filter2Groupby for 3,000 Files (100,000 Rows/Files)
- Pandas vs. Julia – cheat sheet and comparison
-
Polars supports SQL statement in Python Plus CLI Verion (Polars.exe 24.4MB)
DuckDB is also a SQL/Python app, refer to this benchmark, seem it run very fast https://duckdblabs.github.io/db-benchmark/
- The Return of the H2o.ai Database-Like Ops Benchmark
- I discovered that the fastest way to create a Pandas DataFrame from a CSV file is to actually use Polars
learning-notes
-
What are your thoughts on DDIA?
I only have 2 years of experience but DDIA along with Alex Xu's book have been extremely helpful to me. Here's some other links worth checking out: https://github.com/keyvanakbary/learning-notes/blob/master/books/designing-data-intensive-applications.md ; https://comeshare.net/category/study/system-design/page/2/
-
I discovered that the fastest way to create a Pandas DataFrame from a CSV file is to actually use Polars
Wow defensive. Do what works for you, but there are general approaches that people use. This is a good reference text: https://github.com/keyvanakbary/learning-notes/blob/master/books/designing-data-intensive-applications.md
-
Could you please recommend your favorite book in the cybersecurity field?
I also read (listened) to this recently and I really liked it! Wish I'd kept notes on some things but then I found an in depth writeup in a gitrepo, SPOILERS so use after reading the book.
- Any good resources to get started with testing?
-
My job search experience with 2 YoE as a backend software engineer
For systems design, I read through the Designing Data Intensive Applications Summary. Since I already worked with highly distributed and data-intensive systems at work, this was a lot easier for me to digest than I thought. I watched a few systems designs interviews on YouTube, and practiced with a few friends. The cheat codes here are: autoscaling, loadbalancing, trading consistency for consensus, and caches. Learn them and learn them well.
-
Ask HN: What was your experience like moving from an IC to a manager role?
I've been going through this transition for the past year or so. And it started out with it being more stressful (e.g., on COVID walks with my wife she noted more days than not I'd be complaining and thinking it best to go back to being an IC).
But the more I've done it, the more I really enjoy it. It's sort of like having kids (I imagine). I.e., it's sort of insane and counterproductive, but ultimately very satisfying. And as someone else mentioned it's a two-way door (if your engineering interests are long-term).
I've read probably 20-30 engineering management books in the past year and started our 'manager learning hub' wiki at work. There's a few good books I'll link to below.
But yeah you will likely have a new peer group of other managers. So you now have your team and your manager peer group. So if you find yourself not learning as much as an IC before (I'd been one more or less for 16 years before starting to transition), it can be a fun new area to learn.
https://www.youtube.com/watch?v=tTik4nt1knY
https://github.com/keyvanakbary/learning-notes (but read the books for real)
Will Larson's An Elegant Puzzle - Systems of Engineering Management is a nice more modern book. I would balance it with The Empathy Edge by Maria Ross.
A lot of management theory begins with Peter Drucker (and then Andy Grove and then John Doerr etc. - that OKR lifestyle approach - which has various and basically comes down to focusing on input metrics).
https://www.listennotes.com/podcasts/the-stack-overflow/writ...
mentions Smart & Gets Things Done (nice final chapter on different types of team challenges). Peopleware though aging has a lot of interesting examples. Leaders Eat Last was recommended to me by another manager. And in general that is basically the sentiment.
Like having kids, it's an opportunity to reflect on what you would've wanted differently in all the previous managers you have (and a realization that it's tougher than it looks). While managing engineers is different from leadership, it puts leadership on display perhaps more. Your blast radius is bigger. Most companies won't necessarily encourage it right away for ICs because it has a greater negative effect when it doesn't work.
And yeah to reiterate what others have said - it's a skill (i.e., a set of areas to make mistakes in) that takes time to develop/learn. And I'd say a lot of your growth comes down to who your manager will be in the process. If you have a good manager who mentors (in whichever style), you can learn a whole lot and improve your growth areas fairly quickly.
Also if you have more of an e2e and slight business focus, it can be interesting. It probably works best for those who are more generalist/breadth-first search types (though if good at communicating and delegating it's not strictly necessary); since there is a lot of "herding" for lack of a better word; and less time to depth-first dive into areas.
But yeah it gets better with time (or it should - and if it doesn't like I say it should be a two-way door). At first you might be like - what are all these 1:1s and why am I running around so much? And then with time, you'll start looking forward to all your 1:1s. And learning from others re: strategy (in general you'll participate with one level up meetings more; so if you're L6 at a company, you'll start seeing more what L7s see and be asked for input on, if you're responsible for a team). And starting to exhibit the best parts that you've learned before.
More links for the curious
https://www.youtube.com/watch?v=v3NLetsLYu4
https://randsinrepose.com/archives/
https://www.youtube.com/watch?v=u6XAPnuFjJc
The experiences I've heard about it (as mentioned here and which I'd reiterate) are that you can't think of your performance in days, but rather in weeks. You "work" through people. Though automation can be your programming outlet. And most importantly it's best to be honest with people. The people on your team everywhere will know when you're trying to cover things up, etc. Senior leaders will know when you're trying to cover things up. So just be honest and open and keep everyone's best interests in mind. And unlike my default learn to be "crisp" :)
What are some alternatives?
Tidier.jl - Meta-package for data analysis in Julia, modeled after the R tidyverse.
hl-mods - Modifications for Half-Life on GoldSrc. Includes active development of Cold Ice Remastered.
DataFramesMeta.jl - Metaprogramming tools for DataFrames
book-notes - Notes from books and other interesting things that I've read. Table of contents at the end 👇
db-benchmark - reproducible benchmark of database-like ops