Cookbook
data-engineering-zoomcamp
Cookbook | data-engineering-zoomcamp | |
---|---|---|
21 | 119 | |
12,945 | 22,562 | |
- | 2.4% | |
7.8 | 9.4 | |
about 1 month ago | 9 days ago | |
Jupyter Notebook | ||
Apache License 2.0 | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Cookbook
-
Tranzitie catre data engineering
https://github.com/andkret/Cookbook arunca un ochi aici. Omul are si youtube channel https://www.youtube.com/@andreaskayy
-
How do i become a data engineer?
I can recommend https://learndataengineering.com by Anreas Krenz. Will guide you via all important topics starting from sql & python to building pipelines using AWS/GCP. I used to participate for 1 year (costs ~ 200 Euro/220$). It's a self-paced. So for ~15h/week you can switch into DE position for appr. 6 months.
-
I start my first day as a Data Engineer next Monday, any tips?
I wonder if anyone involved in this post and comments have tried this? https://learndataengineering.com/
-
Data engineering certificates
I think it's allowed: https://learndataengineering.com
-
Can Mechanical Engineers become MLOps?
From your post, you seem to be trained for data science for physics modeling, so I'd recommend to get started with https://ml-ops.org/ and for the data engineering part, I found this https://github.com/andkret/Cookbook open source cookbook to be invaluable.
-
Furthering SQL career
I am doing this currently to fill in the blanks: https://learndataengineering.com. Also, do you know Python? If not take class on Udemy on that. Finally, data engineering is all about tools these days. I saw someone recommended this book here: Data Engineering with Python, I find it super hopeful. You download these tools (Apache Airflow, etc) and get a go with it. I am going to build some data pipelines via this book :)
-
Any online bachelor/masters degree to recommend for data engineering?
the best way to be a dev or DE is to build stuff, not learning about algorithms. Just google DE academy, bootcamp or so. The linked one is quite good for a cheap price. A degree prepares you mostly for a PhD, not for a job. So dont look for degrees preparing you for a job in general.
-
Beginner DE Courses on Coursera/Udemy?
I usually don't do self promotion, but because you directly asked for a good source. Look at my academy: https://learndataengineering.com
-
Women in data engineering
Find something like https://learndataengineering.com/, udemy or any other 'bootcamp/course' that goes on for few months and learn it. It is important that you will have some mentors or study buddies to exchange ideas or so.
- Data Engineering - consigli
data-engineering-zoomcamp
-
Data Engineering Zoomcamp Week 6 - using redpanda 1
References: Data engineering zoomcamp week 6 course and homework notes: https://github.com/DataTalksClub/data-engineering-zoomcamp/tree/main/cohorts/2024/06-streaming
-
Final project part 5
dbt is the main part of my data engineering project for Data Talks Club's data engineering zoomcamp. After a few frustrating errors on my part, I finally figured out how to make models, where to put the staging models and where to put the core models, how to compile a seed file, and how to join it to the main file in order to produce data for visualization. I also used the git interface to continually upgrade my repository. This was extremely convenient and helpful.
-
Building a project in DBT
For Week 4 of DataTalksClub's data engineering zoomcamp, we had to install dbt and create a project. This was a formidable task. dbt is a data transformation tool that enables data analysts and engineers to transform data in a cloud analytics warehouse, BigQuery in our case. It took me a very long time to do this, and in this case I needed the homework extension.
-
Testing and documenting DBT models
In this video we learned how to test and document dbt models. We also learned about the codegen library. This is part of Week 4 of the data engineering zoomcamp by DataTalksClub.
-
Extracting data with dlt
If you want to run these commands yourself, either in a Jupyter notebook or in Google Colab, you can get the file from HERE. You can get an overview of the workshop HERE. When I ran in a Jupyter notebook, I had to delete the first line (%%capture) and put quotes around dlt[duckdb] in the second line.
-
Data engineering at home?
Take a look.DE zoomcamp
-
Rockstar Data Engineers making big bucks: what are you doing exactly?
If you need guidance you can attend the data engineering zoomcamp, it's free and quite solid.
-
Self study material
Welcome. Start with Data Engineering Zoomcamp, try and build a project, see if you like it, then continue to get into deeper resources.
-
What is the best way to learn Python if I want to become a data engineer
Can take a look at this - https://github.com/DataTalksClub/data-engineering-zoomcamp
-
Course Recommendations for a New Grad
I think you can start with something free with this pretty practical course on Data Engineering from DataTalksClub - https://github.com/DataTalksClub/data-engineering-zoomcamp
What are some alternatives?
Shuffle - Shuffle: A general purpose security automation platform. Our focus is on collaboration and resource sharing.
mlops-zoomcamp - Free MLOps course from DataTalks.Club
data-engineering-book - Accumulated knowledge and experience in the field of Data Engineering
AdventureWorks - Projects using the AdventureWorks database
Coursera-Clone - Coursera clone
versatile-data-kit - One framework to develop, deploy and operate data workflows with Python and SQL.
data-engineer-roadmap - Roadmap to becoming a data engineer in 2021
Reddit-API-Pipeline
applied-ml - 📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.
udacity-capstone
self-hosted-cookbook - A cookbook, for docker-compose based recipes, for self-hosted applications and services.
DataEngineerZoomCamp - I'm partaking in a Data Engineering Bootcamp / Zoomcamp. I'll store files and progress here.