bigquery-utils
starter-workflows
bigquery-utils | starter-workflows | |
---|---|---|
6 | 263 | |
1,037 | 8,464 | |
1.4% | 1.5% | |
6.1 | 8.6 | |
about 6 hours ago | 4 days ago | |
Java | TypeScript | |
Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
bigquery-utils
-
Swirl: An open-source search engine with LLMs and ChatGPT to provide all the answers you need 🌌
Using the Galaxy UI, knowledge workers can systematically review the best results from all configured services including Apache Solr, ChatGPT, Elastic, OpenSearch, PostgreSQL, Google BigQuery, plus generic HTTP/GET/POST with configurations for premium services like Google's Programmable Search Engine, Miro and Northern Light Research.
-
Modern data stack: scaling people and technology at FINN
Data Transformations: This phase involves modifying and integrating tables to generate new tables optimized for analytical use. Consider this example: you want to understand the purchasing behavior of customers aged between 20-30 in your online shop. This means you'll need to join product, customer, and transaction data to create a unified table for analytics. These data preparation tasks (e.g., joining fragmented data) for analysis are essentially what "Data Transformations" entail. At FINN, technologies utilized in this phase include BigQuery as a data warehouse, dbt for data transformation, and a combination of GitHub Actions and Datafold for quality assurance.
-
Running Transformations on BigQuery using dbt Cloud: step by step
Introduction In today's data-driven world, transforming raw data into valuable insights is crucial. This process, however, often involves complex tasks that demand efficiency, scalability, and reliability. Enter dbt Cloud—a powerful tool that simplifies data transformations on Google BigQuery. In this article, we'll take you through a step-by-step guide on how to run BigQuery transformations using dbt Cloud. Let's dive in!
-
Do I need a cloud computing–based data cloud company
You'll want to evaluate what BigQuery has to offer and see if it makes sense for you to move over.
-
I used ChatGPT to get an Internship
Watch the introductory videos on BigQuery on the Google Cloud Platform website (https://cloud.google.com/bigquery)
-
Wrangling BigQuery at Reddit
Within the audit logs you can find BigQueryAuditMetadata details in the protoPayload.metadataJson submessage in the Cloud Logging LogEntry message. GCP has offered several versions of BigQuery audit logs so there are both older “v1” and newer “v2” versions. The v1 logs report API invocations and live within the protoPayload.serviceData submessage while the v2 logs report resource interactions like which tables were read from and written to by a given query or which tables expired. The v2 data lives in a new field formatted as a JSON blob within the BigQueryAuditMetadata detail inside the protoPayload.metadataJson submessage. In v2 logs the older protoPayload.serviceData submessage does exist for backwards compatibility but the information is not set or used. We scrape details from the JobChange object instead. We referenced the GCP bigquery-utils Git repo for how to use INFORMATION_SCHEMA queries and audit logs queries.
starter-workflows
- Say Goodbye to Manual Deployments: Automate Your EC2 Autoscaling with CodeDeploy and GitHub Actions
-
Level Up Your Projects with GitHub Actions & CI/CD
GitHub, as one of the leading web-based Git repository hosting service, provides a powerful suite of CI/CD tools in the form of GitHub Actions. These are directly integrated into the platform which empowers developers to increase the speed, efficiency and reliability of delivering products. In this brief article, we will take a look at what CI/CD is, why we should use it, as well as some of its applications in my projects.
-
How to Manage Terraform with GitHub Actions
GitHub Actions is a modern CI/CD tool integrated natively on GitHub. Itenables the rapid automation of build, test, deployment, and other custom workflows on GitHub with no need for external tools.
-
Kubernetes CI/CD Pipelines
GitHub Actions is GitHub's CI/CD solution. You can use it to run automated tasks each time you change your code. Although the platform lacks a built-in Kubernetes integration, third-party plugins such as Azure's Deploy to Kubernetes Cluster action can automate deployments and manage different rollout strategies.
-
Most Useful CI/CD Tools for DevOps
GitHub Actions is a feature-rich CI/CD platform embedded within GitHub, enabling developers to automate, customize, and execute software development workflows directly in their repositories. An Action inside GitHub Actions is a discrete unit of automation that performs a specific task within a workflow. All the Actions are reusable, and there are many to choose from. You can even create your own reusable ones.
-
Awesome GitHub Action Workflows
actions/starter-workflows
-
Laravel code-quality tools
The real power of using PHP code-quality tools is when it’s added to your continuous integration process, which means it automatically checks the code every time someone makes a push or pull request to your project repo. In this section, we'll be looking at how to do just that. GitHub actions is available for free so we'll use it for demo purposes. Note that there are some limits to private repos, so set your test repo to public if you can.
-
Elevate Your GitHub README Game
You can even automate the running of this script — hence the directory name automation — to happen every time the data changes, using GitHub Actions.
-
GitHub Status Checks and Branch Protection Made Easy
# Based on https://github.com/actions/starter-workflows/blob/main/ci/node.js.yml name: CI on: pull_request: branches: - main jobs: ci: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - uses: actions/setup-node@v2 with: node-version: lts/* cache: 'npm' - run: npm ci - run: npm run build --if-present - run: npm test
-
GitHub Actions for Perl Development
You might remember that I’ve been taking an interest in GitHub Actions for the last year or so (I even wrote a book on the subject). And at the Perl Conference in Toronto last summer I gave a talk called “GitHub Actions for Perl Development” (here are the slides and the video).
What are some alternatives?
solr - Apache Solr open-source search software
argocd-image-updater - Automatic container image update for Argo CD
swirl-search - Swirl is an open-source search platform that uses AI to search multiple content and data sources simultaneously and return AI-ranked results. And provides summaries of your answers from searches using LLMs. It's a one-click, easy-to-use Retrieval Augmented Generation (RAG) Solution.
CppCon2020 - Slides and other materials from CppCon 2020
DataflowTemplates - Cloud Dataflow Google-provided templates for solving in-Cloud data tasks
NewPipe - A libre lightweight streaming front-end for Android.
dataproc-templates - Dataproc templates and pipelines for solving simple in-cloud data tasks
react-native-dotenv - Load react native environment variables using import statements for multiple env files.
spark-bigquery-connector - BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.
nnn - nÂł The unorthodox terminal file manager
appengine-java-standard - Google App Engine Standard Java runtime: Prod runtime, local devappserver, Cloud SDK Java components, GAE APIs, and GAE API emulators.
Real_Time_Image_Animation - The Project is real time application in opencv using first order model