appengine-java-standard
bigquery-utils
Our great sponsors
| appengine-java-standard | bigquery-utils | |
|---|---|---|
| 1 | 6 | |
| 192 | 1,028 | |
| 0.0% | 1.9% | |
| 9.1 | 6.4 | |
| 1 day ago | 10 days ago | |
| Java | Java | |
| Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
appengine-java-standard
-
Google Open-Sources The App Engine Standard Java runtime
Github: https://github.com/GoogleCloudPlatform/appengine-java-standard
bigquery-utils
-
Swirl: An open-source search engine with LLMs and ChatGPT to provide all the answers you need 🌌
Using the Galaxy UI, knowledge workers can systematically review the best results from all configured services including Apache Solr, ChatGPT, Elastic, OpenSearch, PostgreSQL, Google BigQuery, plus generic HTTP/GET/POST with configurations for premium services like Google's Programmable Search Engine, Miro and Northern Light Research.
-
Modern data stack: scaling people and technology at FINN
Data Transformations: This phase involves modifying and integrating tables to generate new tables optimized for analytical use. Consider this example: you want to understand the purchasing behavior of customers aged between 20-30 in your online shop. This means you'll need to join product, customer, and transaction data to create a unified table for analytics. These data preparation tasks (e.g., joining fragmented data) for analysis are essentially what "Data Transformations" entail. At FINN, technologies utilized in this phase include BigQuery as a data warehouse, dbt for data transformation, and a combination of GitHub Actions and Datafold for quality assurance.
-
Running Transformations on BigQuery using dbt Cloud: step by step
Introduction In today's data-driven world, transforming raw data into valuable insights is crucial. This process, however, often involves complex tasks that demand efficiency, scalability, and reliability. Enter dbt Cloud—a powerful tool that simplifies data transformations on Google BigQuery. In this article, we'll take you through a step-by-step guide on how to run BigQuery transformations using dbt Cloud. Let's dive in!
-
Do I need a cloud computing–based data cloud company
You'll want to evaluate what BigQuery has to offer and see if it makes sense for you to move over.
-
I used ChatGPT to get an Internship
Watch the introductory videos on BigQuery on the Google Cloud Platform website (https://cloud.google.com/bigquery)
-
Wrangling BigQuery at Reddit
Within the audit logs you can find BigQueryAuditMetadata details in the protoPayload.metadataJson submessage in the Cloud Logging LogEntry message. GCP has offered several versions of BigQuery audit logs so there are both older “v1” and newer “v2” versions. The v1 logs report API invocations and live within the protoPayload.serviceData submessage while the v2 logs report resource interactions like which tables were read from and written to by a given query or which tables expired. The v2 data lives in a new field formatted as a JSON blob within the BigQueryAuditMetadata detail inside the protoPayload.metadataJson submessage. In v2 logs the older protoPayload.serviceData submessage does exist for backwards compatibility but the information is not set or used. We scrape details from the JobChange object instead. We referenced the GCP bigquery-utils Git repo for how to use INFORMATION_SCHEMA queries and audit logs queries.
What are some alternatives?
Light-Java - A fast, lightweight and more productive microservices framework
solr - Apache Solr open-source search software
jasypt-spring-boot - Jasypt integration for Spring boot
swirl-search - Swirl is an open-source search platform that uses AI to search multiple content and data sources simultaneously and return AI-ranked results. And provides summaries of your answers from searches using LLMs. It's a one-click, easy-to-use Retrieval Augmented Generation (RAG) Solution.
Calculator-FX - A scientific calculator, capstone, and passion project made in JavaFX
DataflowTemplates - Cloud Dataflow Google-provided templates for solving in-Cloud data tasks
amaya-tomcat - About Amaya is a fairly lightweight web framework for Java, which guarantees speed, ease of creating plugins/addons, flexibility and ease of use. Tomcat implementation.
dataproc-templates - Dataproc templates and pipelines for solving simple in-cloud data tasks
Reified - Reified in Java 11 and upwards
spark-bigquery-connector - BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.
echothree - Echo Three Mirror
nodejs-bigquery - Node.js client for Google Cloud BigQuery: A fast, economical and fully-managed enterprise data warehouse for large-scale data analytics.