deploy-cloud-functions VS workflows-samples

One of the issues developers can encounter when developing in Cloud Functions is the time taken to deploy changes. You can help reduce this time by dynamically loading some of your Python classes. This allows you to make iterative changes to just the area of your application that you’re working on.

I've been looking at Google Secret Manager which sounds promising but I've not been able to find any examples or tutorials that help with the actual practical details of best practice or getting this working. I'm currently reading about Cloud Functions which also sound promising but again, I'm just going deeper and deeper into GCP without feeling like I'm gaining any useful insights.

Serverless computing was also introduced, where the developers focus on their code instead of server configuration.Google offers serverless technologies that include Cloud Functions and Cloud Run.Cloud Functions manages event-driven code and offers a pay-as-you-go service, while Cloud Run allows clients to deploy their containerized microservice applications in a managed environment.

Lambda is made for your use case :). It doesn’t have to be AWS there are plenty of other serverless computing services like: - Google cloud functions - Azure functions Etc

Once you have some basic familiarity with programming, try deploying one of your Python programs to the cloud. Start with Cloud Functions, because that doesn't require any knowledge of Linux server administration.

If the customer prefers making REST-like calls: Deploy a simple Cloud Function that the customer would invoke by making a regular HTTP call with some parameters. The Cloud Function would validate the customer's credentials, and then send a query to BigQuery using one of the client libraries. You can write Cloud Functions in Node.js, Python, Go, Java, C#, Ruby, or PHP. You are only charged when the function runs.

Cloud Functions: A serverless platform for event-driven functions

I propose starting with Cloud Functions. You can use your Python experience, you can do rapid prototyping by writing your code in a text-box in the Google Cloud Console, and there will be no server setup or maintenance.

AWS Lambda If you're in Azure, your equivalent service is Azure Functions. For Google, this is Google Functions (yes, AWS just HAD to be different). Regardless of its name, all of these services fulfill the same purpose - a small compute building block to house your business logic code. An AWS Lambda function is simply the code you want to run, written in your language of choice (I preference Python, but Typescript and Java are popular options). In your infrastructure code, you specify some lambda function basics, like name, path to the business logic code, security role, and what runtime you're using, and optionally have the ability to control more parameters like timeout, concurrency, aliases, and more. Lambda even has built in integrations to other AWS services, such as S3 and SQS (we'll get to these) to make application development even easier. Additionally, lambda functions are priced based on the number of times they're invoked and the duration of time they run, making them exceptionally affordable.

Have you seen this? https://cloud.google.com/workflows

Cloud Workflows: A serverless workflow orchestration service

Workflows can trigger actions, like Cloud Run Jobs, in a sequence of steps. The Workflows product waits for the job to complete, fail, or time out before it moves on to the next step. It uses polling to check on the job, so there may be a delay between the job finishing and the next step.

Has anyone integrated firestore/realtime database together with GCP workflows? What was your use case? How was your experience with it? Why have you decided to go that way?

In the past few years, YAML (http://yaml.org) has become an essential part of software, particularly for infrastructure-as-code tools. Yaml at the heart of kubernetes configuration, kubernetes-inspired APIs like Google's config connector, and a number of workflow systems like Google Cloud Workflows and Github Actions.

If your routine is just a bunch of API calls, you can also replace steps 1-2 with Workflows.

I have come at this problem from a bit of a different angle by asking what is the closest I can possibly get to the hypothetical dream state of everything is automated, autoscaling blah blah blah as possible with the absolute smallest budget in terms of not only actual costs but time budget as well.

I only know the GCP ecosystem kind of well so I don't fully know to what extent these things exist in AWS and Azure but there I think there is a really nice path you can get on with the serverless route that skips K8s entirely but keeps you very well aligned in case you ever need to "upgrade" or get out of the GCP ecosystem.

I write very stock standard gRPC services and then put them onto Cloud Run (which has a very Heroku like workflow) and stick https://cloud.google.com/api-gateway in front of things and now my API is running on the exact same setup as any other service Google is running in production. Huge amounts of logic get moved out of my code base as a result.

If you are also willing to write your APIs a fairly particular way https://google.aip.dev/ it starts to become trivial to integrate other things like https://cloud.google.com/workflows, https://cloud.google.com/pubsub and https://cloud.google.com/tasks which is traditionally where a lot of the "state" and weirdly complicated logic previously lived in my code. I'm now not really writing any of that.

Now it's all declarative where I just say what I want to happen and I don't have to think about much else beyond that because it too is using that same internal GCP infrastructure to handle all the complicated parts around what to do when things go wrong.

But to me they are all extremely heavily aligned with the K8s path so the lock in certainly doesn't feel as scary.

Google Workflows combines Google’s cloud services and APIs to help developers build reliable large-scale applications, process automation, and deploy machine learning and data pipelines.

Lately, I build a system using Cloud Workflows which can combine Google Cloud Services such as Cloud Functions and Cloud Run. Sometimes, I was in a situation where I want to examine more efficiently using logs on Cloud Logging when debugging or daily monitoring.

Look at Cloud Workflows for simple workflows