amplify-cli VS BentoML

Bug: Before continuing, we need to add some manual changes to the generated output since there is a bug in the 10.8.1 version of the Amplify CLI. To fix the issue, open the amplify/backend/predictions/identifyText<>/parameters.json file and add the following three key-value pairs to it:

AWS Amplify CLI - GitHub

If this is the first time using Amplify you'll need to install the Amplify CLI. This tool will help us setup and add Amplify's services.

This disconnection between the initial business cases of DynamoDB and Amplify can even be seen within the AWS teams themselves. [4] We don't believe any of them are to blame. The solo Front End Engineer bootstrapping a quick Amplify app for a PoC feels like a way different use case than a team of highly trained data engineers working on their Single Table Design for their micro-service. Amplify rightfully tries to offer an easy way to deal with storing data. And so it follows a standard SQL design with DynamoDB. This though leads to bad performance (original selling point of DynamoDB) or other limitations hard to anticipate.

Overall it is pretty clear and fine that Amplify focuses on PoC projects rather than production ones (with features like Geo-tagging [5] but no way to migrate data). However, when starting to get traction, it is a shame we need to completely eject instead of being able to extend because of lack of (boring but necessary) fundamentals.

[1] https://github.com/aws-amplify/amplify-cli/issues/10164

AWS Amplify code is open-sourced and could be found on GitHub at https://github.com/aws-amplify/amplify-cli. We could make use of the NPM packages deployed independently to recreate the functionality offered by the AWS Amplify CLI regarding the GraphQL transformers. The entry point for generating the Appsync resolvers could be found here and the class we are interested in is GraphQLTransform which takes as a parameter all the individual transformers and iterates over them to generate the GraphQL resolvers and the associated CloudFormation stacks for deploying those resolvers.

AWS Amplify - The AWS Amplify CLI is a toolchain which includes a robust feature set for simplifying mobile and web application development. The CLI uses AWS CloudFormation and nested stacks to allow you to add or modify configurations locally before you push them for execution in your account.

Just wanted to share my experience as someone who has an app using AppSync and Amplify on the App Store: using the iOS libraries has been painless (though I should note that I'm using the older AppSync SDK), but the Amplify CLI, which I use for the backend, has been a frequent source of frustration. It does a lot of magic which is great when it works, but frustrating when it doesn't. Some of my issues may have been the result of early adoption, but if you are thinking about using it for something with a backend that you are going to continue to maintain and develop, I would suggest considering other AWS options. I'm currently looking into transitioning away from the Amplify CLI to using AWS CDK with the Amplify transformers.

adding environment variables to lambda functions not surprisingly there is a github issue to request this feature. The TLDR is - the Environment variables tab in your Amplify console is used to add build configuration environment variables, not backend function environment variables. So to add lambda environment variables you would have to do that in your AWS Lambda console, for every function individually.

adding permissions to lambda functions to access resources outside the amplify stack in a reliable way. For example you have a lambda function that needs to talk to SES for sending emails, then you would have to grant your lambda permissions for SES actions. However you can't do that using AWS Amplify CLI, and updating the lambdas' permission in the cloudformation template is a no go, because it gets overwritten if you then add permission to the same lambda function using the CLI, which is a very sticky situation, because your only option is to modify the permissions again using the IAM console. More info in the SES permissions issue.

There is no straight forward way to add typescript support for the lambda functions

Link to GitHub -->

For 2), I am aware of a few options. Triton inference server is an obvious one as is the ‘transformer-deploy’ version from LDS. My only reservation here is that they require the model compilation or are architecture specific. I am aware of others like Bento, Ray serving and TorchServe. Ideally I would have something that allows any (PyTorch model) to be used without the extra compilation effort (or at least optionally) and has some convenience things like ease of use, easy to deploy, easy to host multiple models and can perform some dynamic batching. Anyway, I am really interested to hear people's experience here as I know there are now quite a few options! Any help is appreciated! Disclaimer - I have no affiliation or are connected in any way with the libraries or companies listed here. These are just the ones I know of. Thanks in advance.

In this category I’m a big fan of https://github.com/bentoml/BentoML

What I like about it is their idiomatic developer experience. It reminds me of other Pythonic frameworks like Flask and Django in a good way.

I have no affiliation with them whatsoever, just an admirer.

Github Page: https://github.com/bentoml/BentoML

Elastic License 2: https://github.com/bentoml/bentoctl/blob/v0.3.1/LICENSE.md which also applies to their Yatai kubernetes thing, but strangely not (yet?) to the similarly named repo which is Apache-2: https://github.com/bentoml/BentoML/blob/main/LICENSE

Using a general-purpose framework such as FastAPI involves writing a lot of boilerplate code just to get your API endpoint up and running. If deploying a model for a demo is the only thing you are interested in and you do not mind losing some flexibility, you might want to use a specialized serving framework instead. One example is BentoML, which will allow you to get an optimized serving endpoint for your model up and running much faster and with less overhead than a generic web framework. Framework-specific serving solutions such as Tensorflow Serving and TorchServe typically offer optimized performance but can only be used to serve models trained using Tensorflow or PyTorch, respectively.

BentoML - BentoML is a flexible, high-performance framework for serving, managing, and deploying machine learning models.

Also mlflow is not that optimized because it doesnt microbatch like torchserve/tfserving/bentoml. https://github.com/bentoml/BentoML/tree/master/benchmark