ck VS aws-deployment-framework

If you have an idle Nvidia GPU and Linux, can you please help the community test the beta version of the open-source framework for composable benchmarking and design space exploration of ML systems: https://github.com/mlcommons/ck/blob/master/cm-mlops/project/mlperf-inference-v3.0-submissions/docs/crowd-benchmark-mlperf-bert-inference-cuda.md ?

Hi! Just sharing this tutorial from the Student Cluster Competition at SuperComputing'22 to learn how to modularize and run ML Systems benchmarks. 10 international teams had about 30 minutes to run it and most of them succeeded while sharing their results at the live dashboard . It is a part of the ongoing effort to modularize ML Systems and automate their benchmarking and optimization. Feedback is very welcome!

We would like to ask for a favor: we have prepared a tutorial to help students run the MLPerf inference benchmark across different platforms at the Student Cluster Competition at SuperComputing'22 in a few days: https://github.com/mlcommons/ck/blob/master/docs/tutorials/s... .

We would like to test it across different machines before students run it ;) . If you have time, please help us go through this tutorial and run this benchmark on any available system - it should not take more than 20..30 minutes.

If you encounter any issues, please report them at https://github.com/mlcommons/ck/issues so that we could fix them before the competition.

Thank you for supporting this community project!

Just to let you know that we are preparing a new working group at MLCommons to help the community modularize ML/AI Systems and automate their benchmarking, optimization and deployment. It will be based on the MLPerf methodology and MLCommons "Collective Knowledge" automation meta-framework that was already used to automate recent MLPerf inference benchmark submissions from Qualcomm, HPE, Lenovo, Krai, DELL and OctoML. Please join the group here to provide your feedback and help with this community effort! Thank you!

Hi! Just sharing our open-source project to automate MLPerf benchmarks and make it easier for everyone to plug in their real-world ML models, data sets, frameworks/SDKs and hardware. Feedback is very welcome!

– Their solution product https://cknowledge.io/ and source code https://github.com/ctuning/ck\

I guess it should be helpful to the researchers community.

In some scenarios you might have the need to replicate an AWS CodeCommit repository. When I was setting up a test organization using AWS Deployment Framework (ADF) I ran into this myself. Because I want to test the deployment of my landing zone I needed to have a close replica. This includes the CodeCommit setup. But at the same time I did not want to change the development workflow. The workflow is pretty straight forward. You create a feature branch to work in. When you are ready you merge it to a development branch. When it needs to go to production you merge it into the main branch. So we will use the development branch to deploy to the test organization. But, because the test organization is a replica of production. Merging to the development branch would not have effect on the test organization. For this we need to synchronize the development branch to the test organization.

Within AWS Organizations you can apply Service control policies (SCPs). All AWS Accounts under the OU (Organization Unit) with the SCP will be subjective to this SCP. What if you need to make a change in this SCP? How can you test this change? SCPs are not the only things you might want to test. Remember that I mentioned that ADF is also bootstrapping the accounts? That could be a VPC with subnets for networking. How do you ensure that the change that you made works as intended? When merging to your main branch. It will trigger a rollout process depending on your configuration.

AWS Deployment Framework (ADF)