Automated-performance-test-result-analysis Alternatives

automated-performance-test-result-analysis reviews and mentions

• [Question] What did I just do?
  3 projects | reddit.com/r/statistics | 12 Aug 2021

  Awesome thank you for your reply my knowledge of statistics is not good enough to really define what I have done here or how it is similar to something that already exists, or what might be a better solution for this. (so sorry for my ignorance on the subject) What I have done is write all of this in Python and I published it on my GitHub there you can see the matrix and everything else and run it on your own PC: https://github.com/JoeyHendricks/StatisticalTesting/blob/master/regression.py To give you a bit more context about why I use this test and how I define its success: My data is the results of a performance test on a IT system these test produce a large amount response times a small test would contain about 20k of measurement. We run a test every time we do a release on our system my test will be executed. This test is the final catch before something goes to a customer so it is important that if there is noticeable difference in the performance that the release is failed automatically so we can start debugging it.

  3 projects | reddit.com/r/statistics | 12 Aug 2021

  Thank you reply! Yes, I think what I have built really looks a lot Kolmogorov–Smirnov test I haven't gone through all the resources yet but going over it briefly really shows me the similarities. To clarify a bit what I am doing in step 2 at this point I have 2 arrays of bins that contain percentiles for samples A and B I then just calculate the difference in percentage between A and B. That number is then fed into the matrix which will output a score. below the matrix (the first column are percentages right column is the amount the score is reduced by if the percentage is higher than that number.): [2, 0.07688676], [4, 0.13395531], [8, 0.20272898], [16, 0.21770635], [32, 0.36872261], For some more details about my calculation, you could best check out my GitHub repo it contains the calculation written in Python. https://github.com/JoeyHendricks/StatisticalTesting/blob/master/regression.py Just to be clear on the lingo these "tests" like the Kolmogorov–Smirnov test can be called statistical tests correct? So in the correct language what I have built by myself is a statistical test right?

  3 projects | reddit.com/r/statistics | 12 Aug 2021
• A note from our sponsor - InfluxDB
  www.influxdata.com | 27 Mar 2023

  Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression. Learn more →