Traceroute VS Scientist

add traceroute

For some google-fu, the ruby / .NET equivalent of this is https://github.com/github/scientist / https://github.com/scientistproject/Scientist.net

The readme (here https://github.com/github/scientist#alternatives) doesn't mention, but here is one for Rust: https://crates.io/crates/scientisto

Something I've learned in Ruby land (prob standard in other places, forgive my ignorance) that seems a bit different than what the article advocates for (fake services):

- Write your service wrapper (eg your logic to interact with Twilio)

- Call the service and record API outputs, save those as fixtures that will be returned as responses in your tests without hitting the real thing (eg VCR, WebMock)

- You can now run your tests against old responses (this runs your logic except for getting a real response from the 3rd party; this approach leaves you exposed to API changes or you have edge cases not handled)

For the last part, two approaches to overcome this:

- Wrap any new logic in try/catch and report to Sentry: you avoid breaking prod and get info on new edge cases you didn't cover (this may not be feasible if the path where you're inserting new logic into does not work at all without the new feature; address this with thoughtful design/rollout of new features)

- Run new logic side by side to see what happens to the new logic when running in production (https://github.com/github/scientist)

I use the first approach bc small startup.

Check out GitHub scientist if you are doing a migration with a ruby based system: https://github.com/github/scientist

Great support and functionality for testing differences between two systems of record.

As far as this idea, I have seen this before in a few different forms. The closest thing that I've personally witnessed being used is the scientist gem for Ruby applications. You have to do it manually, but you can instrument your code to compare old and new versions of some code. It also does some fancy stuff like randomly choosing which version gets run, almost like an A/B test. I wonder if there's a similar library for Python?

Not sure what OP had in mind, but for my dream strangler (that's a phrase I never expected to use), I'd love functionality like github's scientist library; basically, the ability to implement a route, continue to serve most requests through the original service, but duplicate a small percentage to the new implementation, compare the outputs of the two services, and log wherever the responses differ, so you get live production tests to exercise the new service without impacting users.

However, the good news is that it’s easy and safe to do so in Ruby and Rails using the Scientist gem. Scientist's name is based on the scientific method of conducting experiments to verify a given hypothesis. In this case, our hypothesis is that the new code does the job.

Github scientist.