Knapsack
Ruby Tests Profiling Toolbox
Knapsack | Ruby Tests Profiling Toolbox | |
---|---|---|
5 | 7 | |
507 | 1,802 | |
0.6% | 1.1% | |
3.4 | 7.8 | |
3 months ago | 12 days ago | |
Ruby | Ruby | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Knapsack
-
Ask HN: What is the most impactful thing you've ever built?
I've created a knapsack ruby gem for CI parallelisation that has over 122 million downloads. Primarily due to the fact, Gitlab is using it.
I spin off https://knapsackpro.com from the knapsack gem and we are helping our customers run fast CI builds.
- Run 1 hour test suite in 2 min with optimal parallelisation on existing CI infra
-
Testing Ruby on Rails on Github Actions with RSpec
name: Main on: [push] jobs: test: runs-on: ubuntu-latest # If you need DB like PostgreSQL, Redis then define service below. # https://github.com/actions/example-services/tree/master/.github/workflows services: postgres: image: postgres:10.8 env: POSTGRES_USER: postgres POSTGRES_PASSWORD: "" POSTGRES_DB: postgres ports: - 5432:5432 # needed because the postgres container does not provide a healthcheck # tmpfs makes DB faster by using RAM options: >- --mount type=tmpfs,destination=/var/lib/postgresql/data --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5 redis: image: redis ports: - 6379:6379 options: --entrypoint redis-server # https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix strategy: fail-fast: false matrix: # [n] - where the n is a number of parallel jobs you want to run your tests on. # Use a higher number if you have slow tests to split them between more parallel jobs. # Remember to update the value of the `ci_node_index` below to (0..n-1). ci_node_total: [8] # Indexes for parallel jobs (starting from zero). # E.g. use [0, 1] for 2 parallel jobs, [0, 1, 2] for 3 parallel jobs, etc. ci_node_index: [0, 1, 2, 3, 4, 5, 6, 7] env: RAILS_ENV: test GEMFILE_RUBY_VERSION: 2.7.2 PGHOST: localhost PGUSER: postgres # Rails verifies the time zone in DB is the same as the time zone of the Rails app TZ: "Europe/Warsaw" steps: - uses: actions/checkout@v2 - name: Set up Ruby uses: ruby/setup-ruby@v1 with: # Not needed with a .ruby-version file ruby-version: 2.7 # runs 'bundle install' and caches installed gems automatically bundler-cache: true - name: Create DB run: | bin/rails db:prepare - name: Run tests env: KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC: ${{ secrets.KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC }} KNAPSACK_PRO_CI_NODE_TOTAL: ${{ matrix.ci_node_total }} KNAPSACK_PRO_CI_NODE_INDEX: ${{ matrix.ci_node_index }} KNAPSACK_PRO_LOG_LEVEL: info # if you use Knapsack Pro Queue Mode you must set below env variable # to be able to retry CI build and run previously recorded tests # https://github.com/KnapsackPro/knapsack_pro-ruby#knapsack_pro_fixed_queue_split-remember-queue-split-on-retry-ci-node KNAPSACK_PRO_FIXED_QUEUE_SPLIT: true # RSpec split test files by test examples feature - it's optional # https://knapsackpro.com/faq/question/how-to-split-slow-rspec-test-files-by-test-examples-by-individual-it KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES: true run: | bundle exec rake knapsack_pro:queue:rspec
-
How to run fast RSpec tests on CircleCI with parallel jobs and have nice JUnit XML reports in CircleCI web UI
# Ruby CircleCI 2.0 configuration file # # Check https://circleci.com/docs/2.0/language-ruby/ for more details # version: 2 jobs: build: parallelism: 10 # https://circleci.com/docs/2.0/configuration-reference/#resource_class resource_class: small docker: # specify the version you desire here - image: circleci/ruby:2.7.1-node-browsers environment: PGHOST: 127.0.0.1 PGUSER: my_db_user RAILS_ENV: test # Split slow RSpec test files by test examples # https://knapsackpro.com/faq/question/how-to-split-slow-rspec-test-files-by-test-examples-by-individual-it KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES: true # Specify service dependencies here if necessary # CircleCI maintains a library of pre-built images # documented at https://circleci.com/docs/2.0/circleci-images/ - image: circleci/postgres:10.6-alpine-ram environment: POSTGRES_DB: my_db_name POSTGRES_PASSWORD: "" POSTGRES_USER: my_db_user # Rails verifies Time Zone in DB is the same as time zone of the Rails app TZ: "Europe/Warsaw" - image: redis:6.0.7 working_directory: ~/repo environment: TZ: "Europe/Warsaw" steps: - checkout # Download and cache dependencies - restore_cache: keys: - v2-dependencies-bundler-{{ checksum "Gemfile.lock" }}-{{ checksum ".ruby-version" }} # fallback to using the latest cache if no exact match is found - v2-dependencies-bundler- - run: name: install ruby dependencies command: | bundle install --jobs=4 --retry=3 --path vendor/bundle - save_cache: paths: - ./vendor/bundle key: v2-dependencies-bundler-{{ checksum "Gemfile.lock" }}-{{ checksum ".ruby-version" }} # Database setup - run: bin/rails db:prepare - run: name: run tests command: | export CIRCLE_TEST_REPORTS=/tmp/test-results mkdir $CIRCLE_TEST_REPORTS bundle exec rake "knapsack_pro:queue:rspec[--format documentation --format RspecJunitFormatter --out tmp/rspec.xml]" # collect reports - store_test_results: path: /tmp/test-results - store_artifacts: path: /tmp/test-results destination: test-results
-
Six things you should consider while designing a test architecture
One, the interesting out-of-the-box solution is Knapsack Pro. It helps run your tests in a parallel efficient way.
Ruby Tests Profiling Toolbox
-
Must-have gems for mature Rails
gem "test-prof" - https://github.com/test-prof/test-prof | Toolkit for inspecting and optimising your test-suite, a must-have.
-
Measuring load time on fixtures
You might be able to do something with https://test-prof.evilmartians.io, but I'm not sure it has anything specific to fixtures out of the box. Maybe using the event profiler on sql.active_record events would be close enough. In the limit, you could wire together your own ActiveSupport::Notifications.instrument around the relevant blocks + an ActiveSupport::Notifications.subscribe to that event so as to log the relevant information. Docs: https://api.rubyonrails.org/classes/ActiveSupport/Notifications.html
-
How to optimize factory creation.
To have a better vision of what objects are created in our spec file we can use test-prof, a powerful gem that provides a collection of different tools to analyse your test suite performance. One of this tool is really useful to identify a factory cascade, let’s introduce factory profiler.
-
A Trick For Reading Flamegraphs
TestProf can be used to get flamegraphs for Ruby test suites.
- How to improve a test suit made with Rspec, Capybara, FactoryBot and Siteprism
-
Does pytest break a lot of coding rules?
Rspec has spec_helper.rb.
-
Learning resources to broaden the knowledge
I also love the test-prof documentation, there's a lot of good ideas on how to improve test performance.
What are some alternatives?
Parallel Tests - Ruby: 2 CPUs = 2x Testing Speed for RSpec, Test::Unit and Cucumber
DuckRails - Development tool to mock API endpoints quickly and easily (docker image available)
vcr - Record your test suite's HTTP interactions and replay them during future test runs for fast, deterministic, accurate tests.
Spinach - Spinach is a BDD framework on top of Gherkin.
Spring - Rails application preloader
minitest - minitest provides a complete suite of testing facilities supporting TDD, BDD, mocking, and benchmarking.
timecop - A gem providing "time travel", "time freezing", and "time acceleration" capabilities, making it simple to test time-dependent code. It provides a unified method to mock Time.now, Date.today, and DateTime.now in a single call.
Wrong - Wrong provides a general assert method that takes a predicate block. Assertion failure messages are rich in detail.
factory_bot - A library for setting up Ruby objects as test data.
Appraisal - A Ruby library for testing your library against different versions of dependencies.
faker - A library for generating fake data such as names, addresses, and phone numbers.