Generating Code Coverage¶
Scrutinizer can process code coverage information that you generate during your build. We support various formats for code coverage. As long as your test runner generates one of the compatible formats, Scrutinizer can automatically process and merge coverage data from multiple test runs (even of different runners or languages).
You can also define failure conditions related to coverage like how much coverage a build must have or how well new code should be covered for a build to pass.
When you run a command, simply tell Scrutinizer where to find the coverage file and what format it has:
build: nodes: coverage: tests: override: - command: ./my-test-run coverage: file: path/to/coverage-data # These formats are supported: # clover, cobertura, jacoco, python-cc, ruby-cc, go-cc, lcov format: name-of-format
Clover is the most common format, and generated by virtually all modern test runners. When running your tests, simply tell Scrutinizer where to find the clover file and it will pick up the coverage data from there:
build: nodes: coverage: tests: override: - command: ./run-my-tests coverage: file: target/coverage.xml # <- Set this to where we find the coverage data. # The path is relative to the current directory. format: clover
For Python, we support data generated by
coverage. When setting up your test commands, simply specify where we find
.coverage file, and potentially your
.coveragerc configuration file:
build: nodes: coverage: tests: override: - command: './run-tests.sh' coverage: file: '.coverage' config_file: '.coveragerc' format: 'py-cc'
For Ruby, code coverage can be generated using our
scrutinizer-ocular gem that uses
SimpleCov internally. When
you have set-up the scrutinizer-ocular gem, make sure to set the
SCRUTINIZER_CC_FILE environment variable:
build: nodes: coverage: tests: override: - command: 'bundle exec rspec spec' environment: 'SCRUTINIZER_CC_FILE': 'my-coverage' coverage: file: 'my-coverage' format: 'rb-cc'
For Java, Scrutinizer also supports the
build: nodes: coverage: tests: override: - command: mvn clean package coverage: file: path/to/cobertura format: cobertura
As another Java format, Scrutinizer also supports
build: nodes: coverage: tests: override: - command: mvn clean package coverage: file: '[path-to-coverage-reports]' format: 'jacoco'
lcov coverage format was originally created for Linux test coverage, but is supported by various other runners, too:
build: nodes: coverage: tests: override: - command: make test coverage: file: '/path/to/lcov.coverage' format: 'lcov'
For Go projects, Scrutinizer supports the default coverage file format of the go tool Cover <https://golang.org/cmd/cover/> along with the cross language formats like
If you use the
Cover tool as an example, the configuration will look like:
build: nodes: coverage: tests: override: - command: go test -coverprofile=cover.out coverage: file: 'cover.out' format: 'go-cc'