This user guide helps the user optimize the PinDown settings for different test suites.

1 Small Test Suites

Small test suites that run frequently are a great way of catching a lot of regression bugs fast. Most often the complete test suite passes but occassionally tests fail due to either a new regression bug or a random seed generating a new test scenario that catches a bug that has always existed. When failures occur it is normally due to very few underlying bugs, often only just one bug. Consequently PinDown should be setup to debug one bug at a time as fast as possible.

2 Large Test Suites

Large test suites are run to get good coverage and capture most regression bugs. Each run produces are lot of failures that needs to be debugged. Consequently PinDown should be setup to debug the complete set of bugs as fast as possible. This is done by debugging several bugs in parallel. Compared to section 1 above it takes longer for the first bug report to be issued but the throughput of bug reports is greater.

3 Mutation Testing

The special configuration for Mutations testing allows for testing of inserted mutations. For best performance the mutations should be able to be enabled without rebuilding anything. In this configuration it is possible to use a high bandwidth since no code checkout needs to be duplicated.