When to stop testing? This question gets asked more often than not. However, as simple and
straightforward it sounds, the answer to this question spans multiple aspects and
variables which should be considered when we really want to stop testing. We
just wish the answer could be “When all defects are found!”. Like humans, the
software is also mere mortal and never bug-free.
This blog aims to discuss multiple factors which should be pondered before we
make a decision to stop testing.
The release Deadline has been met
We can stop testing when the release deadlines have been met. We live in a world
full of constraints, so time like any other resource is constrained too. Every testing
cycle will have a schedule, and we need to stop testing when we reach the end of
The key point here is to plan the testing well so that all the
requirements have been tested and coverage has been done before the
scheduled end date. After all, you won’t like to ship a partly tested product just
because the deadline had arrived.
Too Buggy To Go Deep
One of the interesting answers to the question, when to stop testing is when it is too buggy that you can’t go beyond the first few screens.There will be times when the software will have a lot of bugs on the surface which
will not allow the testing team(s) to go deep into the software or perform more
sophisticated tests. In such cases, even though schedule permits, we might have
to stop (read pause) testing, only till the time blockers or superficial bugs are
solved and it allows us to go deep into the product.
While as tester’s we only have visibility of the product we test, but the company’s
management has the visibility of all the products under its umbrella. The
company’s management may take decisions to maximize the interests of the
the company which could lead to the stopping of testing of the project you are
Some of the common reasons in such cases are – a) Budget or/and resources were
allocated to some other high priority project b) The project was canceled (from the
client’s side or the company’s side) c) The allocated budget is over and no more
budget is being allocated.
Exit Criteria Has Been Met
By far one of the most important criteria to decide testing stop is when exit
criteria are met. However, the key point is here it not just meeting the exit criteria,
the key point here is designing them. Designing exit criteria for any product or
software testing cycle is easier said than done. It is imperative that we consider
multiple factors while designing exit criteria, some of which are mentioned below:
1. Strong buy-in from the development team. In the end, your job is to
discover defects and theirs is to fix.
2. If there are clients who will further test the product, that should also be
considered as to what will be an acceptable input quality.
Here are some examples of exit criteria:
1. When all the planned tests have been run (it could include multiple cycles
of some or all tests as well)
2. When the number of open defects reaches a certain milestone (this could
also be enhanced based on the allowed open defects across various
3. When the planned regression tests have been done.
Exit criteria once designed are not meant to be carved in stone. It should be
reviewed constantly in light of the adherence and the post-release issues which
come after we meet a set exit criterion.
How to track testing progress to meet exit criteria?
While designing and concluding exit criteria is indeed major work done. The next
step is to regularly monitor the progress that the exit criteria are met. As testing
teams, we should understand that if the product is not shipped on time (after
meeting exit criteria), then we might have a severe business impact in terms of
lost sales, competition gaining significant advantages to name a few. It is
extremely important to monitor the testing progress viz-a-viz exit criteria
regularly. Some of the suggested methods of doing it are as follows:
1. Set up a Dashboard that can powerfully summarize all the variables against
2. Set up regular sync-ups with various stakeholders to unblock blocked
people, discuss risks and mitigation plans.
As mentioned earlier, while there is no straightforward answer to when to stop
testing, we hope this blog gives you enough thinking points to decide your exit
criteria and stop testing points.
To read more exciting and insightful blogs related to software testing, visit our page – https://thetesttribe.com/blog/