A place to discuss acceptance testing
Acceptance testing tests an application for conformance to specification and/or contractual obligations. Creating acceptance tests makes it less likely that the developer and client will have different opinions as to what constitutes "meeting requirements".
From Managing The Testing Process:
A software or hardware development test phase designed to demonstrate that the system under test meets requirements. This phase is unique in all test activities in that its purpose is to demonstrate sufficiency and correctness, not to find problems. Acceptance testing is usually the last test phase before a product is released
Has the plan for acceptance testing been submitted?
Have all possible interactions been described?
Are all input data required for testing available?
Is it possible to automatically document the test runs?
Have the customer specific constraints been considered?
Have you defined acceptance criteria (e.g. performance, portability, throughput, etc.) on which the completion of the acceptance test will be judged?
Has the method of handling problems detected during acceptance testing and their disposition been agreed between you and the customer?
Have you defined the testing procedure, e.g. benchmark test?
Have you designed test cases to discover contradictions between the software product and the requirements, if existent?
Have you established test cases to review if timing constraints are met by the system?
Tests should be free of jargon, using screen shots instead of descriptions wherever possible.
Tests should be stored in a way that makes them easy to review and modify. If you're not an expert in creating Word documents that can be easily navigated, consider a tool like Acceptance Test Builder (written in FoxPro) from Affordable Custom Software.
Don't wait until the product is finished: Although testing has to wait until the product is ready, tests should be built (and revised) along with each user story.
Test Execution and Evaluation checklist
Has the acceptance test been performed according to the test plan?
Have all steps of the test run been documented?
Have the users reviewed the test results?
Are the services provided by the system conform to user requirements stated before?
Have the users judged about acceptability according to the predetermined criteria?
Has the user signed-off on output?
Although it's possible to create automated tests (see FIT and FitNesse), they can't completely take the place of human testing, for the following reasons:
Each automated test is itself a program, which introduces a new opportunity for error.
Appropriate look-and-feel is usually a requirement, but it's too subjective to be tested automatically.
In many cases, the benefits of automated testing won't compensate for the time it takes to write the testing code.
Changing purpose of Acceptance Testing
If a customer finds too many issues during acceptance test execution, the customer may feel that they have been taken hostage by the contract. Even if the system does not meet the acceptance criteria, it is needed for use. In order to prevent this, the customer should state requirements to supplier quality assurance and testing and follow them up. The customer should also make the supplier include possible acceptance test scenarios in his/her system testing. Problems should, in general, not be found during acceptance testing, but should have been found before!
Contributors: Steven Black, Hans Schaefer, Jim Wiggins
Category Testing Category Check Lists
( Topic last updated: 2012.06.23 06:55:14 PM )