Acceptance test-driven development
| Part of a series on |
| Software development |
|---|
Acceptance Test-Driven Development (ATDD) is a development methodology based on communication between the business customers, the developers, and the testers.[1] ATDD encompasses many of the same practices as Specification by Example,[2][3] Behavior Driven Development (BDD),[4] Example-Driven Development (EDD),[5] and Story Test-Driven Development (SDD).[6] All these processes aid developers and testers in understanding the customer’s needs prior to implementation and allow customers to be able to converse in their own domain language.
See Acceptance_testing.
ATDD is a development process using Acceptance Tests as its starting point and to establish a common means for all the parties in the development cycle to communicate.
An acceptance test is a black box test. As such, it is impossible to set the system into a given starting state without exercising the system's interface. Consequently, black box testing is a test that exercises the system through multiple states until the desired state is reached, whereupon the subject transition of the test is executed, and the concluding state verified. By necessity, a black box test is a journey, or journey test, and consequently is not a BDD test as a BDD test is only concerned with one transition. (Given a starting state, when a transition occurs, then we have a finished state). It is fair to say that a black box test is multiple BDD tests chained together in the form Given, When, Then, When, Then, When, Then...etc.
It is closely related to TDD only in the way that both tests are defined before the development code is written. In this case, the acceptance test is a story written by the BA, and once the system provides the necessary interface to the testing framework, the story can be translated into the testing infrastructure. Its not related to TDD in the sense that TDD is often used to develop the API interface as the test is written. This is possible with APIs because the test and the code itself is written in a common language. Acceptance tests most often focus on the UI where a human being looks at a web page. Computers can't look at web pages in the same way as humans can, so we use a substitute, typically marks in web pages that only appear if the page is rendered correctly. The markers in the web page are a contract of a kind, but only for the tests, so TDD gives you a valuable interface for a component, but black box testing does not.
Because of the nature of black box testing, the tests are often several orders of magnitude slower to run. Consequently they are few in number. But, they are the most valuable form of test as they most accurately prove that users requirements are met. All other forms of testing are impacted by the chosen architecture that is defined by the developer. Consequently when that architecture is modified or refactored, those tests are subject to change, whereas acceptance tests should only change if requirements have changed.
Overview
Acceptance tests are from the user’s point of view – the external view of the system.[1] They examine externally visible effects, such as specifying the correct output of a system given a particular input. Acceptance tests can verify how the state of something changes, such as an order that goes from “paid” to “shipped”. They also can check the interactions with interfaces of other systems, such as shared databases or web services. In general, they are implementation independent, although automation of them may not be.[7][8]
Creation
Acceptance tests are created when the requirements are analyzed and prior to coding.[1] They can be developed collaboratively by requirement requester (product owner, business analyst, customer representative, etc.), developer, and tester. Developers implement the system using the acceptance tests. Failing tests provide quick feedback that the requirements are not being met. The tests are specified in business domain terms. The terms then form a ubiquitous language that is shared between the customers, developers, and testers.[9] Tests and requirements are interrelated. [10] A requirement that lacks a test may not be implemented properly. A test that does not refer to a requirement is an unneeded test. An acceptance test that is developed after implementation begins represents a new requirement.[11]
Testing Strategy
Acceptance tests are a part of an overall testing strategy. They are the customer tests that demonstrate the business intent of a system. Component tests are technical acceptance tests developed by an architect that specify the behavior of large modules. Unit tests are created by the developer to drive easy-to-maintain code.[12] They are often derived from acceptance tests and unit tests. Cross-functional test include usability,[13] exploratory,[14] and property (scaling and security).[15]
Acceptance criteria and tests
Acceptance criteria are a description of what would be checked by a test. Given a requirement such as “As a user, I want to check out a book from the library”, an acceptance criterion might be “Verify the book is marked as checked out." An acceptance test for this requirement gives the details so that the test can be run with the same effect each time.
Test format
Acceptance tests usually follow this form:[1]
Given (setup)
- A specified state of a system
When (trigger)
- An action or event occurs
Then (verification)
- The state of the system has changed or an output has been produced
For the example requirement, the steps could be listed as:
Given:
- Book that has not been checked out
- User who is registered on the system
When:
- User checks out a book
Then:
- Book is marked as checked out
Complete test
The previous steps do not include any specific example data, so that is added to complete the test:
Given:
- Book that has not been checked out
| Books | |
|---|---|
| Title | Checked out |
| Great book | No |
- User who is registered on the system
| Users |
|---|
| Name |
| Sam |
When:
- User checks out a book
| Checkout action | |||
|---|---|---|---|
| User | Sam | Checks out | Great book |
Then:
- Book is marked as checked out
| Books | ||
|---|---|---|
| Title | Checked out | User |
| Great book | Yes | Sam |
Test examination
Examination of the test with specific data usually leads to many questions. For the sample, these might be:
- What if the book is already checked out?
- What if the book does not exist?
- What if the user is not registered on the system?
- Is there a date that the book is due to be checked-in?
- How many books can a user check out?
These questions help illuminate missing or ambiguous requirements. Additional details such as a due-date can be added to the expected result. Other acceptance tests can check that conditions such as attempting to check out a book that is already checked out produces the expected error.
Another test example
Suppose the business customer wanted a business rule that a user could only check out one book at a time. The following test would demonstrate that:
Scenario:
Check that checkout business rule is enforced
Given:
- Book that has been checked out
| Books | ||
|---|---|---|
| Title | Checked out | User |
| Great book | Yes | Sam |
| Another great book | No |
| Users |
|---|
| Name |
| Sam |
When:
- User checks out another book
| Checkout action | |||
|---|---|---|---|
| User | Sam | Checks out | Another great book |
Then:
- Error occurs
| Error occurred |
|---|
| Description |
| Violation of checkout business rule |
Project acceptance tests
In addition to acceptance tests for requirements, acceptance tests can be used on a project as a whole.[1] For example, if this requirement was part of a library book checkout project, there could be acceptance tests for the whole project. These are often termed SMART objectives. An example test is "When the new library system is in production, the users will be able to check books in and out three times as fast as they do today".
See also
References
- ^ a b c d e Pugh, Ken (2011). Lean-Agile Acceptance Test-Driven Development: Better Software Through Collaboration. Addison-Wesley. ISBN 978-0321714084.
- ^ Adzic, Gojko. (2009) Bridging the Communication Gap: Specification by Example and Agile Acceptance Testing, Neuri Limited,
- ^ Adzic, Gojko (2011). Specification by example: How successful teams deliver the right software. Manning. ISBN 978-0-321-27865-4.
- ^ Chelimsky, David, Dave Astels, Zach Dennis, Aslak Hellesøy, Bryan Helmkamp, and Dan North. The RSpec Book: Behaviour Driven Development with RSpec, Cucumber, and Friends. The Pragmatic Bookshelf.
- ^ "Example Driven Design". Retrieved 2013-04-15.
- ^ "Story Test-Driven Development" (PDF). Retrieved 2013-04-15.
- ^ Melnik, Grigori, and Frank Maurer. "Multiple Perspectives on Executable Acceptance Test-Driven Development". Retrieved 2013-04-15.
- ^ Koskela, Lasse. (2007) Test Driven: TDD and Acceptance TDD for Java Developers. Manning Publications
- ^ Evans, Eric. (2003) Domain-Driven Design: Tackling Complexity in the Heart of Software. Addison-Wesley Professional.
- ^ Weinberg, Gerald; Gause, Donald (1989). Exploring Requirements: Quality Before Design. Dorset House. ISBN 0-932633-13-7.
- ^ Martin, Robert C., and Grigori Melnik."Tests and Requirements, Requirements and Tests: A Möbius Strip" (PDF). Retrieved 2013-04-15.
- ^ [Test-driven_development]
- ^ Meszaros, Gerard, and Janice Aston. (2006) "Adding Usability Testing to an Agile Project."Agile Conference
- ^ "Exploratory Testing Explained" (PDF).
- ^ Meszaros, Gerard.(2007) xUnit Test Patterns: Refactoring Test Code. Addison-Wesley.