Test Anything Protocol

The Test Anything Protocol (TAP) is a general purpose format for recording the result of tests primarily used by Perl modules.

TAP's general format is:

   1..N
   ok 1 Description # Directive
   # Diagnostic
   ....
   ok 47 Description
   ok 48 Description
   more tests....

For example, a test file's output might look like:

   1..4
   ok 1 - Input file opened
   not ok 2 - First line of the input valid
   ok 3 - Read the rest of the file
   not ok 4 - Summarized correctly # TODO Not written yet

TAP separates the test program from the thing which interprets the results using simple text pipelining. This allows one to run several tests isolated in different processes together in one harness. It allows one to write tests in different languages and run them together with unified results. It also allows the runner of the tests to decide how the results will be displayed, rather than the author of the tests. The results can be transformed by the runner into other formats (for example, XML) or even cause actions to happen (a smoke test server can send email on failure).

TAP is designed to be human readable, the simple "ok", "not ok" output and descriptions do not require a parser or even extensive knowledge of TAP for the basic intention of each line to be understood by the user.

TAP is designed to be portable, its plain text format means it will work with any language or operating system.

TAP is designed to be streamable, the results can be interpreted a line at a time without having to wait for the test to complete.

Specification

Despite its being almost 20 years old and in very wide use it is only now being formally specified. This is to allow for adoptation by other languages, the writing of additional parsers and extension of TAP. The current spec is the TAP module on CPAN. This spec is under development and incomplete. Where the spec is lacking the behavior of Test::Harness should be used as a guide.

A formal grammar is in development.

History

TAP came into being with the first version of Perl (1987) although it wasn't called anything then. For most of its existence the only TAP parser was Perl's core test harness (t/TEST) written by Larry Wall. When Perl 5 introduced modules, Test::Harness was written by Tim Bunce and Andreas König allowing Perl module authors to take advantage of TAP. Testing has become very popular in Perl in the last few years and Perl authors working with other languages wishing to take their tools with them have begun to write TAP parsers and TAP-based testing libraries in other languages.

TAP Parsers

These are libraries which parse TAP and display the results.

• Test::Harness is the oldest and most complete TAP parser. It is limited in how it displays TAP and can only run tests written in Perl. Most of the TAP spec is taken from the behavior of Test::Harness.
• The t/TEST parser contained in the Perl source code.
• TAPx::Parser is a new and more flexible parser being written by Curtis Poe.
• Test::Run is a fork of Test::Harness being written by Shlomi Fish.

TAP Producers

These are libraries for writing tests which output TAP.

• Test::More is the most popular testing module for Perl.
• libtap is a TAP parser written in C.
• Test.Simple is a TAP parser written in Javascript by David Wheeler.