Test Anything Protocol

The Test Anything Protocol (TAP) is a protocol to allow communication between unit tests and a test harness. It allows individual tests (TAP producers) to communicate test results to the testing harness in a language-agnostic way. Originally developed for unit testing of the Perl interpreter in 1987, producers and parsers are now available for many development platforms.

History

TAP was created for the first version of the Perl programming language (released in 1987), as part of the Perl's core test harness (t/TEST). The Test::Harness module was written by Tim Bunce and Andreas König to allow Perl module authors to take advantage of TAP.

Development of TAP, including standardization of the protocol, writing of test producers and consumers, and evangelizing the language is coordinated at the TestAnything website.

Specification

A formal specification for this protocol exists in the TAP::Spec::Parser and TAP::Parser::Grammar modules. The behavior of the Test::Harness module is the de facto TAP standard implementation, along with a writeup of the specification on http://testanything.org.

A project to produce an IETF standard for TAP was initiated in August 2008, at YAPC::Europe 2008.

Usage examples

Here's an example of TAP's general format:

    1..48
    ok 1 Description # Directive
    # Diagnostic
    ....
    ok 47 Description
    ok 48 Description

For example, a test file's output might look like:

    1..4
    ok 1 - Input file opened
    not ok 2 - First line of the input valid.
        More output from test 2. There can be
        arbitrary number of lines for any output
        so long as there is at least some kind
        of whitespace at beginning of line.
    ok 3 - Read the rest of the file
    #TAP meta information
    not ok 4 - Summarized correctly # TODO Not written yet

List of TAP parsers

These are libraries which parse TAP and display the results.

• The t/TEST parser contained in the Perl source code.
• Test::Harness is the new and more flexible parser written by Curtis "Ovid" Poe, Andy Armstrong and other people. It is the successor of the original Test::Harness 2.64, which is the oldest and most complete TAP parser. It is limited in how it displays TAP. Though it most often runs tests written in Perl, it can launch any process which generates TAP. Most of the TAP spec is taken from the behavior of Test::Harness. The new Test::Harness is a wrapper around TAP::Parser and provides a minimal compatibility layer with previous behavior, but any new development should use the TAP::Harness module instead.
• Test::Run is a fork of Test::Harness being written by Shlomi Fish.
• test-harness.php A TAP parser for PHP.
• nqpTAP A TAP parser written in NotQuitePerl (NQP), a smaller subset of the Perl 6 language.
• Tapir A TAP parser written in Parrot Intermediate Representation (PIR).
• tap4j A TAP implementation for Java.
• C TAP Harness A TAP parser for C
• TAP.py for Python, using PyParsing
• simple-tap-parser A TAP parser for Javascript

List of TAP producers

These are libraries for writing tests which output TAP.

• Test::More is the most popular testing module for Perl 5.
• Test::Most puts the most commonly used Perl 5 testing modules needed in one place. It is a superset of Test::More.
• PHPUnit is the xUnit implementation for PHP.
• test-more.php is a testing module for PHP based on Test::More.
• test-more-php implements Test::Simple & Test::More for PHP.
• libtap is a C library to produce TAP
• libtap is a TAP producer written in C.
• The Comprehensive C Archive Network has a C tap module based on libtap.
• There's a more actively maintained version of libtap by Shlomi Fish.
• libtap++ is a TAP producer for C++
• ArduinoTap is a TAP producer written in C++ for Arduino board.
• Test.Simple is a port of the Perl Test::Simple and Test::More modules to JavaScript by David Wheeler.
• MATLAB's unit test framework includes a TAPPlugin.
• PyTAP A beginning TAP implementation for Python.
• MyTAP MySQL unit test library used for writing TAP producers in C or C++
• Bacon A Ruby library that supports a spec-based syntax and that can produce TAP output
• PLUTO PL/SQL unit testing for Oracle
• pgTAP PostgreSQL stored procedures that emit TAP
• SnapTest A PHP unit testing framework with TAP v13 compliant output.
• etap is a simple Erlang testing library that provides TAP compliant output.
• lua-TestMore is a port of the Perl Test::More framework to Lua.
• tap4j A TAP implementation for Java.
• lime A testing framework bundled with the Symfony PHP framework.
• yuitest A JavaScript testing library (standalone)
• Ahven An Ada unit testing library.
• Mocha A JavaScript testing library
• Jasmine A JavaScript testing library (through the TapReporter).
• bash-test-utils A Bash testing library (incl. Linux/Xen/KVM support)
• Sharness A shell test harness, derived from the Git project's test harness
• node-tap A TAP framework for Node.js
• busted is a unit testing framework for Lua
• test-tap is a unit testing for JavaScript (browser and CLI)
• cl-test-more is yet another unit testing library for Common Lisp
• bats is a TAP-compliant testing framework for Bash. It provides a simple way to verify that the UNIX programs you write behave as expected.

External links

• http://testanything.org/ is a site dedicated to the discussion, development and promotion of TAP.