User:J-at-ywalters-dot-net/sandbox

J-at-ywalters-dot-net (j@ywalters.net) sandbox page

Open box testing

Illustration of Open Box Testing discipline

Open box testing (also known as white box testing, clear box testing, glass box testing, transparent box testing, and structural testing) describes a software validation and verification discipline that is typically performed by the application engineers that have directly contributed to a project's source code. Individuals that have direct access to source code are considered to be able to "see within the open box" and with that ability comes a significant understanding of the computer programming within the project. The ability to directly read and understand the source code enables certain activities but also creates some bias challenges.

This term is an attempt to provide a more practical description and illustration of this software testing discipline as well as an intentional pivot away from the racial connotations of the long used "white box" and "black box" terminology.

Types of Open box testing

The following, commonly referenced types of testing activities often comprise open box testing disciplines within organizations that are following modern software engineering practices:

Unit testing

• A focus on and ability to verify the smallest piece of code that can be logically isolated is consistently working as designed and expected.

Component testing

• This discipline, also known as program or module testing, involves verifying that larger, individual components and encompass multiple smaller units of work are working correctly in isolation from the entirety of the overall project. This practice is performed after the smaller and often considerably faster unit tests.

Integration testing

• individual software modules are combined and tested as a group

System testing

• verification of a completely integrated system

In practice

Practitioners of open box testing disciplines are subject to various cognitive biases, particularly confirmation bias, as it is impractical to have people objectively evaluate the quality of work that they have done. Third party, unprejudiced validations of the work of others is a long determined best practice throughout many industries.

We suck at testing our own code. We suck so badly at it that it has led to entire professions like as Quality Assurance analysts and test engineers. We're not necessarily bad at coding, but we're notoriously bad at finding our own bugs.
– Confirmation Bias: How your brain wants to wreck your code

The ability to review the specifications, designs and coding implementations that comprise the internal logic and structure of the underlying application enables teams to create sustainable projects that support the addition of, or transition to, future project contributors.

All software testing disciplines involve identifying flaws and errors in the application code that must be fixed. The processes that are undertaken provide for confidence that the functionality and correctness of a software has been analyzed and verified to be correct.

Shift-left testing

The open box testing disciplines are among the central tenets of the Shift-left testing approach to software development. As part of this "test early and often" modern and progressive approach, developer responsibilities are incorporated into the overall testing cycle earlier than ever before. Focusing on finding and remediating software defects as early as possible within the Software Development Life Cycle (SDLC) has profound benefits to organizations that support it because quality is clearly acknowledged as a shared responsibility and testing is prevalent throughout the process. Significant cost savings over time are often the return on investment achieved for teams that are highly successful at these practices but it is important that people do not try to “boil the ocean” and achieve unreasonable levels of test coverage for their projects.

Open box testing activities are typically performed prior to an official, fully integrated installation of a piece of software and the start of an official, objective testing cycle begins. Post-installation, a wide variety of closed box testing disciplines are performed by a variety of different types of software testing professionals.

Illustration of Closed Box Testing discipline

Agile software practices

Within Agile software development enabled projects, teams are asked to reduce the length of time of software delivery while continuously improving quality of each release of their projects. At the same time, there is typically an increased pressure to reduce testing costs.

Highly capable application engineers that demonstrate significant open box testing skills typically produce well designed units of work that are well covered with meaningful, open box type tests. Investments in these types of tests provide for the long term reliability, maintainability and comprehensive documentation of the expected functionality within the project by ensuring that the units of work continue to function correctly over time.

Numerous surveys and studies over the past decade illustrate that software engineers frequently spend large portions of their time working with, maintaining and needing to improving existing code. Software maintenance related code changes are particularly risky when project contributors do not have the appropriate level of open box type tests in place that demonstrate that software functionality that was working correctly before the change continues to work correctly after the change.

Continuous integration

In software engineering, continuous integration (CI) means the repeated application of quality control processes on every small discrete change or addition. A fundamental tenet of this practice is that the project's automated tests are first run locally then run on a remote system to provide fast feedback to the project's contributors and prevent the advancement of code that is known to be functioning incorrectly. Open box tests being the first line of defense to ensure that code changes did not change the overall program in unintended ways. The intertwined nature of open box testing disciplines and all derivatives of continuous integration practices including have led to a wide variety of articles about how the process does not work without open box type tests. Articles like "Continuous Integration is Absurd without Unit Testing" and "Unit Tests, How to Write Testable Code and Why it Matters" illustrate how these two disciplines are inextricably linked.

See also

• White box testing
• Component testing
• Integration testing
• System testing
• Test driven development

Illustration of Open Box and Closed Box Testing disciplines

Illustration of Closed Box and Open Box Testing disciplines