Software testing controversies

There is considerable variety among software testing writers and consultants about what constitutes responsible software testing. Prominent members of the Context-Driven School of Testing consider much of the writing about software testing to be doctrine, mythology, and folklore. Some contend that this belief directly contradicts standards such as the IEEE 829 test documentation standard, and organizations such as the Food and Drug Administration who promote them. The Context-Driven School's retort is that Lessons Learned in Software Testing includes one lesson supporting the use IEEE 829 and another opposing it; that not all software testing occurs in a regulated environment and that practices appropriate for such environments would be ruinously expensive, unnecessary, and inappropriate for other contexts; and that in any case the FDA generally promotes the principle of the least burdensome approach.

Many members of the Context-Driven School of Testing believe that there are no best practices of testing, but rather that testing is a set of skills that allow the tester to select or invent testing practices to suit each unique situation. James Bach wrote "...there is no practice that is better than all other possible practices, regardless of the context." However, some testing practitioners do not see an issue with the concept of "best practices" and do not believe that term implies that a practice is universally applicable.

Some writers believe that test automation is so expensive relative to its value that it should be used sparingly. Others, such as advocates of agile development, recommend automating 100% of all tests. A challenge with automation is that automated testing requires automated test oracles (an oracle is a mechanism or principle by which a problem in the software can be recognized).