Wikipedia:Articles for deletion/Deterministic simulation testing

Deterministic simulation testing

[Hide this box] New to Articles for deletion (AfD)? Read these primers!

(Find sources: Google (books · news · scholar · free images · WP refs) · FENS · JSTOR · TWL)

This term seems to only be used by a fairly small group of companies, I'm not sure the methodology is currently notable enough to warrant an article. WP:BEFORE search turns up a fair amount of results, but they mostly seem to be primary sources or unreliable blogs. I think we need more reliable secondary sources covering this topic before it can be an article.

Considering the article in its current state, I don't think it provides much value as a stub. Every current reference is only indirectly relevant, none speaks directly to the topic or includes the phrase "deterministic simulation testing". There are 2 external links, and only one uses the phrase. StereoFolic (talk) 01:38, 11 December 2024 (UTC)[reply]

Note: This discussion has been included in the list of Computing-related deletion discussions. StereoFolic (talk) 01:38, 11 December 2024 (UTC)[reply]
Delete: There are many Internet forums that can explain this better, even on Reddit, it is explained well at the moment, not here. 190.219.101.225 (talk) 16:47, 13 December 2024 (UTC)[reply]

Mkubler (talk) 18:12, 17 December 2024 (UTC)[reply]

Note: There's now more resources like

As such it seems this should be expanded from a stub as especially the resonatehq blog post seems to explain the concept in detail and the Turso blog post talks about how they are using it for the SQL Lite rewrite in Rust.

Below is a Claude 3.5 Sonnet summary of the concept that could be used:

Deterministic Simulation Testing

Deterministic Simulation Testing (DST) is a software testing methodology primarily used for testing distributed systems and concurrent programs. It enables developers to reproducibly test complex systems by controlling non-deterministic aspects such as timing, randomness, and component interactions within a simulated environment.

Overview

Deterministic Simulation Testing works by running the system under test in a controlled environment where all sources of non-determinism are managed through a simulator. The simulator replaces real-world components like clocks, network connections, and disk operations with deterministic alternatives that can be controlled via a seed value. This approach allows developers to consistently reproduce specific execution scenarios, making it easier to identify and debug issues that might be difficult to catch with traditional testing methods.

Key Concepts

Deterministic Execution

A system's execution is considered deterministic if, given the same initial state, it always produces the same sequence of external events. In DST, this is achieved by:

Controlling random number generation through seeded pseudo-random number generators
Managing time progression through virtual clocks
Simulating network and disk operations with controlled alternatives

System Components

A DST implementation typically divides the system into two main parts:

1. Application: The core business logic being tested 2. Environment: The simulated infrastructure (network, disk, time) that the application interacts with

The interaction between these components occurs through well-defined protocols or APIs, allowing the testing framework to monitor and control all external interactions.

Advantages

Reproducibility: Bugs can be consistently reproduced by using the same seed value that triggered the original failure
Time Control: Tests can simulate long periods of system operation in a fraction of the actual time
Comprehensive Testing: Enables testing of edge cases and failure scenarios that would be difficult to create in real environments
Fault Injection: Allows systematic testing of system behavior under various failure conditions

Limitations

Implementation Complexity: Requires significant architectural changes to make systems deterministic
Platform Constraints: Some programming languages and frameworks make it challenging to control all sources of non-determinism
External System Integration: Cannot fully test interactions with external systems that must be mocked
Testing Scope: Only tests the deterministic core of the system, leaving some edge components untested

Applications

Several notable distributed systems use Deterministic Simulation Testing:

FoundationDB: Pioneered the use of DST in distributed databases, spending 18 months developing their simulation framework before allowing the database to interact with physical storage
TigerBeetle: Employs DST for testing their financial transactions database
WarpStream: Uses DST to test their entire SaaS platform, including control plane and data plane components

Tools and Implementations

Antithesis

Antithesis provides a commercial platform for Deterministic Simulation Testing that includes a custom hypervisor for deterministically simulating Docker containers and injecting faults. It can control thread scheduling, network behavior, and other system-level interactions.

TigerBeetle VOPR

The Viewstamped Operation Replicator (VOPR) is TigerBeetle's deterministic simulator, which focuses on testing consensus and cluster recovery mechanisms. It includes capabilities for fault injection and accelerated time simulation.

History

Deterministic Simulation Testing gained prominence through its successful implementation by FoundationDB in the early 2010s. The approach proved so effective that Kyle Kingsbury, known for the Jepsen distributed systems testing framework, declined to test FoundationDB because their deterministic simulator provided more comprehensive testing than Jepsen could achieve.

References

@todo: Add the links