Application performance engineering

Application performance engineering is a method to develop and test application performance in various settings, including mobile computing, the cloud, and conventional information technology (IT).

APE Methodology

According to the National Institute of Standards and Technology, nearly four out of every five dollars spent on the total cost of ownership of an application is directly attributable to finding and fixing issues post-deployment. A full one-third of this cost could be avoided with better software testing.^[1] The value or return on investment for proper performance testing is not dependent on just avoiding costs. Business benefits must also be considered, as performance testing leads to improved client satisfaction, increased sales, increased employee productivity and helps avoid the cost of post-deployment issues.^[2]

An effective Application Performance Engineering strategy enables rapid remediation and performance optimization in the pre-deployment stages of the application development lifecycle, when it is most cost-effective to do so.^[3] While APE practices vary from organization to organization, the core of APE is the accurate emulation or recreation of real-world conditions, especially those introduced when the use of an application or access to data involves traversing a network. These conditions can have an especially significant impact when end users are gaining access to a system via a mobile device.^[4] In the case of mobile access, the impact of the mobile network not only effects the experience of mobile users but will likely impact all users as a result of the impact of mobile network latency on back-end infrastructure and user sessions. Therefore, Network virtualization is an essential component of APE as it enables a reliable emulation of the network, creating an accurately predictive performance test environment.

Within the APE methodology, there are several foundational levels or building blocks that comprise various best practices and technologies: Discovery, Network Virtualization and Testing, and Analysis and Optimization.^[5]

Discovery

The Discovery level of APE encompasses those technologies, services and methodologies designed to provide insight into the actual production environment experienced by end users. It includes awareness of user locations, user devices, application dependencies such as third-party content delivery networks and services, business processes, service level objectives and the actual network conditions.

Understanding these actual usage conditions is critical to predicting user experience and establishing and validating baseline performance levels. The rise of Mobile introduces a new variant of APE and Discovery since new factors such as type of network connection (2.5G, 3G, 4G, LTE), time of day and distance from a cellular tower must also be discovered and taken into consideration when evaluating application performance.

Discovery provides insight into the conditions experienced by users in different locations. From a network perspective, discovery includes metrics such as bandwidth availability, latency, jitter and packet loss. Each of these metrics are required in order to effectively employ the next stage of APE, Network Virtualization and Testing.

Network Virtualization & Testing

A failure in traditional application testing has been a lack of a true representation of the real-world conditions affecting distributed end users. Effective Discovery enables accurate emulation of real-world application usage scenarios within the test environment. Devices, services (see Service Virtualization) and load (see Load Testing) can be brought into the test environment, but connections to and between those elements must be reflective of the real-world network.

Network Virtualization is the technology and process by which those connections are created and made to represent the real world. In the absence of real-world conditions, testing will yield unreliable results because the effect of the network on the end user experience, as well as communication with external services, remains unaccounted for.

With the massive global uptake in mobile devices and the rise of cloud computing, the effect of the network has become even more pronounced since conditions across the “last mile” affect performance more than other factors, and mobile conditions in particular are intrinsically dynamic. Therefore, incorporating network virtualization is particularly crucial in any mobile or cloud deployment.

Once the test environment has been appropriately configured to represent the real world, testing takes place. Within the APE methodology, performance testing is integrated throughout the application life cycle and testing process, including single user, load, scalability, capacity and functional testing. Performance testing in this manner delivers a definitive quality assurance test that ensures adherence to performance goals or Service Level Objectives (SLOs). Note that testing is not a one-time occurrence. Since the demand and the performance of the production network will vary over time, especially with mobile, testing must be repeated with modified demands and network characteristics.

Analysis & Optimization

Care is taken in discovering and setting up test conditions so that analysis of test data can be as accurate and as predictive of real-world application behavior as possible. Within the Application Performance Engineering framework, performance analysis will allow organizations to: • Predict and validate performance levels • Determine compliance with Service Level Objectives (SLOs) or Service Level Agreements (SLAs) • Identify, diagnose and remediate the root cause of performance bottlenecks • Identify areas for performance optimization

Once functional and non-functional performance objectives have been tested, the application response characteristics can be measured and key bottlenecks analyzed. This information must be location-specific so testers and performance engineers can identify poorly performing business transactions and the root cause of performance issues for each user population and/or network. In addition, further test results segmentation may be required; for example, analyzing results per device type in a mobile app testing scenario might be needed to understand how end user experience varies by device or mobile OS.

Isolating and remediating performance issues leads to another step at this level of APE – validation. Remediation efforts must be validated by re-testing under the precise same conditions as the initial test. Network virtualization enables the same network characteristics to be applied to the test environment, so test results reliably indicate performance gains and/or new issues that must be analyzed.

In addition to remediation, optimization efforts may also improve performance even when performance issues are not evident. By exploring development and deployment best practices, including engaging third-party products and services, additional performance gains can be achieved. For example, network accelerators and application delivery controllers may offer potential performance benefits. By evaluating these types of solutions and testing their impact on performance under precisely the same conditions under which an application was initially tested, a determination can be made as to which products (if any) offer the largest performance gains and at what cost. Price per performance gain analysis ensures optimization dollars are best allocated.