Application performance engineering

Application Performance Engineering (APE) is a specific methodology designed to meet the challenges associated with application performance in increasingly distributed mobile, cloud and terrestrial IT environments. It includes the roles, skills, activities, practices, tools and deliverables applied at every phase of the application lifecycle that ensure an application will be designed, implemented and operationally supported to meet non-functional performance requirements. APE is the recognition of the fact that IT organizations need to design for application performance and then test, measure and tune performance throughout the application lifecycle.

Application Performance Engineering vs. Application Performance Monitoring

Application Performance Engineering differs from Application Performance Monitoring (APM), though efforts associated with both are essential to ensuring a quality end user experience. Whereas APM tends to monitor and report on actual end user experience, APE is primarily a pre-production set of technologies and best practices aimed at validating and optimizing expected performance prior to deployment in order to proactively avoid costly performance incidents.

Over the past decade, organizations have placed increasing emphasis on ensuring application delivery. The result was a focus on APM and network and application optimization (NAO). Both of these approaches are generally reactive in nature as they are employed after an application is developed or deployed. As application infrastructure has become more complex and businesses have greater realization of the importance infrastructure plays in supporting business processes, IT management focus has necessarily shifted to proactive performance testing and validation. This has given rise to the set of best practices, technologies, and skills that make up application performance engineering.^[1]

That is not to say that APE is a replacement for APM. Both are needed. Together, APE and APM create a continuous feedback loop of performance information: APE leverages real-world application behavior in order to establish benchmarks for performance and service levels; APM leverages APE test results to determine the thresholds at which monitoring alarms are triggered.

APE Methodology

According to the National Institute of Standards and Technology, nearly four out of every five dollars spent on the total cost of ownership of an application is directly attributable to finding and fixing issues post-deployment. A full one-third of this cost could be avoided with better software testing.^[2] By adopting APE best practices and incorporating performance testing into the earliest stages of the software development life cycle, organizations have reported a 25% reduction of performance incidents occurring post-deployment.^[3]

An effective Application Performance Engineering strategy enables rapid remediation and performance optimization in the pre-deployment stages of the application development lifecycle, when it is most cost-effective to do so.^[4] While APE practices vary from organization to organization, the core of APE is the accurate emulation or recreation of real-world conditions, especially those introduced when the use of an application or access to data involves traversing a network. These conditions can have an especially significant impact when end users are gaining access to a system via a mobile device. In the case of mobile access, the impact of the mobile network not only effects the experience of mobile users but will likely impact all users as a result of the impact of mobile network latency on back-end infrastructure and user sessions. Therefore, Network virtualization is an essential component of APE as it enables a reliable emulation of the network, creating an accurately predictive performance test environment.

Within the APE methodology, there are three foundational levels or building blocks that comprise various best practices and technologies: Discovery, Network Virtualization and Testing, and Analysis and Optimization.

Discovery

The Discovery level of APE encompasses those technologies, services and methodologies designed to provide insight into the actual production environment experienced by end users. It includes awareness of user locations, user devices, application dependencies such as third-party content delivery networks and services, business processes, service level objectives and the actual network conditions.

Understanding these actual usage conditions is critical to predicting user experience and establishing and validating baseline performance levels. The rise of Mobile introduces a new variant of APE and Discovery since new factors such as type of network connection (2.5G, 3G, 4G, LTE), time of day and distance from a cellular tower must also be discovered and taken into consideration when evaluating application performance.

Discovery provides insight into the conditions experienced by users in different locations. From a network perspective, discovery includes metrics such as bandwidth availability, latency, jitter and packet loss. Each of these metrics are required in order to effectively employ the next stage of APE, Network Virtualization and Testing.

Network Virtualization & Testing

A failure in traditional application testing has been a lack of a true representation of the real-world conditions affecting distributed end users. Effective Discovery enables accurate emulation of real-world application usage scenarios within the test environment. Devices, services (see Service Virtualization) and load (see Load Testing) can be brought into the test environment, but connections to and between those elements must be reflective of the real-world network.

Network Virtualization is the technology and process by which those connections are created and made to represent the real world. In the absence of real-world conditions, testing will yield unreliable results because the effect of the network on the end user experience, as well as communication with external services, remains unaccounted for.

With the massive global uptake in mobile devices and the rise of cloud computing, the effect of the network has become even more pronounced since conditions across the “last mile” affect performance more than other factors, and mobile conditions in particular are intrinsically dynamic. Therefore, incorporating network virtualization is particularly crucial in any mobile or cloud deployment.

Once the test environment has been appropriately configured to represent the real world, testing takes place. Within the APE methodology, performance testing is integrated throughout the application life cycle and testing process, including single user, load, scalability, capacity and functional testing. Performance testing in this manner delivers a definitive quality assurance test that ensures adherence to performance goals or Service Level Objectives (SLOs). Note that testing is not a one-time occurrence. Since the demand and the performance of the production network will vary over time, especially with mobile, testing must be repeated with modified demands and network characteristics.

Analysis & Optimization

Care is taken in discovering and setting up test conditions so that analysis of test data can be as accurate and as predictive of real-world application behavior as possible. Within the Application Performance Engineering framework, performance analysis will allow organizations to: • Predict and validate performance levels • Determine compliance with Service Level Objectives (SLOs) or Service Level Agreements (SLAs) • Identify, diagnose and remediate the root cause of performance bottlenecks • Identify areas for performance optimization

Once functional and non-functional performance objectives have been tested, the application response characteristics can be measured and key bottlenecks analyzed. This information must be location-specific so testers and performance engineers can identify poorly performing business transactions and the root cause of performance issues for each user population and/or network. In addition, further test results segmentation may be required; for example, analyzing results per device type in a mobile app testing scenario might be needed to understand how end user experience varies by device or mobile OS.

Isolating and remediating performance issues leads to another step at this level of APE – validation. Remediation efforts must be validated by re-testing under the precise same conditions as the initial test. Network virtualization enables the same network characteristics to be applied to the test environment, so test results reliably indicate performance gains and/or new issues that must be analyzed.

In addition to remediation, optimization efforts may also improve performance even when performance issues are not evident. By exploring development and deployment best practices, including engaging third-party products and services, additional performance gains can be achieved. For example, network accelerators and application delivery controllers may offer potential performance benefits. By evaluating these types of solutions and testing their impact on performance under precisely the same conditions under which an application was initially tested, a determination can be made as to which products (if any) offer the largest performance gains and at what cost. Price per performance gain analysis ensures optimization dollars are best allocated.