Scalability testing
Scalability Testing, is the testing of a software application to measure its capability to scale up or scale out in terms of any of its non-functional capability.
Performance, scalability and reliability testing are usually grouped together by software quality analysts.
The primary goals of scalability testing are to determine the user limit for the Web application, determine and end user experience under load, for example, can users access a web page in a timely manner. Another goal is to determine server-side robustness and degradation i.e. Will the server crash if it is under a heavy load? [1]
Dependent on the application that is being tested, different parameters are tested. If a webpage is being tested, the number of simultaneous users making requests of the site would be tested.[2] Also dependent on the application being tested is the attributes that are tested - these can include CPU usage, network usage or user experience [3] which can be tested by response times in the case of web pages.
Successful testing will project most of the issues which could be related to the network, database or hardware/software.[4]
How to Create a Scalability Test
When creating a new application, it is difficult to know the number of users it will have at any given time. Although an estimate can be made, it is not a definite number. An issue with an increasing number of users is that it can create new areas of failure. For example, if you have 10,000 new visitors, it’s not just access to an application that could be a problem; you might also experience issues with the database where you need to store all the data on these new customers.[5]
This is why when creating a scalability test, it is important to scale up in increments. These steps can be split into small, medium and high loads.
The environment should be constant throughout testing in order to provide accurate and reliable results. If the testing is a success, we should see a proportional and linear change in performance. For example, if we double the users on the system, we should see a drop in user end performance of 50% [6]
Alternatively, if measuring system statistics such as memory or CPU usage over time, this may have a different graph that is not proportional as users are not being plotted on either axis.
Outcomes of Scalability Testing
Once we have collected the data from our various stages, we can begin to plot the results on various graphs to show the results. However, the graphs can vary depending on what is being plotted. In Figure 1[7], we can see a graph showing a resources usage (in this case,memory) over time. The graph is not proportionate but can still be considered a passed test as initially there is a ramp up phase as the system beings to run, however, as more users are added, there is little change in memory usage. This means that the current memory capacity can cope with all 3 stages of the test.
In figure 2, we can see a more proportional increase, comparing the number of users to the time taken to execute a report. With a low load of 20 users, the average time is 5.5 seconds, as we increase the load to medium (40 users) and a high load (60 users), the average time increases to 9.5 and 18 seconds respectively [6].
In some cases, there may be changes that have to be made to the server software or hardware. This can include improving algorithms, employing data caching or upgrading CPU/RAM. Once the necessary upgrades have been made, we must re-run the tests to ensure the upgrades are effective.[8]
Vertical and Horizontal Scaling
As a result of scalability testing, upgrades can be required to software and hardware. These upgrades can be split into vertical or horizontal scaling.
Vertical scaling, also known as scaling up, is the process of replacing a component or server with a more powerful, better equipped device. For example, replacing a processor with a faster one. Horizontal scaling, also known as scaling out is replicating the device/server so it runs in parallel to the original. For example, adding a new server for use alongside the current one. [9]
There are advantages and disadvantages to both methods of scaling. Although scaling up may be simpler, the addition of hardware resources can result in diminishing returns. [10] By this, we mean every time we change the processor for example, we do not always get the same level of benefits as the previous change. However, vertical scaling can be extremely costly, not only with the purchase of entire systems such as servers, but their maintenance costs also. [10]
References
- ^ "Planning for Load Testing". docs.oracle.com. Retrieved 2015-10-23.
- ^ "Scalability Testing". Performance Blog. Retrieved 2015-10-25.
- ^ "What is Scalability Testing? | Software Testing Class". www.softwaretestingclass.com. Retrieved 2015-10-25.
- ^ Joshi, Prateek. "Why Do We Need Performance Testing?". Perpetual Enigma. Retrieved 2015-10-25.
- ^ "Discovering the right metrics for scalability testing". www.theserverside.com. Retrieved 2015-10-25.
- ^ a b "IBM Cognos Proven Practices: Designing a Successful Performance and Scalability Test for IBM Cognos BI". www.ibm.com. 2011-11-17. Retrieved 2015-10-25.
- ^ "Enterprise performance and test results" (PDF). Serena. 2011.
- ^ "Scalability Testing: Checking Whether a Site Performance Can Scale Up". support.smartbear.com. Retrieved 2015-10-28.
- ^ Bondi, André (2014). Foundations of Software and System Performance Engineering: Process, Performance Modeling, Requirements, Testing, Scalability, and Practice. Section 11.2. ISBN 0321833821.
- ^ a b "Scalability Testing" (PDF). Comp Nus Education.
External links
 | This software article is a stub. You can help Wikipedia by expanding it. |