It is important to check that the server response is valid under load. This is to make sure not only that the scenario works as expected, but also that the load imposed does not cause errors in the application
NeoLoad automatically detects requests containing errors, in particular using the HTTP response code. For example, NeoLoad will log a request error if the response contains '500 internal error'. However, many web applications do not return the appropriate HTTP error code when handling an error, which means that for these, NeoLoad cannot automatically detect the fault.
These cases must be processed individually by checking the validity of the content returned by the server at key points within the application. This may entail checking for the presence of an expected text, for example "The operation was successful", or making sure the response does not contain "Error".
To define a container value:
In the Virtual User profiles, click on the request whose server response requires validating.
Click on the "Validation..." button
underneath, to the right.
Add a Validation for the content.
Select a validation method: contains or does not contain a simple text, or use a regular expression.
Once the test is finished, select the requests containing errors, or those where the validation failed, in the errors panel. The content of the corresponding server response may then be analyzed to determine the cause of the problem.
Note that validation uses up resources (CPU, memory) on the Load generator, and that it is usually wiser to restrict its use to testing key pages only (for example, those that provide access to a database, or those more likely to produce errors).
Proceed step by step:
Use the Check
function to validate the scenario for a Virtual User type. This will
play the Virtual User once, displaying the details of the sent
request and the server response to each HTTP request.
Begin with a short test and a light load. Correct any application errors occurring under the light load before carrying out tests using heavier loads.
When a Virtual User receives an error, it should normally stop running. If this does not happen, it could continue playing requests that have no meaning. For example, if the user login fails, there is little point sending further browsing or search requests to the application as it will only distort the response time statistics for those pages.
Each Virtual User type may be configured to stop running in case of error or failed assertion.
Each transaction (registration, on-line purchase...) may be composed of several web pages. Statistics may be obtained by transaction by grouping these pages in a Container.
NeoLoad allows you to retrieve performance data from most server infrastructures using monitors. Configure monitors on all your infrastructure's key servers to monitor: CPU, memory, hard disk and network usage on the web, application and database servers. See Chapter 7, Monitors.
An efficient way to highlight the performance variations between two sets of test results is to plot the statistics for a same item (page, request, Container) in both tests simultaneously.
This provides a visual comparison of the application's behavior under different scenarios, or further to its modification (e.g. update or optimization).
An efficient way to pinpoint performance problems is to filter a test's results. The aim is to limit the test statistics to the items (request, page, container, Virtual User, Population..) that are showing the problems.
For example, you may narrow down the statistics to a specified time period during the test run; this will display the statistics as if the test had been carried out over that time period only.
NeoLoad provides two advanced statistics:
Standard deviation - allows the variation in values to be measured, compared to the average. A high standard deviation indicates that response times vary widely, whereas a low standard variation is a sign that response times are consistent and even.
Average 90% - calculated by removing the highest and lowest 5% of values. This eliminates any outliers and produces values that are close to those that will be obtained by most end users.
The statistics that show anomalies, such as a significant rise in response times or the occurrence of errors, can be correlated with the variations in the readings obtained by certain performance monitors.
These correlations will usually provide an explanation for the performance slowdown and give a clue to the main cause of the problem, be it merely a server setting or the overload of one of the main resources (memory for example).
After multiple test runs, the volume of data can become difficult to manage. It is important therefore to add a short description to each test prior to its running. This description is included in each test summary and the reports generated.
The Results Manager allows the user to delete the results of a previous test session or use them to generate a report (XML, HTML, PDF).