Viewing Continuous Testing Results

Overview

You can view the results of individual Continuous Tests and compare up to five Continuous Test instances. The Continuous Test Results page provides a comprehensive overview of measurements, including charts, graphs, and detailed events for each Test. You can configure and download SLA reports and view Events. You can also download single Test results as PDF reports for easy sharing and analysis. See the instructions below for more details.

Single Test

You can start looking at the Continuous Test results at any time. For this, in the Login Enterprise sidebar menu > Results, navigate to Continuous testing.

Single Test Summary

This page displays a list of previously executed Continuous Tests. The table is sorted by default on “Login failures,” and includes the following:

All results displayed on the page are filtered based on the selected time span. Use the date picker in the top right corner to select any time range. Default setting: Last 1 hour.

Search results dynamically filter the table content. Use the search field to find tests by Test Name and/or Description.

There are two additional tabs available for extended functionality:

• SLA Reports

• Export Data

See the SLA Reports and Export Data sections for details.

Single Test Results

To view the test results for a single test, select a test and click View.

The following page displays the Test summary, Charts (EUX metrics, Latency, Logins, Applications, and Session metrics), Application Performance, and Events.

For the Charts section, you can select any time span using the date picker. Based on the selected time span, different aggregation options are available. For details, see Aggregation options.

Application Performance

Use the Application performance tab to review performance details for applications in a Continuous Test result. The tab shows a list of applications included in the test. For each application, you can review:

• Above-threshold measurements compared with total measurements

• Failure rate

• Status bar showing failed and successful measurements

• Success rate

Select an application to open the application details page.

The application details page shows performance data for the selected application. The statistics table includes:

• Above-threshold measurements compared with total measurements

• Failure rate

• Status bar showing failed and successful measurements

• Success rate

Select the row in the statistics table to open the related events list.

The application details page also includes charts that show measurements for the selected application.

Generating a Test PDF Report

You can generate a single Continuous Test PDF report at any time. To generate the report:

1. In the Continuous Testing results, select the checkbox next to the test you’re interested in.

2. Click Generate report in the top menu toolbar.

3. In the Generate PDF report pop-up, configure the report according to your needs by ticking off the info you want to include. For details and examples, see Configuring a Test PDF report.

4. Click Generate, and then Save.

To learn about what the Test PDF report consists of, see the Test PDF Report Breakdown.

Compared Tests

Compared Test Summary

This page displays a list of compared Continuous Tests. The information you’re viewing is the same as in the single Continuous Test summary. For details, see the Single Test Summary.

Compared Test Results

To view the test results for the compared tests, select up to 5 tests and click Compare.

The following page displays the Test summary, Charts (EUX metrics, Latency, Logins, Applications, and Session metrics), and Events.

For the Charts section, you can select any time span using the date picker. Based on the selected time span, different aggregation options become available. For details, see Aggregation options.

Aggregation Options

EUX Metrics

This is where you can see the EUX score and EUX timers. Available EUX timers:

• Generic Applications score

• CPU score

• My Documents I/O score

• Local AppData I/O score

• Mixed CPU I/O score

• User input score

Here’s an example of an EUX score chart:

For more information on the EUX score, see Calculating the EUX score and VSImax.

Latency

This is where you can see the latency metrics.

To learn more about latency, see Monitoring Latency.

Logins

This is where you can see login performance metrics, such as:

• Display Protocol Connection

• Profile load

• GPO Processing

• Total login time

Here’s an example of a Profile Load chart:

For more information on login performance, see Monitoring Login Performance.

Applications

This is where you can analyze application performance metrics, such as start times and Virtual User workload functions.

Here’s an example of a start time chart:

For details on Applications, see Configuring Applications for Testing.

Session Metrics

This is where you can view session metrics, such as CPU and Memory.

Here’s an example of a CPU chart:

To learn more about Session Metrics, see Managing Session Metrics. For general guidelines on how to analyze Continuous Testing Session Metrics, refer to Viewing Session Metrics.

Events

The Events tab includes events related to Continuous Test and system events. To access Events:

1. In the Login Enterprise sidebar menu, navigate to Results > Continuous testing.

2. Select the test you’re interested in, click View, and then select Events from the tab menu.

The Events table lists all recorded events:

3. Click an event to view more detailed information:

Time Span Selection

You can select a custom time span to view events. The date picker includes quick-select options:

• Today

• Last 15 minutes

• Last 30 minutes

• Last 1 hour

• Last 3 hours

• Last 6 hours

• Last 12 hours

• Last 24 hours

Custom Time Span

To select a custom time span:

1. Choose Custom in the date picker.

2. Use the From and To fields or the calendar view to select a start and end date.

3. Click Apply.

By default, the page shows events for Last 1 hour. When you change the time span, your selection is saved in the browser. Returning to the page will display your previously selected time span.

Live Updates

The page automatically updates with new events:

• If you select a quick option like Today, new events appear immediately in the list.

• If you select a time span such as Last 15 minutes or Last 1 hour, new events are added automatically, and the time span updates to include the new events.

Possible Events

• accountCapacityExceeded

• accountDisabled

• appExecutionAbandoned

• applicationFailure

• applicationThresholdExceeded

• connectionInitializationTimeout

• customUserSessionEvent

• dataRetentionFailed

• dataRetentionFinished

• dataRetentionStarted

• databaseFailure

• emailFailure

• emailRequest

• emailSent

• engineLogs

• enginePaused

• engineResumed

• euxExecutionFailure

• euxInitializationFailure

• latencyMeasurementFailed

• latencyThresholdExceeded

• launcherCapacityExceeded

• launcherLogs

• launcherOffline

• licenseSessionLimit

• loginFailure

• loginTimeMeasurementFailed

• loginTimeThresholdExceeded

• remoteSessionDisconnected

• screenshotFailure

• scriptEvent

• scriptScreenshot

• sessionDiscoveryError

• sessionFailure

• sessionRequestEndedBeforeEngineBecameOnline

• testRunCancelled

• testRunFailed

• testRunFinished

• testRunStarted

Configuring a Test PDF Report

You can choose what you would like your Continuous Test PDF report to include. Once you click Generate PDF report, you can configure the following:

Test PDF Report Breakdown

The Test PDF report summarizes the findings obtained from the testing process. It provides an overview of the Test specifics, results, and measurements. The report is divided into the following sections:

• Test details section: Shows the current configuration of the test.

• Environment summary

  • A table that shows the failure and success rates for logins, login performance, and latency performance

  • A chart that shows the success rates of all 3 measurements over time

    • Logins

    • Login performance

    • Latency performance

• Application summary

  • A table that shows the failure and success rates for application execution and performance

  • A chart that shows the success rates of the 2 measurements over time

    • Application execution

    • Application performance

• EUX score

  • 2 charts with the results of the EUX measurements of the selected time span.

    • EUX score

    • EUX timers

• Logon time

  • A table that shows the failure and success rates for the logon time. It contains one row for the total logon time.

  • A chart that contains the login-related measurements over time.

• Latency performance

  • A table that shows the failure and success rates for the latency (of all launchers).

  • A graph that shows the latency results over time for each launcher.

• Application performance

  • A table with all applications in the test. For each application, the table shows failure and success rates at the application level.

• Application details: Detailed results for each application

  • A table that shows failure and success rates for each measurement within the application.

  • Chart(s) that show the results of each measurement over time.

• Session metrics

  • Each session metric has its own chart.

SLA Reports

SLA reporting provides a concise and straightforward document that delivers key performance information. These reports are generated from historical data gathered by Login Enterprise during its operational processes, making them ideal for monitoring service-level agreements, understanding infrastructure performance over time, and facilitating operations reporting to management and service customers.

Benefits of SLA reporting

Configuring SLA Reports

1. To create a new report, in the Login Enterprise sidebar menu, navigate to Results > Continuous Testing.

2. In the Continuous Testing results, select SLA Reports from the tab menu.

3. In the SLA reports, click the green "+" in the upper-right corner to add a new SLA report:

4. Configure the SLA report with the following settings:

5. Click Next.

6. Click Add new threshold. This allows you to configure the following:

7. Click Save to apply SLA report configurations.

Generating SLA Reports

Reports are not generated immediately unless you click Manually generate report. If the schedule is set to Daily, the first report will be available after one day. To generate a report manually, go to Available reports:

1. Click Manually generate report.

2. Select the desired date.

3. Click Generate.

To view the report, go to Available reports and click the report you just generated. The PDF file will download automatically:

You can also download a sample SLA report from Additional Resources.

Scheduled Reports

• Daily reports: Generated every day.

• Weekly reports: Generated every week.

• Monthly reports: Generated every month.

Scheduled reports can be deleted as needed.

The SLA report displays the report date, which indicates the time frame it covers.

Export Data

You can export data from Continuous Tests in CSV format. This is useful for integrating data from different sources.

You can export the data using:

1. The Export Data Tab:

   1. In the Continuous Testing Results page, select Export Data from the tab menu.

   2. Click the green + icon to create a new export.

   3. In From date and To date, specify the timeframe for exporting data, and click Next.

   4. In Test selection, select environments for exporting data, and click Export.

   5. In Export data, click the folder icon to download the .zip file.

2. The Charts Tab:

   1. In the Continuous Test Results page, select Charts from the tab menu.

   2. Use the drop-down menu on the left to select the configuration.

   3. Click Export selected data to CSV under the chart window.

   4. The RawData.zip file will be prepared and downloaded.

RawData.zip File

The RawData.zip file is named according to the following schema: RawData <TEST_NAME> <DATE> <TIME>.zip

An example of the RawData.zip file looks as follows:

EUX Raw Data

To enhance debuggability and facilitate root-cause analysis, EUX raw timer data can be essential. Login Enterprise stores the raw data by default, and the data retention can be adjusted based on your specific needs.

Retrieving the EUX Raw Data

You can retrieve the EUX raw data via the Public API with the following request:

GET https://appliance.loginvsi.com/publicApi/v7/test-runs/{testRunId}/eux-script-executions

The API will return all the EUX measurements for each user session. Below is an example of how the response is structured:

{
"items": [
{
"userSessionId": "6f838f90-b9ed-4e63-b04e-2291a50e34cb",
"startTime": "2025-01-06T15:20:55.288615Z",
"endTime": "2025-01-06T15:22:08.481351Z",
"successful": true,
"euxMeasurements": [
{
"timestamp": "2025-01-06T15:21:55.998094Z",
"timer": "appSpeedUserInput",
"duration": 635
},
{
"timestamp": "2025-01-06T15:21:32.025Z",
"timer": "appSpeed",
"duration": 5656
},
{
"timestamp": "2025-01-06T15:22:00.373194Z",
"timer": "cpuSpeed",
"duration": 210623
},
{
"timestamp": "2025-01-06T15:21:37.61784Z",
"timer": "diskAppDataLatency",
"duration": 4329
},
{
"timestamp": "2025-01-06T15:22:02.388249Z",
"timer": "diskAppData",
"duration": 16952
},
{
"timestamp": "2025-01-06T15:22:03.918992Z",
"timer": "diskMyDocsLatency",
"duration": 4424
},
{
"timestamp": "2025-01-06T15:22:04.419015Z",
"timer": "diskMyDocs",
"duration": 14851
},
{
"timestamp": "2025-01-06T15:21:42.172781Z",
"timer": "fastCompression",
"duration": 920
},
{
"timestamp": "2025-01-06T15:21:19.825964Z",
"timer": "highCompression",
"duration": 925
}
],

Cleaning the EUX Data (Data Retention)

You can configure the data retention period for EUX raw data, ranging from 1 to 9999 days. Customizing this setting is crucial to managing storage resources effectively. New installations and upgrades default to a 30-day retention period, but any customizations persist through subsequent upgrades. To adjust the EUX raw data:

1. In the Login Enterprise sidebar menu, navigate to Other > System.

2. In the System, click General.

3. In Data retention, configure EUX raw data.

Tips and Tricks

Chart Legend

The chart legend provides additional statistical insights for the selected time span:

• High: Highest value within the selected time span

• Low: Lowest value within the selected time span

• Average: Average value within the selected time span

• Latest: Most recent value within the selected time span

Chart Control

Use the Chart Control to navigate between the chart sections and select or deselect measurements.

Downloading Charts as an Image

You can download charts as images for sharing or use in custom reports and presentations. Downloading a chart as an image applies to all charts: Load Test, Continuous Test, and Appliance Health. To download a chart:

1. Locate the Download icon in the top right corner of the chart.

2. Select image format: PNG, SVG, or JPG.

3. Download the image. It will be saved to your Downloads folder, or the directory you've configured as your default Downloads location.

Additional Resources

SLA2024-06-26T13_02_18.2231530.pdf