Understanding WinApp Recorder Implementation

Overview

To improve the robustness, maintainability, and flexibility of Script Recorder, the following changes have been made:

The existing automation library has been replaced with FlaUI.
XPath matching has been made more flexible. Previously, the full sequence of element descriptors had to be provided, and every part had to match exactly. Now you can omit steps or use wildcards, making element lookup less rigid and easier to maintain.
A new IAutomationElement abstraction replaces the former IAutomatableComponent, reducing bloat and decoupling it from the Engine.
A more stable foundation for creating reliable and reproducible automation scripts has been established.
Script Recorder can now record multiple applications at once, regardless of the specified target application. For example, if you set Calculator as a target when creating a new application script, you can launch and record Calculator. However, you can also record any other window present on the desktop, such as Microsoft Word, etc.

This page describes the inventory of methods used by Script Recorder and XPath changes. For information on how to use WinApp Recorder, see Using the WinApp Recorder.

Inventory of Methods (Generated by Script Recorder)

IOperatingSystem Methods

The OperatingSystem interface includes the following methods, accessible directly through Execute(...). These methods do not require a window to be located beforehand (unlike searching for a specific element, e.g., a button). They are also independent of the MainWindow, unlike the old Engine, where you needed to use the main window. As a result of this decoupling, you can now record multiple applications, so recording is no longer limited to the specified target application.

The MainWindow will still be populated with the target application window if you want to use the old Engine methods.

Note: When using FindByInformation or FindByXpathOrInformation without providing window information, the search traverses all windows on the desktop, which may result in slower performance in some scenarios.

Method: FindAutomationElementByXPath(…)
Description	Finds an`AutomationElement` that matches the specified XPath expression. The entire element sequence must match for the element to be considered found. You can omit ancestor or window information to make the search more flexible, or include it to improve performance.
Returns	An `AutomationElement` that implements the `IAutomationElement` interface.
XPath Language Changes	There is a change in the XPath syntax between the old and new methods. For more details, see the XPath Query Changes.
Parameters:
XPath	The XPath query used to locate the automation element. Must match exactly one element for a successful result.
timeout	The maximum time, in milliseconds, to wait for the element to be found. Must be greater than or equal to zero.

A FindAllVariant variant of the method can also be generated manually. For details, see Appendix.

Method: FindAutomationElementByInformation(…)
Description	Finds an `AutomationElement` by its properties, without using information about ancestor elements. This method is useful in scenarios where the element hierarchy may change frequently. It provides the most flexible search approach; however, this flexibility may result in slower performance. A new optimization improves performance by splitting the element search into two steps: First, it locates the window via XPath. Then, it finds the desired element using `FindByInformation`, starting from that window, substantially narrowing the search scope. Example (Before optimization): var buttonNine0 = FindAutomationElementByInformation("num9Button", "Button", "Nine", "Button", 5, continueOnError:false); Example (After optimization): var buttonNine0 = FindAutomationElementByXPath("/Window[@ClassName=\"ApplicationFrameWindow\"][@Name=\"Calculator\"]", 5, continueOnError:false).FindAutomationElementByInformation("num9Button", "Button", "Nine", "Button", 5, continueOnError:false); This optimization is configurable in the Recorder UI. For details, see Using the WinApp Recorder.
Returns	A list of `AutomationElement` objects that implement the `IAutomationElement` interface.
Remarks
Search Root	The old Engine required window information to locate elements. The new method performs searches from the Desktop as the root. Because ancestor and window information are omitted, the search covers all elements on the Desktop. As an optimization, you can configure the element search to first attempt to locate the window.
Parameters
AutomationId	The automation ID of the element to find. Can be `null` or empty to ignore this property.
className	The element's class name to find. Can be `null` or empty to ignore this property.
name	The name of the element to find. Can be `null` or empty to ignore this property.
controlType	The control type of the element to find. Can be `null` or empty to ignore this property.
timeout	The maximum time, in milliseconds, to wait for the element to be found. Must be greater than or equal to zero.

A FindAll variant of the method can also be generated manually. For details, see Appendix.

Method: FindAutomationElementByXPathOrInformation(…)
Description	Finds an automation element by using the specified XPath expression. If no match is found, the method attempts to locate the element by matching automation properties such as automation ID, class name, name, and control type. This approach first uses a strict, performant search strategy (findByXpath) and then falls back to a more flexible property-based search (findByInformation) if metadata in the XPath chain has changed or is no longer reliable.
Returns	`AutomationElement` that implements the `IAutomationElement` contract
Remarks:
Search Root	`FindByInformation` previously required window information to locate elements. The new method performs searches from the Desktop as the root, so you do not need to provide window information. This increases flexibility, but because it traverses all windows on the Desktop, performance may be slower in some scenarios. `FindByXpath`, by contrast, follows a specific set of breadcrumbs from the Desktop, making it considerably faster. In Win365 scenarios, window metadata (such as title or class name) can change between recording and execution. For example, if the document moves or the window title differs. Hiding window information ensures that scripts continue to work even when window details vary. For instance, if a script is recorded on one document window but executed when only a different window is open, it will succeed when window information is omitted.
XPath Language Changes	There is a change in the XPath syntax between the old and new methods. For details, see XPath Query Changes.
Parameters:
XPath	The XPath expression used to locate the automation element. If `null` or empty, the search uses the provided automation properties instead.
AutomationId	The automation ID of the element to find. Used if the XPath expression is not successful. Can be `null` or empty to ignore this property.
className	The element's class name to find. Used if the XPath expression is not successful. Can be `null` or empty to ignore this property.
name	The name of the element to find. Used if the XPath expression is not successful. Can be `null` or empty to ignore this property.
controlType	The control type of the element to find. Used if the XPath expression is not successful. Can be `null` or empty to ignore this property.
timeout	The maximum time, in milliseconds, to wait for the element to be found. Must be greater than or equal to zero.

This method provides a FindAll variant that returns a list of elements instead of a single element. Unlike the single-element version, it does not throw an exception when multiple elements are found.

Note: A FindAll command is not currently generated automatically by Script Recorder. Script Recorder uses Find[single]ElementByXXXX, a method only available for manual use. For details, see Appendix.

XPath Query Changes

Script Recorder has transitioned to a standard XPath-style syntax. In the new format:

The ControlType is used as the XPath node (tag)
All other metadata is expressed as named attributes

/MenuBar/MenuItem[@Name='SpecificName'][@AutomationId : ‘123’][@ClassName='ClassName']

The standardized XPath query language makes each property explicit, readable, and independently matchable. It allows you to perform complex queries and leverage built-in XPath functions to create flexible and resilient selectors.

/MenuBar/MenuItem[@Name='SpecificName'][@AutomationId : ‘123’][Contains(@ClassName, 'Name’)]

/MenuBar/MenuItem[@Name='SpecificName'][@AutomationId : ‘123’][Starts-with(@ClassName, 'Name’)]

You can also omit requirements from an element search in two ways: by leaving metadata fields blank or by omitting specific metadata from the query altogether.

Blank metadata: /RandomWindow[@Class, “ “]

Omitted metadata: /RandomWindow________

Finally, the XPath language allows you to exclude entire chains of elements and to search within a specific subsection of elements using the following syntax:

Original Hierarchy:

/Window/Pane/Pane/Pane/Pane/Pane/Pane/Pane/Pane/Pane/Pane/Pane/Document/Group/Button[@Name=\"Sign in\"]

Trimmed Hierarchy:
//Group/Button[@Name=\"Sign in\"]

Note: The first tags don’t enforce any metadata, and Button enforces the Name value to be “Sign in”.

An element search is considered successful when a sequence of elements includes both:

A group node
A button node with the Name property set to "Sign In"

These capabilities provide the following benefits:

Partial matches for dynamic values
More tolerant element selection
Fewer failures caused by minor UI changes

Appendix

The following are methods that are not generated by Script Recorder, but can be performed manually.

IAutomationElement Methods

This interface provides the same automation methods as described above, with identical parameters. The main difference is that these methods use the current element as the root instead of the Desktop.

Using a specific element as the root allows you to chain method calls and constrain searches to a particular subtree. This can improve performance, but the search becomes more rigid.

For example, defining an element as the root lets you chain searches to include only relevant nodes in the hierarchy, skipping elements that are not relevant. This approach refines the search and avoids unnecessary paths compared with searches starting from the Desktop.

Note: The new element search syntax allows you to ignore portions of the XPath tree. For details, see XPath Query Changes.

The IAutomationElement interface provides the same methods as described in the tables above.

Method: FindAutomationElementByXPathOrInformation(…) & FindAllAutomationElementByXpathORInformation
Description	Finds an `AutomationElement` using the specified XPath expression. If no elements are found, the search attempts to match automation properties such as automation ID, class name, name, and control type. This variant starts the search from the specified automation element rather than the Desktop.

Method: FindAutomationElementByXPathOrInformation(…) & FindAllAutomationElementByXpathORInformation

Description

Finds an AutomationElement using the specified XPath expression. If no elements are found, the search attempts to match automation properties such as automation ID, class name, name, and control type.

This variant starts the search from the specified automation element rather than the Desktop.

Method: FindAutomationElementByXPath(…) & FindAllAutomationElementByXPath
Description	Finds an `AutomationElement` that matches the specified XPath expression. The entire element sequence must match for an element to be considered found. This variant starts the search from a specified automation element rather than the Desktop. Starting from a specific element allows you to constrain searches to a subtree, which is useful for complex queries involving multiple nested panes or containers (see the bold elements, and unbolded elements are ignored): Window/Pane₁/…/Group/List/…./Pane_i/Pane_j/…./Pane_n/Button For such a query, you can perform a search called: Root.FindByXpath(//Pane).FindByXpath(//Pane/Pane).FindByXpath(//Pane/Button)

Method: FindAutomationElementByXPath(…) & FindAllAutomationElementByXPath

Description

Finds an AutomationElement that matches the specified XPath expression. The entire element sequence must match for an element to be considered found.

This variant starts the search from a specified automation element rather than the Desktop. Starting from a specific element allows you to constrain searches to a subtree, which is useful for complex queries involving multiple nested panes or containers (see the bold elements, and unbolded elements are ignored):

Window/Pane₁/…/Group/List/…./Pane_i/Pane_j/…./Pane_n/Button

For such a query, you can perform a search called:

Root.FindByXpath(//Pane).FindByXpath(//Pane/Pane).FindByXpath(//Pane/Button)

Method: FindAutomationElementByInformation(…) & FindAllAutomationElementByInformation
Description	Finds an `AutomationElement` by its properties, without using ancestor information. This variant starts the search from a specified automation element rather than the Desktop. Note: Searching from a specific window or element as a root improves the performance of `FindByInformation`, because the search does not need to cover the entire desktop window hierarchy. Use `FindByInformation` from any element below the Desktop to optimize performance.

Method: FindAutomationElementByInformation(…) & FindAllAutomationElementByInformation

Description

Finds an AutomationElement by its properties, without using ancestor information.

This variant starts the search from a specified automation element rather than the Desktop.

Note: Searching from a specific window or element as a root improves the performance of FindByInformation, because the search does not need to cover the entire desktop window hierarchy. Use FindByInformation from any element below the Desktop to optimize performance.

In addition to element search methods, the interface provides the following methods. These methods cover user interactions within a window environment, such as clicks and keyboard input.

Method Name	Description
Click	Performs a click at the specified element’s location (as defined by its bounding rectangle). The mouse pointer may not move directly over the element, but the click action is still executed on the requested element.
Move Mouse To Center	Performs the mouse move from its initial location to the center of the specified element.
Move Mouse Relative To Coordinates	Performs the mouse move from its initial location to the specified coordinates relative to the current element’s location. In other words, the movement is not based on the global pixel location, but on the pixel location within the element’s bounding rectangle. These coordinates are captured during recording.
Right Click	Performs a right-click at the specified element’s location (as defined by its bounding rectangle). The mouse pointer may not move directly over the element, but the click action is still executed on the requested element.
Double Click	Performs a double-click at the specified element’s location (as defined by its bounding rectangle). The mouse pointer may not move directly over the element, but the click action is still executed on the requested element. The “delta” (the amount of time between clicks) used to register a double-click is based on the window's default values.
GetRootWindow	Attempts to fetch the containing top-level window for the current element. This allows access to window-specific methods, such as `Close()`.
AsWindow	Attempts to convert the current element to a window. The conversion fails if the element does not have a control type of `Window`.

FindAll Methods

Method: FindAllAutomationElementByXPath
Description	Finds all `AutomationElement` objects that match the specified property criteria.
Returns	A list of `AutomationElement` objects that implement the `IAutomationElement` interface.

Method: FindAllAutomationElementByInformation(…)
Description	Finds all `AutomationElement` objects that match the specified property criteria.
Returns	A list of `AutomationElement` objects that implement the `IAutomationElement` interface.

Method: FindAllAutomationElementByXPathOrInformation(…)
Description	Finds all automation elements that match the specified XPath expression. If no elements are found, the method attempts to locate matches by automation properties such as automation ID, class name, name, and control type. Use this method when you need to interact with all elements that meet the specified criteria.
Returns	A list of `AutomationElement` objects that implement the `IAutomationElement` interface.

Method: FindAllAutomationElementByXPathOrInformation(…)

Description

Finds all automation elements that match the specified XPath expression. If no elements are found, the method attempts to locate matches by automation properties such as automation ID, class name, name, and control type.

Use this method when you need to interact with all elements that meet the specified criteria.

Returns

A list of AutomationElement objects that implement the IAutomationElement interface.