​

Self-healing and Flakiness Avoidance

The Testkit AI has two main goals:

• Self-healing - If an element is not found, the Testkit AI will try to find it in a different way.
• Flakiness Avoidance - The Testkit AI tries to detect flakiness and bypass or fix it.

​

Self-healing

The Testkit AI will try to find the element in a different way if it is not found.

Testkit follows a set of priorities for this, and will try to find the element in the following order:

1. Try multiple different selectors (Usually around 6-8 different selectors)
2. Find the element with an innerText if applicable
3. Go through a few synonyms or metaphors for the elements old innerText if applicable
4. See if there is an element with the same tag near the old element position and check if variations of the old selectors or innerText work

If Testkit finds the element with a different selector, it will notify you and ask you if you want to update the selector or ignore the newly found selector in the test case view.

We’re constantly improving the Testkit AI and adding more ways to find elements as we cover more use cases. The current cases are also improved over time.

​

Flakiness Avoidance

The first thing to know here is that Testkit tries to get elements for up to 10 seconds. That doesn’t mean each step takes 10 seconds though. For instance, if Testkit finds the element after 500 milliseconds, it will not wait for the other 9.5 seconds. This alone already helps a lot with flakiness.

Apart from that, there are a few other things Testkit does to avoid flakiness:

• Assertions are retried for up to 10 seconds if they don’t pass
• If an element is not interactable (i.e. it’s not visible or disabled), Testkit will wait up to 10 seconds for it to become interactable
• If an element reference somehow gets lost (i.e. because the element was removed from the DOM), Testkit will retry the specific step up to 3 times, with a 1 second delay between each retry
• If an interaction gets intercepted (i.e. because of a modal), Testkit will retry the specific step up to 3 times, with a 1 second delay between each retry

This covers most of the flakiness cases we’ve seen so far, but we’re constantly improving this as well. In the future, Testkit will take more drastic measures if needed to avoid flakiness, such as going back a few steps, reloading the page, restarting test cases, or even restarting the entire test suite. All of this will be configurable, so you can decide how aggressive the flakiness avoidance should be.