Tests

Available commands

Grouped by how often you will reach for them.

Runtime variables

Six commands return a value, and all six must capture it. That is what saveAs is for. A command that reads something and throws the result away is almost always a mistake, so it is rejected rather than silently accepted.

Once captured, refer to the value anywhere as {{runtime.NAME}}.

[
  {
    "command": "smartt_read_element",
    "input": { "role": "heading", "name": "Order confirmed" },
    "saveAs": "confirmation"
  },
  {
    "command": "smartt_assert",
    "input": {
      "actual": "{{runtime.confirmation}}",
      "operator": "contains",
      "expected": "Order confirmed"
    }
  }
]

Assertion operators

smartt_assert takes actual, an operator, and usually an expected:

• Comparison: equals, notEquals, contains, notContains, matches (regular expression).
• Numeric: greaterThan, lessThan, greaterOrEqual, lessOrEqual.
• No expected needed: isTrue, isFalse, isEmpty, isNotEmpty.

How to point at an element

Most commands need to identify an element. The order below is not stylistic. It is how resistant each one is to a redesign that does not change behaviour:

• Role and name: { "role": "button", "name": "Sign in" }. Matches what a user sees, and survives markup changes.
• Label: for form fields, the visible label.
• Test id: stable, but only if your app actually maintains them.
• CSS selector: the last resort. It breaks on refactors that changed nothing a user would notice, and that is exactly what makes a suite feel unreliable.

Names match by substring, so a link named Free also matches "Start for Free". When more than one element matches, the step fails instead of picking one at random, which is the right call: a test that silently chose would pass or fail depending on render order.

• exact: true requires the whole name to match. This is the fix you want most of the time.
• nth: 0 picks one when several elements really are identical, like a row in a list.

The same screen, two tests

The scorecard sits above the automation plan. These two tests were written against the same application on the same afternoon.

Nothing about the first test is unusual. It was written quickly to move a coverage number, it runs, and it goes green. That is exactly why a deterministic score is worth having: nobody would have caught this by reading it.

It catches flakiness before you have any

A flaky test is usually not born flaky. It is written with a pattern that will eventually flap, and the flapping only shows up weeks later on a slower day. Much of what the engine flags is exactly that pattern, caught while the automation is still on screen:

• A fixed sleep instead of a wait. Waiting three seconds passes on your machine and fails on a loaded CI box. The fix is waiting for the element, and the finding says so.
• A CSS selector where a role and name would do. It survives until somebody reorders a div, and then it fails for a reason that has nothing to do with the product.
• Waiting on text that is not guaranteed to appear. A wait that sometimes has nothing to wait for is a timeout waiting to happen.
• A value read and never asserted. The test looks like it checks something and does not, so it passes through real breakages and fails on unrelated noise.

Every finding comes with a prompt

Each finding has a copy fix prompt action. It generates an instruction written for that specific finding, on that specific step, ready to paste into a terminal agent such as Claude Code or Cursor. The prompts are built per rule rather than generically, because "fix this test" makes an agent guess, and a guess is how a brittle locator becomes a differently brittle locator.

A side effect worth having: accessibility

The engine pushes you towards locating elements by their role and accessible name, because that is what survives a redesign. That happens to be the definition of an accessible interface too: a button a screen reader can announce is a button your test can find by name.

Plenty of legacy applications have neither. When a test cannot be written against roles and names because the page exposes none, that is not a testing problem to work around with a CSS selector. It is an accessibility gap the automation just found. Teams automating older applications with SmarTTest tend to end up adding labels and roles to the product itself, which makes the tests stable and the application usable by more people at the same time.