Automation
MCP tool reference
All forty tools, grouped by what they touch. Setup instructions are in SmarTT MCP.
Remote or local#
Most tools work on both the hosted connector and the local server. Four exist only locally, because they drive a real browser and the server does not have one.
Note
run_on_runner, which hands the work to a machine that has a browser. See SmarTT Runner CLI.Projects & folders#
Where an agent starts. Almost every other tool needs a projectId, and this is where it comes from.
list_projects
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| includeInactive | booleandefault false | Include deactivated projects. They are read-only, so writes to them will be refused. |
Use it when
- First call of almost any session, because nothing else works without a project id.
- Confirming which project a request refers to when the user says "the checkout tests" and there are three projects.
list_project_folders
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| projectIdreq | string | From list_projects. |
Use it when
- Finding the folder id needed to create a test.
- Understanding how a team organizes its library before adding to it.
- Deciding where a new test belongs, rather than dropping it in the first folder found.
create_folder
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| projectIdreq | string | From list_projects. |
| namereq | string | Maximum 30 characters. |
| parentId | string | Omit for a root folder. Otherwise the folder this one nests under. |
Use it when
- Setting up structure for a new feature area before writing its tests.
- Splitting a folder that has grown past the point of being navigable.
Tests#
The core of the API. Reading, writing, archiving, and checking the quality of automation.
list_tests
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| projectIdreq | string | From list_projects. |
| folderId | string | Restrict to one folder. |
| recursive | booleandefault true | Whether subfolders are included when folderId is set. |
| includeArchived | booleandefault false | Include archived tests. Needed to find something to restore. |
Use it when
- Finding a test by title before reading it in full.
- Surveying what is automated and what is not across an area.
- Locating an archived test so it can be restored.
get_test
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| testCaseIdreq | string | From list_tests or get_test. |
| includeArchived | booleandefault true | Whether an archived test can be returned. |
Use it when
- Reading the written steps before writing automation for them.
- Inspecting an existing automation plan before changing it.
- Checking which reusable preconditions are attached, and in what order.
create_test
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| projectIdreq | string | From list_projects. |
| folderIdreq | string | Must already exist. |
| titlereq | string | What the test verifies, in one line. |
| priority | LOW | MEDIUM | HIGH | CRITICAL | Defaults if omitted. |
| description | string | Context the reader needs. |
| preconditions | string | Written state required before step 1. |
| stepsreq | step[] | At least one. Each may carry automation. |
| expectedResult | string | What must be true at the end. |
| expectedResultAutomation | action[] | Commands that verify the expected result, normally the assertions. |
| expectedResultAttachments | string[] | URLs of supporting images. |
| references | string | Tickets, requirement ids. |
| automated | No | Yes | Impossible | Discardeddefault "No" | Only these four values. Anything else is rejected. |
| automationStatus | Completed | In Progress | Needs Attention | Broken | Only allowed when automated is Yes. |
| automatedRejectReason | string | Why the test was not automated. Only for Impossible and Discarded. |
| tagNames | string[] | By name, not id. Tags must already exist. |
Use it when
- Turning a requirement or a bug report into a written test.
- Creating a test and its automation together, in one call.
- Bulk-creating a set of cases for a new feature.
automated values are exact. "yes", "true", or "automated" are all rejected rather than coerced.update_test
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| testCaseIdreq | string | From list_tests or get_test. |
| folderId | string | Move the test to a different folder. |
| title | string | |
| priority | string | |
| description | string | null | Pass null to clear. |
| preconditions | string | null | Pass null to clear. |
| steps | step[] | Replaces the whole list. It is not a merge. |
| expectedResult | string | null | |
| expectedResultAutomation | action[] | null | Replaces the verification actions entirely. |
| references | string | null | |
| automated | No | Yes | Impossible | Discarded | |
| automationStatus | string | null | Only when automated is Yes. |
| automatedRejectReason | string | null | Only for Impossible or Discarded. |
| addTagNames | string[] | Tags to add, by name. |
| removeTagNames | string[] | Tags to remove, by name. |
Use it when
- Attaching automation to a test that was written manually.
- Marking a test
Brokenafter the UI changed under it. - Adding tags in bulk when a new suite is defined.
- Moving a test to the folder it should have been in.
steps replaces the entire list. To change one step, read the test first, modify the array, and send all of it back.list_automation_commands
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| command | string | Return only this command. Omit it to get all twenty-six. |
Use it when
- Before writing any automation plan. It is the difference between knowing a command exists and knowing what to put in it.
- After a rejected save, to see what the command actually accepts instead of guessing again.
analyze_test_quality
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| testCaseIdreq | string | From list_tests or get_test. |
Use it when
- Immediately after writing automation. This is the single most valuable tool for an agent, because it turns "looks right" into a checkable result.
- Auditing an existing suite to find the tests worth fixing first.
- Confirming a fix actually improved things rather than moving the problem.
archive_test
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| testCaseIdreq | string | From list_tests or get_test. |
Use it when
- Retiring a test for a feature that no longer exists.
- Removing a duplicate, once you have confirmed which one to keep.
restore_test
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| testCaseIdreq | string | From list_tests or get_test. |
Use it when
- Undoing an archive.
- Recovering a test after a folder was deleted. Find it first with
list_testsandincludeArchived: true.
Reusable actions#
Shared blocks of steps, and how tests attach to them. Editing a block changes every test that uses it. See Reusable Actions for the model.
list_reusable_actions
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| projectIdreq | string | From list_projects. |
Use it when
- Checking whether a login block already exists before writing another one.
- Seeing the blast radius of a change before making it.
- Finding unused blocks worth deleting.
get_reusable_action
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| reusableActionIdreq | string | From list_reusable_actions. |
Use it when
- Reading a block's steps before modifying them.
- Understanding what a precondition actually does when debugging a failing test.
create_reusable_action
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| projectIdreq | string | From list_projects. |
| namereq | string | Describe the outcome: "Log in as admin", not "Fill form". |
| description | string | What state the system is in once it finishes. |
| usableAs | PRECONDITION | STEP | BOTHdefault "PRECONDITION" | Where the block may be used. |
| stepsreq | step[] | At least one. |
Use it when
- Extracting a sequence that has been copied into several tests.
- Defining login once, at the start of automating a project.
smartt_run_reusable_action. Blocks do not nest. If two share steps, inline them.update_reusable_action
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| reusableActionIdreq | string | |
| name | string | |
| description | string | null | |
| usableAs | PRECONDITION | STEP | BOTH | |
| steps | step[] | Replaces the whole list. |
Use it when
- Fixing every test at once after a login form changed.
- Making a block more robust, replacing a CSS selector with a role-and-name locator.
delete_reusable_action
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| reusableActionIdreq | string |
Use it when
- Removing a block nothing uses any more.
attach_reusable_action_to_test_preconditions
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| testCaseIdreq | string | From list_tests or get_test. |
| reusableActionIdreq | string | Must be in the same project. |
Use it when
- Giving a new test the standard login setup.
- Replacing steps that were inlined into a test with the shared block, usually right after
analyze_test_qualityflags the duplication.
detach_reusable_action_from_test_preconditions
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| linkIdreq | string | The link id, from get_test, not the reusable action id. |
Use it when
- A test that no longer needs that setup.
- Clearing links before deleting a block.
reorder_test_reusable_preconditions
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| testCaseIdreq | string | From list_tests or get_test. |
| orderedLinkIdsreq | string[] | Every attached link id, exactly once, in the order you want. |
Use it when
- Fixing setup that runs in the wrong order: selecting an account before logging in, for instance.
- Putting a newly attached precondition in the right place instead of last.
Environments & variables#
What an execution needs. These tools tell an agent which keys exist and where they are defined, never what a secret contains.
get_execution_settings
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| projectIdreq | string | From list_projects. |
Use it when
- Reporting how tests will run, without guessing.
- Deciding whether to override the browser for one execution.
execute_test and execute_run already apply these. You only need this tool to show or reason about them.list_environments
remote + localParameters
Takes no parameters.
Use it when
- Getting the
environmentIdthat every execution tool requires. - Confirming which target the user means before running anything against it.
get_org_variables
remote + localParameters
Takes no parameters.
Use it when
- Discovering what is available to reference as
{{orgVar.KEY}}before writing automation. - Checking a base URL is defined before writing a test that depends on it.
get_project_variables
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| projectIdreq | string | From list_projects. |
Use it when
- Finding what to reference as
{{projectVar.KEY}}. - Understanding which values are project-specific rather than shared.
get_test_variables
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| testCaseIdreq | string | From list_tests or get_test. |
Use it when
- Checking what a test defines for itself as
{{testVar.KEY}}. - Diagnosing a test that fails only in one environment.
resolve_test_execution_variables
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| testCaseIdreq | string | From list_tests or get_test. |
| environmentIdreq | string | From list_environments. |
Use it when
- Confirming a test will get the values it expects before running it.
- Working out why the same test passes against staging and fails against production. Usually it is a variable defined for one environment and not the other.
Runs#
Creating runs, recording results, and closing them. A run stays open until complete_run is called.
list_runs
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| projectIdreq | string | From list_projects. |
| limit | numberdefault 50 | Between 1 and 200. |
Use it when
- Finding the run a user is referring to.
- Checking whether an open run already exists before creating another.
- Summarising recent testing activity.
get_run
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| runIdreq | string | From list_runs. |
Use it when
- Reading what failed and why, in order to act on it.
- Watching results land while a runner executes.
- Producing a report from a completed run.
create_run
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| projectIdreq | string | From list_projects. |
| namereq | string | What this run is verifying. |
| includeTimestamp | booleandefault false | Append a date and time to the name, the same option the interface offers. |
| comment | string | What this run is for. Worth filling in. |
| folderIds | string[] | Include everything in these folders. |
| tagNames | string[] | Include every test carrying these tags. |
| testCaseIds | string[] | Include these specific tests. |
Use it when
- Building a smoke run from a tag before a release.
- Creating a targeted run for the area a change touched.
- Setting up a run to hand straight to a runner.
update_run_test_result
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| runIdreq | string | |
| testCaseIdreq | string | From list_tests or get_test. |
| statusreq | PENDING | IN_PROGRESS | PASSED | FAILED | SKIPPED | Case-insensitive. |
| commentreq | string | Cannot be empty. |
| videoUrl | url | Recording of the execution. Left by the runner. |
| executionId | string | Which runner execution produced this. |
| durationMs | number | How long it took. |
Use it when
- Recording the outcome of a test executed outside SmarTTest.
- Marking a test skipped, with the reason it was out of scope.
- Correcting a result after investigating a failure.
complete_run
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| runIdreq | string | |
| commentreq | string | The closing note. Cannot be empty. |
Use it when
- Finishing a run once every test has a result.
- Closing a run so it starts counting toward the Health Dashboard.
Execution#
Two paths. Queue the work onto a runner, which works from anywhere, or execute it directly, which only the local server can do.
list_runners
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| projectIdreq | string | From list_projects. |
Use it when
- Getting the
runnerIdforrun_on_runner. - Checking one is online before queueing work, so the user is not left waiting on a stopped machine.
run_on_runner
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| runIdreq | string | The run to execute. |
| runnerIdreq | string | From list_runners. |
| environmentIdreq | string | From list_environments. |
| concurrency | 1 – 8default 1 | How many tests at once. Raise it only when the tests are genuinely independent. |
| only | all | pending | faileddefault "all" | Which results to execute. failed is the usual second pass. |
| headless | booleandefault true | Whether the browser window is hidden on the runner machine. |
Use it when
- The only way to execute tests from the remote connector.
- Running a full regression suite on a dedicated machine instead of your laptop.
- Re-running just the failures after a fix.
get_runner_job, or read the run to watch results land.get_runner_job
remote + localParameters
| Parameter | Type | Description |
|---|---|---|
| jobIdreq | string | Returned by run_on_runner. |
Use it when
- Polling until execution finishes.
- Distinguishing "the runner never picked this up" from "the tests are failing". The job carries its own error separate from the results.
validate_automation
local server onlyParameters
| Parameter | Type | Description |
|---|---|---|
| urlreq | string | The page to open. Resolve any variable placeholder yourself first. |
| locatorsreq | object[] | The locators to check, each the same input object an action would use. |
| headless | booleandefault true | Whether the browser window is hidden. |
Use it when
- Before asking anyone to execute a run. A locator that matches four elements fails the whole test, and this finds that out in one step instead of one execution.
- When a test fails and you cannot tell whether the element moved or the page did. The reply includes the final URL and title, which is how a redirect gives itself away.
execute_test
local server onlyParameters
| Parameter | Type | Description |
|---|---|---|
| testCaseIdreq | string | From list_tests or get_test. |
| environmentIdreq | string | From list_environments. |
| headless | booleandefault true | Whether the window is visible. |
| stopOnFailure | booleandefault true | Stop at the first failing step, rather than carrying on. |
| timeoutMs | numberdefault 180000 | Overall limit, from 1s to 15 minutes. |
| runId | string | Log this result into an existing run. Omit it while iterating, and the execution then leaves no trace anywhere. |
| recordVideo | boolean | Defaults to true when runId is set, false otherwise. |
| videoWidth | numberdefault 1280 | Viewport and recording width. |
| videoHeight | numberdefault 720 | Viewport and recording height. |
| browserChannel | chromium | chrome | msedge | Which browser to drive. Defaults to the Chromium bundled with Playwright. |
Use it when
- The develop-run-develop loop: write automation, run it, see the failure, fix it, without polluting any run with attempts.
- Reproducing a failure locally with a visible browser to see what is happening.
- Recording a real result, by passing
runId.
runId is the important detail. Without it the execution is recorded nowhere, which is exactly what you want while iterating.get_test_execution
local server onlyParameters
| Parameter | Type | Description |
|---|---|---|
| executionIdreq | string | From execute_test. |
Use it when
- Polling a long execution.
- Reading which step failed, and the error it produced.
execute_run
local server onlyParameters
| Parameter | Type | Description |
|---|---|---|
| runIdreq | string | |
| environmentIdreq | string | |
| concurrency | 1 – 8default 1 | Each test gets its own browser. Only raise it for independent tests. |
| only | all | pending | faileddefault "all" | Which results to run. |
| headless | boolean | |
| recordVideo | boolean | |
| timeoutMs | number | Per test. |
| videoWidth | number | |
| videoHeight | number | |
| browserChannel | chromium | chrome | msedge |
Use it when
- Executing a whole suite locally, without setting up a runner.
- Re-running the failures from an earlier pass on your own machine.
get_run_execution
local server onlyParameters
| Parameter | Type | Description |
|---|---|---|
| runExecutionIdreq | string | From execute_run. |
Use it when
- Polling until the suite finishes.
- Reporting progress while a long run is in flight.
