Understanding test status

Test status is the fastest way to understand run health across your automation workflow. It helps you identify what completed successfully, what failed, what has never run, and what is planned next so you can respond quickly and keep releases moving.

Result statuses

These statuses appear on the Test Cases list Result column, run result badges, and Execution History:

Not Executed

The test has never been run.

Passed

All steps completed successfully and assertions succeeded.

Failed

One or more steps hit an error or a failed assertion.

Aborted

The run ended before completion (for example cancelled before it finished).

Stopped

The run was manually stopped with the Stop button.

Not Executed

Appears when a test case has never been run. In the Test Cases list it shows under Result until you run the case for the first time.

Passed

Appears when all test steps complete successfully and all assertions are met. Use passed results to confirm release readiness and as a baseline when comparing flaky or failing runs.

Failed

Appears when execution encounters an error or an assertion does not match expected behavior. Open the failed run immediately, review the failing step, screenshot, and error output, then classify the failure before applying a fix.

Aborted

Appears when a run ends before completion. Partial results may exist and should be interpreted with care. Review why the run ended and re-run if you need a complete result.

Stopped

Appears when someone clicks Stop during an active run. Like Aborted, the run did not finish — re-run if you need a full pass/fail outcome.

Scheduled

Scheduled is not a result status on a finished run. It means a future plan run is registered on a schedule.

Scheduled

A plan run is planned for a future date and time. Verify schedule timing if expected runs do not start. See Scheduling test plans and Dashboard overview for Next Scheduled Runs.

note

While a run is active, the run detail page can show a live Running state with elapsed duration. That is an in-progress indicator, not a final result status.

Where to view status

Test Case list

Shows last run Result at a glance across cases.

Run Detail

Shows per-step results inside a single execution.

Execution History

Shows run-level status over time for previous runs.

Dashboard

Shows trends and recent plan results for monitoring.

Filtering by status

Status filtering is available in the Test Cases Result view only.

warning

Execution History does not currently support filtering by status.

Comparing runs

Run comparison means reviewing two (or more) past executions side by side in Execution History—not a separate visual-regression or pixel-diff product. Use it to see why a case passed yesterday and failed today, or whether a failure is flaky.

What you typically compare:

Overall status

Passed vs Failed (or Aborted / Stopped) across runs of the same case, suite, or plan.

Failing step

Which step number failed in each run, and whether it is the same step every time.

Screenshots

Step images from each run (when Capture Screenshots was enabled). See Screenshots.

PDF evidence

Download a PDF report for each run if you need to share both results.

How to compare two runs

1

Open Execution History

Open the test case, suite, or plan and go to the Execution History tab.

2

Open the newer run

Open the latest failed (or unexpected) run and note the failing step, status, and screenshots.

3

Open an earlier run

Go back to Execution History and open a previous passed run (or another failed run) of the same asset.

4

Diff the evidence

Compare step outcomes and screenshots for the same step number. If results flip without an app change, treat it as possible flakiness; if the UI screenshot changed, update steps or enable Auto-Heal.

info

QAlity does not provide automated visual regression (pixel matching) or a dedicated “diff two runs” UI. Comparison is manual review of history, steps, screenshots, and PDFs.

To filter by status in Test Cases Result:

1

Open Test Cases Result

Open the Test Cases Result view.

2

Open filters

Open the available filters panel.

3

Select statuses

Select one or more statuses to narrow results.

4

Apply

Apply the filter and review matching test cases.

5

Refine

Refine with additional criteria such as time range when needed.

What to do when a test fails

Use this workflow to diagnose and resolve failures quickly:

1

Read the failing step and error details

Open the failed run and identify exactly where failure occurred. Confirm whether it happened during navigation, interaction, or assertion.

2

Classify the failure source

Decide what kind of issue you are dealing with:

App bug

Behavior is incorrect in the application.

Data issue

Required data is missing, stale, or invalid.

Test issue

Selector, wait strategy, or assertion is inaccurate.

3

Apply the correct fix

Update unstable locators, improve waits for asynchronous UI, correct or reset test data, or adjust assertions to match valid expected behavior.

4

Re-run and confirm

Re-run the affected case or suite. Verify the previously failing step now passes, and watch for repeated failures that may indicate flakiness.

Tips

Daily trends

Monitor status trends daily so regressions are detected early, not after large batches fail.

Prioritize

Focus on failures that are new or clustered in the same feature area.

Compare runs

Open a failed run and a recent passed run in Execution History and compare the failing step and screenshots. See Comparing runs.

Act fast

Act on failures quickly while context is fresh, then re-run to validate the fix end to end.

tip

For the fastest triage, review status first, then move immediately into run details for the first failing step.