> ## Documentation Index > Fetch the complete documentation index at: https://docs.mobster.dev/llms.txt > Use this file to discover all available pages before exploring further. # Flaky checks and shards > Retry, measure and quarantine flaky checks, split a suite into shards, and rerun what failed. A check that passes on a retry is flaky. Mobster marks it, keeps its history per device, and suggests quarantine. It never quarantines a check on its own. ## Retry a failed check From your project folder, retry each failed check once and fail the suite if a retry passes: ```sh theme={"theme":"css-variables"} mobster test --retries 1 --strict ``` Mobster tries a failed run again once (`--retries N`, from 0 to 5) with the check's own reset, which reinstalls the app or clears its data unless the check says otherwise. A check that couldn't run because of the machine (WebDriverAgent, the simulator, the launch, a busy device) gets another try too. A usage error, a missing key, the spend cap, a refused app or a phone that stopped answering mid-run (`device_lost`) doesn't. A check that passes on a retry passed, and is **flaky**: the summary marks it `~ flaky`, the report lists it in its Flaky table, and JUnit gives it the property `flaky=true`. ## Measure a flaky check ```sh theme={"theme":"css-variables"} mobster test --repeat 5 .mobster/checks/onboarding.yaml ``` With `--repeat K`, each check runs K times, retries are off unless you pass `--retries`, and each result has a pass rate. A check passes only when all K runs passed. For a check with steps, first record a passing path with `mobster test --fresh CHECK`, then run `mobster test --repeat K CHECK`. Repetition needs that complete approved path with no model, and a missing path refuses before app reset, install or launch. `--fresh --repeat` and `--spread --repeat` are refused. Launch-only checks can repeat without a model. The repeated run pins the check and its expectations, its fixture and condition files, the selected model, the full iOS runtime, and the approved replay path for each device. If one changes before a later attempt, Mobster refuses that attempt before app reset, install or launch. ## Know what a suite pins Every suite pins each check's file, effective steps and assertion files, and the bytes at `app.path`, when it starts. If a pinned input changes before a later attempt, or a fixture, linked step or accessibility baseline changed between planning and the start, Mobster refuses that attempt before app reset, install or launch, and the suite stops with exit 3. Mobster can't pin a missing file, a symlink, a seed folder holding a symlink, a seed or media file over its limit, an app holding a symlink, or an app over 2 GB or 50,000 files. That input stays unpinned, the run prints a note naming it, and its digest is `null` in `results.json`. The check still runs, and the other checks carry on. A missing app fails only its own check, and so does a symlink in a seed folder. `--repeat` refuses an app it can't pin, for that check only, because its runs could otherwise span two builds of one version. ## Quarantine a flaky check A check you know is flaky can keep running without failing the run. List it in `.mobster/quarantine.yaml`: ```yaml theme={"theme":"css-variables"} - onboarding # a check's file name, its path, or its name - check: restore-purchases reason: Flaky on iOS 26.4 (FB13579) ``` A quarantined check runs, and its ordinary failures appear as JUnit `` without changing the exit code. A safety stop still ends the suite and appears as a JUnit error. `--quarantine FILE` reads another file, and stderr says when a listed name matches no check. ## Find your flaky checks Every `mobster test` run adds each check's result, per device, to `.mobster/test-results/history.json`, and keeps the last 20. The folder is in `.mobster/.gitignore`, so the history stays on your Mac. ```sh theme={"theme":"css-variables"} mobster test --history ``` ```text theme={"theme":"css-variables"} flaky Feed loads on iPhone 17 Pro ✗✓✗✓✗✓✓✗✓ 56% of 9 runs, flake score 0.88, slower Flaky: add .mobster/checks/feed.yaml to .mobster/quarantine.yaml, or run with --auto-quarantine. stable Onboarding on iPhone 17 Pro ✓✓✓✓✓✓✓✓✓ 100% of 9 runs, flake score 0.00 ``` `--history` runs nothing and takes a project path. `--json` prints the same rows as JSON. Each row has the pass rate, the duration of every run, the duration trend (`slower`, `faster` or `steady`, newer half of the runs against the older half) and a flake score from 0 to 1. Under each row, `Last run:` gives the launch time, memory and hangs of the check's last run ([Performance](https://docs.mobster.dev/testing/performance.md)). The flake score is the larger of two shares: the runs that passed only after a retry or a repeat that failed, and the neighboring runs where one passed and the other didn't. It counts only verdicts of the same check text on the same app build, so a change after you edit the check or rebuild the app isn't a flake. A run that ended `budget_unmeasured` counts in neither share. With 5 or more runs a check is: | State | Meaning | | - | - | | `flaky` | The score is 0.15 or more and the last run passed | | `failing` | The last run didn't pass. It fails the same way each time, or began to | | `stable` | Everything else, including a check that failed for a while and was fixed | | `new` | Fewer than 5 runs | A run that couldn't run (a lost device, a busy phone, a stopped suite) says nothing about the check, and Mobster doesn't keep it. Mobster suggests quarantining a flaky check in the summary, in `results.json` (`history.suggestQuarantine`, and `history` on each result) and in the HTML report's "Flaky over time" table. With `--auto-quarantine`, Mobster quarantines the checks the history calls flaky, for that run only: they run and are reported, and they don't change the exit code. It never does this for a check whose latest result failed on any device, and `.mobster/quarantine.yaml` stays as you wrote it. A history file Mobster can't read is set aside as `history.json.corrupt`, and the history starts again. Mobster for Mac reads the same numbers from `GET /api/tests/history`, with `?project=PATH` for one project. ## Split a suite with tags and shards Give a check `tags` (`tags: [smoke, checkout]`) and pick by them: `mobster test --tag smoke`. A tag no check has is a usage error, so a typo can't pass CI by running nothing. `--shard 2/4` runs the second of four even shards, every fourth check in file order, so four CI machines split one suite. ## Rerun the checks that failed > [!NOTE] > > You get this in the next release, not in Mobster for Mac 0.4.0 or Mobster CLI 0.4.3. `mobster test --failed` runs only the checks that didn't pass in your project's newest finished run: failed, couldn't run or needs review on any device. A check that passed on a retry counts as passed, unless that run used `--strict`. A quarantined check that failed still runs. The last run is the newest suite in `.mobster/test-results`. If it ran only some of the checks, `--failed` picks from those. The rerun uses the checks as your project is now, on the devices you select for this run: the simulators in `.mobster/matrix.yaml`, or each check's own simulator. Its first line names the suite it picks from, and the device change when the devices differ. `--failed` runs nothing and exits 3 when: * the newest suite has no results yet, because it is still running or stopped first * a check that didn't pass has no file in your project now, unless it is quarantined * an XCTest result didn't pass and you didn't add `--xctest`, or you limit the tests with `--only-testing`, or `--scheme` or `--test-plan` names something the last run didn't use * your project has no finished run ```text theme={"theme":"css-variables"} mobster test: --failed has nothing to run because there is no finished run. Run `mobster test` to run every check ``` When no check needs a rerun, `--failed` exits 0 if the last finished run passed, else 3: ```text theme={"theme":"css-variables"} mobster test: you have no checks to rerun from your project's last finished run, 20261109-090000-aaaa. Run `mobster test` to run every check ``` You can't combine `--failed` with check paths, `--check`, `--tag` or `--shard`. Each machine's last run is already its shard, so `mobster test --shard 2/3 || mobster test --failed` reruns the checks that didn't pass in that shard. `failed: true` on `run_tests` selects the same checks over MCP, and **Run failed checks** does it on your project's page ([Projects](https://docs.mobster.dev/projects.md#run-checks)). This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.