Skip to content

Reading the results

The Results step turns a run into a decision. It opens with the answer, then lets you dig into why, compare configurations fairly, and keep the one you choose.

The Results step: a green banner reading Recommended with the winning settings and why they beat the next row, and the ranked results table. The Baseline row comes first, marked Base, then Best, 2nd, 3rd, 4th and 5th, with a Save as Scenario link on each. A footnote under the table says only one configuration is marked Best.

A band at the top says what the run can honestly claim, and PM Optimizer picks one winner: the configuration with the best overall score, every checked response counted at its weight.

The band names it, “Recommended:” and its settings, and says why it beat the next row, response by response, up to three of them, the ones that moved the score most first, such as a response that came out higher or lower by an amount. For the lead response it then compares the winner with the baseline: how much better or worse it is, at 95 percent confidence, or that the ranges overlap, so the difference is not certain yet, or that it matches the baseline. With one replication per point the run records no baseline result, so the band says there is nothing to compare against and asks for 2 or more replications.

When the winner and the row after it score the same within the run’s noise, the winner is the one that uses less, and the band says so: it “uses 1 fewer resource”, “uses less” at a named setting, or “changes 1 fewer setting from today’s model”.

When the leaders cannot be told apart, the band says so plainly: “N configurations are tied for best”, with their overall scores equal within the run’s noise. If more replications could still help, it advises running more or treating them as equally good. Where the search already ran them again, it says that more replications are not going to separate them, and advises treating them as equally good. Where the tied rows each use less at a different step, it names who uses less where and says “Choose the one that suits you.” It never claims a single perfect answer it cannot back up. If the analysis behind the band fails, Try again appears beside it.

Ties are settled in the run itself, not something to click afterwards. Replication k of every configuration runs on the same random numbers, so configurations are compared run by run. While the search runs, tied leaders automatically earn extra replications, the whole tied group together, until the group stops changing, except a group whose tie is already settled by what each configuration uses, which earns none.

Where configurations score the same within the run’s noise, the app breaks the tie by what each uses, in this order:

  1. Fewer resources used.
  2. Lower capacity, setting against setting. A configuration wins only if it is lower or equal at every capacity setting and lower at one or more. If one is lower at one setting and higher at another, nothing is decided.
  3. Fewer settings changed from today’s model, only where capacity is equal everywhere.

A tie that survives that, because the rows each use less at a different step or are equal on all three, is a real tie, and you choose among those rows yourself.

Below it, every configuration is listed, ranked by its overall score, which adds every checked response by its weight: Rank, Status, Reps (how many replications the row ran), the factor settings, each response with its spread, and Save as Scenario at the end of the row. Sort by any response with the control in the header, and use Export Results beside it to save the full table as a spreadsheet.

A Baseline row leads the table: your model today, unchanged, with its settings filled in as the run recorded them, so you can always see what the search was competing against. It is never ranked, and its Rank cell reads Base. For an experiment that started from a scenario, the Baseline row measures that scenario as seeded, switched care models and removals included, and the Started from caption names the scenario.

One row alone is marked Best only when, at 95 percent confidence, the run separated its score from every other configuration. A row below it can still show a better number on one response: the score weighs them all, or its own results varied more from one replication to the next, so the run cannot say it is ahead. Where several rows are in the best group, each is marked Tied and they share first place, so the table says so rather than pretending the top row is uniquely best.

The Rank cell is the row’s place, and a row that went over a limit reads None. The Status column gives every row a plain account:

  • Best: the one row that is alone in the best group.
  • Tied: one of several rows in the best group, all sharing rank 1.
  • 2nd, 3rd, 4th and so on: a place below the best group.
  • Tied 3rd and so on: rows below the best group that are within the run’s noise of each other share a place, and the next place counts every row above them (Tied, Tied, 3rd).
  • Leader, refined: it earned extra replications as a leader and still could not be separated from the pack.
  • Explored: the first pass every configuration gets.
  • Cut short: the search could already tell it was losing and stopped spending runs on it, with the reason on the row.
  • Over a limit: it finished but broke one of your limits, so it is out of the ranking.
  • Failed: it did not finish, with the reason spanning the row.

Main Effects on Urgent: Completed count, a bar for each of three factors with its change (+7, +3.67 and +2) and the footnote about a factor's two settings, beside Compare With Baseline, which plots Baseline, Rank 1 and Rank 2 against Today and offers the box Show the spread across replications.

The main effects chart shows how much each factor changed the lead response, between a factor’s two settings, or for a factor tried at three or more settings, between its lowest and highest averages, biggest mover at the top. Compare with baseline puts the leading configurations next to your current model as points with 95 percent confidence ranges, so a difference only reads as real when the ranges are clear of each other. For the full statistics behind the picture, open the detail from the button below the chart.

Under each main effect, Level averages opens a small table: for each level of that factor, the average response, with the number of Points and Runs behind it. Under the comparison, Show the spread across replications draws, behind each interval, the lowest to the highest replication, the middle half of them and the median. It needs at least 5 replications per point. With fewer, a note says so: “Spread needs at least 5 replications per point. This run used 2.”

The sweet-spot map, predicting the best area for two factors.

For two numeric factors, the sweet-spot map predicts the result across the whole area from the points that ran, with darker meaning better for your goal and a star on the best predicted point. It fills in the gaps between the configurations you actually ran, and tells you how much to trust the prediction.

At the foot of the step, a panel called What This Run Could Not Answer lists, in plain words, whatever the analysis had to work around. It appears only when there is something to say.

When you have chosen, Save as Scenario copies that configuration into your model as an ordinary scenario, enabled and ready to run. When the button cannot act it is grayed out, and the reason is its hover tooltip: for example, the model has changed since the run was saved, the configuration uses something that is no longer in the model, or the run started from a scenario that has since changed or been deleted. In that last case the tooltip names the scenario and says it “has changed since” or “has been deleted since”, and a renamed scenario reads as deleted. Each reason ends with advice to run the experiment again, then save from the fresh result. A run saved before version 7.0.9313 keeps the winner it was saved with.

The last completed run is saved with the model, so closing the window loses nothing: reopen the model later and PM Optimizer opens straight back on this step, ranked table and all. A line above the results says where they came from, in the form “Results from Oct 3, 2026, saved with this model.”

That holds for a saved experiment. A run belongs to the experiment it ran for, so it is kept with the model only once that experiment is saved. A new draft, or a definition you have edited since the run, has no saved experiment to keep the run with: save it with Save Experiment. Saving a changed definition (a rename does not count) drops the finished run and its saved results, and the Run and Results steps lock again, as they are for a new experiment. When you leave an experiment with unsaved changes, a question offers Save, Discard or Keep editing, and when a finished run would go with them it adds “and the results of its finished run go with it.”