INCLUDED WITH SEMANTICOPS PRO

TEST
RUNNER

Guided unit testing for Power BI semantic models.

Build test suites in a desktop UI. Run them against your model. Catch regressions before changes ship.

View SemanticOps Pro How to install

Watch the 20-minute walkthrough

Native macOS and Windows app

8 built-in test types and 6 starter packs

Power BI Desktop External Tool integration

WHAT YOU CAN TEST

Eight supported test types.

Pick one from the Create Test wizard. The same eight cover measures, queries, security, performance, drift, and metadata.

Measure Assertion
Assert measure values with tolerance.
DAX Assertion
Assert manual DAX query results.
RLS Validation
Validate row-level security filtering by role or identity.
OLS Validation
Validate table and column access expectations.
Performance Budget
Enforce query duration thresholds.
Regression Snapshot
Compare current query output to a stored baseline.
Referential Integrity
Detect orphan keys in relationships.
Metadata Compliance
Lint descriptions, naming, formatting, folders, and model metadata.

START IN ONE CLICK

Apply a starter pack.
Get a suite in seconds.

Preview a built-in test pack against your model before applying. Pick the pack, see exactly what will be created, then apply only what fits.

Metadata QualityDescriptions, naming, formatting
Documentation BaselineTables, columns, measures
Presentation HygieneFormat strings, display folders
Referential IntegrityOrphan keys in relationships
Relationship GovernanceCardinality, direction, key visibility
Time IntelligenceDate table semantics, continuous ranges

INSIDE THE APP

Four steps from a fresh dataset to a passing suite.

Connect, build, run, review - the workflow you keep coming back to.

01CONNECT

Connect to Power BI in seconds.

Power BI Desktop on Windows, or Power BI Service from macOS. Pick a workspace, pick a dataset, and you are testing.

A: Workspace + dataset picker for Power BI Service.
B: Connection status in the app header stays visible while you work.

Connect to Power BI dialog over the Test Runner Overview screen, showing a workspace folder named PBI MCP Testing with two datasets to choose from.

02BUILD

Author tests through a guided wizard.

Pick the test type, point at a measure, set filters, and define the assertion. No MCP payload to hand-author.

A: Six-step wizard: Type, Details, Measure, Context, Assertion, Review.
B: Live test summary updates as you configure each step.

Create Test wizard on the Context step, building a Measure Assertion for Employees Headcount with two evaluation filters.

03RUN

Run the full suite or drill into one.

Filter by folder, status, or type. Group by type. Run all from the header, or run any single test inline.

A: Stat strip: pass rate, fail count, average duration, flaky review, last run.
B: Inline per-row run button for fast iteration on one assertion.

Tests view listing 24 tests grouped by type with stat cards above showing Total, Passed, Failed, Average Duration, Flaky, and Last Run.

04REVIEW

Track trends and drill into failures.

Persisted run history with pass-rate sparklines across sessions. Detect flaky tests, compare runs, and export results when you are ready.

A: Pass-rate trend tile compares the latest run to the previous one.
B: Per-run sparkline gives at-a-glance flakiness across history.

Run History view with five stat cards across the top and a list of recent runs, each with a pass-rate sparkline, test count, duration, and timestamp.

VIDEO

Watch the walkthrough.

The long-form tour for when you want every surface in motion. Or the 60-second pitch if you are short on time.

Full walkthrough

Connect, build, run, review - the long-form tour through every surface.

20 min

Quick tour

The 60-second pitch.

1 min

INSTALL

Install for your platform.

Windows installers register Test Runner as a Power BI Desktop External Tool. macOS installers connect to Power BI Service.

Download Test Runner

Standard Windows installer

Registers the Power BI Desktop External Tool.

01Download and run the Windows installer.
02Approve the External Tool registration when prompted.
03Restart Power BI Desktop.
04Launch Test Runner from the External Tools ribbon.

Power BI Desktop External Tool is Windows-only. macOS connects to Power BI Service.

FAQ

What people ask.

Is Test Runner included with SemanticOps Pro?

Yes. Every Pro subscription includes the desktop app on the same MCP-backed testing surface.

Where are tests, baselines, and run history stored?

SemanticOps MCP stores tests, baselines, and run history. They persist across sessions, not in the desktop UI.

Is this a standalone CI runner?

No. Test Runner exports JUnit XML, Markdown, HTML, and JSON for automation, but CI execution is a separate workflow.

What platforms are supported?

Windows runs the Power BI Desktop External Tool flow. macOS runs the same app and connects to Power BI Service.

How does masking behave in the desktop app?

Test Runner reads model values directly so test results are accurate. Masking policies in other MCP hosts are unaffected by what you see here.