16 · Benchmark | RePro Help Center

The Pipeline Benchmark exercises every source-to-output combination your machine can build — sweeping codec, resolution, bitrate, chroma/bit depth and HDR — then checks the actual captured output frame by frame. Use it to prove a machine's encoders and devices really deliver each format before a show, and to produce a shareable HTML report you can send to Repro support.

⚠️ Never run the benchmark during a live broadcast. Each run snapshots, removes, and then restores all your source and destination rows for the length of the run. Your setup comes back identically when the run finishes (or when you Stop it, or after a crash), but while it runs your rows are gone. Confirm you are off-air first.

Open it

The benchmark lives in the menu bar: Tools → Pipeline Benchmark…

If you don't see the Tools menu, switch it on first: Settings → Diagnostics → Enable pipeline benchmark. The menu appears immediately — no restart. Turn it back off when you're done diagnosing.

While a run is in progress, a live Benchmark Bot readout appears in the main app's Pipeline Flow viewer — see The Benchmark Bot below.

Before you start: ffmpeg

The benchmark needs ffmpeg and ffprobe (with SRT support) installed and on your PATH. If they're missing, an orange banner lists what to install with a Re-check button, and Start stays disabled until the check passes (you'll then see a green ffmpeg ok line).

Common workflows

Quick smoke check

Confirms the basic pipelines build and start.

Make sure you are NOT live on air.
Open Tools → Pipeline Benchmark…, clearing the ffmpeg banner if it appears.
Leave Sources on Built-in generator (no real source needed).
In the Matrix panel set Phase to smoke, with Inputs = SRT + NDI and Outputs = SRT + NDI.
Click Start (top-right) and watch the progress bar and PASS/FAIL counters fill in.
When it finishes, click Open HTML report, or Copy as Markdown to paste results elsewhere.

Full capability matrix (including HDR) and share the report

Confirm you are off-air.
Open Tools → Pipeline Benchmark…
In the Matrix panel set Phase to all (or hdr to test only the HDR axis).
Choose the Inputs and Outputs you care about. HDR is fully measured on SRT output; NDI and DeckLink HDR report unverified.
Optionally type a Tag (e.g. a machine name) to label the run.
Click Start and wait — progress shows in the header and the window title.
If failures occur, accept the Retest now prompt (or let the countdown auto-start it) to re-test just the failures.
When done, click Open HTML report, or use the Recent runs panel's reveal button to find the run folder (it contains report.html) to email to support.

Test a DeckLink/SDI card's output for real

Without a loopback cable, DeckLink output cases are only a build/start smoke. To check real frames, native caps and embedded audio:

Wire an SDI cable from the DeckLink card's OUTPUT back to one of its capture INPUTS (loopback).
Open Tools → Pipeline Benchmark…
In Sources, set DeckLink out device to the card.
Tick SDI loopback — this adds DeckLink to the Outputs list automatically.
Set Loop-in device to the capture input receiving the looped signal. Leaving it on (none) drops DeckLink cases back to a smoke test.
Pick a Phase (e.g. resolution or hdr) and click Start. Looped DeckLink cases now report real frame/caps/audio verdicts.

See the Hardware Capture page for more on DeckLink devices.

Test against a real source

By default every NDI/SRT input uses the built-in generator (SMPTE bars + 1 kHz tone). To test a real feed:

Before opening the benchmark, add and verify the source row(s) in the main Pulsar UI. (See Introduction & core concepts.)
Open Tools → Pipeline Benchmark… and click Refresh rows in the Sources panel if your row isn't listed.
Pick your row in the NDI source / SRT source / EXT source dropdown (leave the rest on Built-in generator).
Configure the Matrix and click Start. Remember your rows are removed for the run and restored at the end.

Re-run only the failed cases

Open Tools → Pipeline Benchmark… and scroll to Recent runs.
Find the run with failures (red F pill).
Click its retest failures (N) button to re-run only the failed cases — the result nests under the parent run with a ↳ REMEDIATION RUN badge.
Use retest all instead to re-run that run's entire matrix.

Controls you'll touch

You set up a run in the Sources and Matrix panels, then Start. Everything is locked while a run is in progress — Stop first to change anything.

Sources

Control	What it does
NDI / SRT / EXT source	Which feed drives that input kind. Default is the built-in generator; pick a real row to test it. EXT is DeckLink capture (input only).
DeckLink out device	Which DeckLink card to play out to for output cases.
SDI loopback	Tells the bench a cable loops the card's output back to a capture input, so DeckLink output is verified for real. Ticking it adds DeckLink to Outputs.
Loop-in device	The capture input receiving the looped signal (shown when loopback is on). Leave it set, or DeckLink cases fall back to a smoke test.
Refresh rows	Reloads the source dropdowns from the main app.

Matrix

Control	What it does
Phase	Which slice of tests to run: `smoke`, `codec`, `resolution`, `bitrate`, `global`, `hdr`, `cruncher`, or `all`. Each option shows its test count.
Inputs	Which input kinds to exercise: SRT, NDI, EXT (DeckLink capture).
Outputs	Which output kinds to exercise: SRT, NDI, DeckLink (DeckLink needs a card).
retest failures at end	After a run with failures, offers to re-run just those to weed out flukes. Leave on.
stop on first fail/error	Aborts the whole run at the first failure. Off by default.
Tag	Optional label saved with the run (e.g. a machine name).

all runs smoke + codec + resolution + bitrate + global + hdr. It does not include cruncher (a capacity-stress profile) — select that explicitly if you want it.

Reading the results

As a run progresses you get coloured status pills, a segmented progress bar (one segment per case — click a finished one to jump to it), and a per-case table showing video/audio/orbit verdicts and a reason. Hover a row with a screenshot to preview the captured frame.

Buttons:

Open HTML report — opens the self-contained report.html in your browser.
Copy as Markdown / Copy as JSON — copies the results to the clipboard.
Copy log — copies the live log (handy for support).

Recent runs

Past runs are listed with their pass/fail counts. Per run you can: report (open its report.html), retest all, retest failures (N), reveal (open the run folder), and delete. Clear all removes every run. Deletes are permanent and prompt for confirmation.

The Benchmark Bot

While a run is active, a purple Benchmark Bot node appears beneath the output in the main app's Pipeline Flow viewer. It shows live what the bench is receiving, highlighting whatever the current test exercises:

bitrate → live Mb/s
resolution → resolution
codec → H.264 / H.265
HDR → HDR PQ / HDR HLG (struck through if HDR wasn't detected)
smoke → Connection state
global / all / cruncher → fps (or Mb/s)

A phase badge shows the current test stage. It's read-only.

Understanding the verdicts

Not every non-pass result is a problem:

PASS (green) — output produced and measured values matched.
FAIL (red) — output produced, but values didn't match what was requested.
ERROR (orange) — the capture failed; no data was produced.
PARTIAL (amber) — one of video/audio passed and the other failed.
UNVERIFIED (magenta) — not a pass. Frames flowed but the requested HDR/format couldn't be measured on that output (e.g. NDI, or DeckLink with no loopback wired).
UNSUPPORTED (slate) — not a failure. The device genuinely can't do that format. Doesn't count against you.
ENV-BLOCKED (indigo) — something in your environment blocked the case (e.g. a VPN or firewall), not a Pulsar issue.

HDR caveat: HDR is only fully verified on SRT output. NDI HDR can't be measured, and DeckLink HDR needs the SDI loopback cable wired — otherwise both show UNVERIFIED. See Encoding, bitrate and color.

Tips & gotchas

It rewrites your setup. Every run removes then restores all your source/destination rows. Never run on air. Rows come back on normal exit, Stop, or even a crash.
ffmpeg + ffprobe (with SRT) must be on PATH, or Start stays disabled.
Inputs default to the built-in generator (SMPTE bars + 1 kHz tone) — select a real row to test one.
all doesn't include cruncher — pick it explicitly for the capacity-stress profile.
SDI loopback: ticking it adds DeckLink to Outputs; leaving Loop-in device unset downgrades DeckLink cases to a smoke test.
Closing the window won't orphan a run — reopening it re-attaches to the live run and catches up.

Platform notes

The benchmark works identically on macOS and Windows. The only differences are hardware: DeckLink out device, SDI loopback, Loop-in device and the EXT input all need a Blackmagic DeckLink card. reveal opens the run folder in Finder (macOS) or Explorer (Windows).