Skip to content
View in the app

A better way to browse. Learn more.

Learn more

Our approach to fleet-wide scripts in Tactical RMM, plus a Cloudflare 1010 surprise

Featured Replies

Running a script on one machine is easy. Running it on four hundred machines across a dozen clients, where half of them are laptops that drift in and out of the office, is how you find out whether the script was any good. We use Tactical RMM (TRMM) for this, but most of what follows carries over to any RMM. It's mostly about knowing what account the script runs as, getting output you can actually read afterwards, and not pushing to everyone on the first try.

What to have ready

  • A working Tactical RMM instance with agents deployed.
  • An account that can manage scripts and run them on agents.
  • An API key (Settings > Global Settings > API Keys) if you want to drive it from your own tooling.
  • One or two test machines you're allowed to break.

Keep a library, not a pile of one-offs

Anything you run more than once goes in the Script Manager with a clear name, a category, a description of what it changes, and a sensible default timeout. TRMM supports PowerShell, batch and Python. Ad-hoc commands from the console are fine for poking around, but if it changes state it belongs in the library, where someone else can review it and re-run it exactly the same way.

Remember it runs as SYSTEM

By default TRMM scripts run as SYSTEM. That's what you want for installing software, changing services, editing HKLM or anything else machine-wide. It also means a few things catch people out:

  • HKCU is SYSTEM's hive, not the user's.
  • Mapped drives and user-profile paths aren't what you expect.
  • Network access goes out as the computer account.

The Run As User option runs the script in the logged-on user's session instead. It only works when somebody is actually signed in, so it's a poor fit for an overnight fleet run. For per-user settings we usually run as SYSTEM and loop through the user hives, or deploy the change somewhere that applies at sign-in.

One script with parameters beats five copies

Instead of keeping five near-identical scripts, accept parameters and pass them from the script's default arguments or at run time:

param(
    [string]$ServiceName = 'Spooler',
    [ValidateSet('Check','Restart')][string]$Mode = 'Check'
)

Arguments are typed the way you would on a command line (for example -ServiceName Spooler -Mode Restart). Never pass secrets as plain arguments, because they end up in run history. Use the RMM's custom fields or environment variables where you can, and limit who can view history.

Write it so running it twice is harmless

A fleet script will hit machines in every state you can imagine, and someone will run it twice. Check before you change anything, make a second run a no-op, and report clearly. We start every output line with a fixed status word so results can be grouped later, and we always set an exit code:

# Ensure-BitLockerRecoveryEscrow.ps1 - safe to run repeatedly
$ErrorActionPreference = 'Stop'
try {
    $vol = Get-BitLockerVolume -MountPoint $env:SystemDrive
    if ($vol.ProtectionStatus -ne 'On') {
        Write-Output "SKIP: BitLocker not enabled on $env:SystemDrive"
        exit 0
    }
    $kp = $vol.KeyProtector | Where-Object KeyProtectorType -eq 'RecoveryPassword'
    if (-not $kp) {
        Write-Output "FAIL: no recovery password protector"
        exit 1
    }
    BackupToAAD-BitLockerKeyProtector -MountPoint $env:SystemDrive -KeyProtectorId $kp[0].KeyProtectorId | Out-Null
    Write-Output "OK: escrowed $($kp[0].KeyProtectorId)"
    exit 0
}
catch {
    Write-Output "FAIL: $($_.Exception.Message)"
    exit 1
}

Don't skip the exit code. When a script is used as a check or an automated task, the RMM uses the return code to decide pass or fail. A script that catches every error and exits 0 will show green forever, and whoever's watching the dashboard has no idea machines are failing underneath it.

Roll it out in rings

Order matters here. This is the sequence we stick to:

  1. Run on one test machine and read the full output.
  2. Run on a small, mixed group: a desktop, a laptop, an older OS build, a machine with a non-admin user.
  3. Use bulk script against a single client or site.
  4. Only then target everything, or attach it to an automation policy.

Put recurring work on a policy

For anything recurring, add the script as an automated task on an automation policy instead of running it by hand every week. Policies apply at the client, site or agent level, so you can pilot on one site before widening it. Offline agents pick up their tasks when they come back, which beats chasing laptops around manually.

Counting the results

Export the bulk results and count statuses. Eyeballing hundreds of rows is how things get missed.

# results.csv exported from the bulk run: hostname, output
Import-Csv .\results.csv | ForEach-Object {
    $status = ($_.output -split ':')[0].Trim()
    [pscustomobject]@{ Host = $_.hostname; Status = $status; Detail = $_.output }
} | Group-Object Status | Select-Object Name, Count

Every machine should land in OK, SKIP or FAIL. Anything with no output at all usually timed out or was offline, so rerun those separately.

Calling TRMM from your own scripts

Once you're comfortable, you can trigger runs from your own tooling. Authentication is an X-API-KEY header. Check field names against your version, since they've changed over time. The easiest way I've found is to watch the request the web UI sends in your browser's developer tools.

$base    = 'https://api.rmm.contoso.com'
$headers = @{ 'X-API-KEY' = (Get-Secret -Name 'TrmmApiKey' -AsPlainText) }
# Cloudflare Browser Integrity Check blocked our POSTs without this
$ua      = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0 Safari/537.36'

# GET worked even without the User-Agent
$agents = Invoke-RestMethod -Uri "$base/agents/" -Headers $headers -UserAgent $ua

# POST returned 403 / error 1010 until -UserAgent was added
$body = @{
    script      = 42            # script ID from the library
    args        = @('-Mode', 'Check')
    output      = 'wait'
    timeout     = 120
    run_as_user = $false
} | ConvertTo-Json
$agentId = '<AgentID>'
Invoke-RestMethod -Method Post -Uri "$base/agents/$agentId/runscript/" `
    -Headers $headers -UserAgent $ua -ContentType 'application/json' -Body $body

The 1010 error, and a few others

Our RMM API sits behind Cloudflare. GET requests worked perfectly, which made the next part confusing. Every POST and DELETE came back 403 with Cloudflare error 1010. Error 1010 means Cloudflare blocked the request based on the client's browser signature, usually via Browser Integrity Check. Scripted clients send a non-browser User-Agent, and only the write requests were being judged that harshly. Sending a normal browser User-Agent header fixed it straight away. The cleaner long-term fix is a WAF rule that skips Browser Integrity Check for the API hostname, restricted to your own source IPs.

  • Timeouts are often too short. Installers on slow laptops easily run past a 30-second default, and a timed-out script may still be running on the endpoint.
  • "It works when I run it" usually means it works as you, in your session, with your drives mapped. SYSTEM is a different world.
  • Keep output short and structured. A script that dumps a full event log for every machine makes the results unreadable.

Create an account or sign in to comment

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.