Skip to content
View in the app

A better way to browse. Learn more.

Learn more

The bulk rename that ate an invoice, and the dry-run habits we use now

Featured Replies

Bulk renames look trivial. Loop over the files, work out a new name, move. The trouble starts when two inputs map to the same output, because a plain mv or Move-Item -Force overwrites the first file with the second and says nothing. We learned this the hard way normalising a folder of scanned documents for a client. Invoice 001.pdf and invoice_001.pdf both became invoice_001.pdf, and one of them was simply gone.

These days every batch job I write follows the same habits. Here they are, with working bash and PowerShell versions.

Before you run either script

  • bash 4 or newer for the associative array in the bash version. macOS ships bash 3.2, so install a newer bash or drop the collision map.
  • PowerShell 5.1 or 7.x for the PowerShell version.
  • A copy of the data, or at least a backup, for the first real run.

The rules I follow now

  1. Dry run by default. The script only prints what it would do unless you explicitly tell it to act.
  2. Detect collisions in the plan, before anything moves.
  3. Never clobber. Check the destination, and use mv -n as a backstop.
  4. Log every change with old and new names, so you can undo it.
  5. Make it re-runnable. Files already in the target state get skipped, so running it twice is harmless and an interrupted run can just be restarted.

Bash

#!/usr/bin/env bash
# normalize-names.sh - lowercase names and replace spaces with underscores
# Usage: ./normalize-names.sh DIR           (dry run)
#        ./normalize-names.sh DIR --apply   (actually rename)
set -euo pipefail
DIR="${1:?usage: $0 DIR [--apply]}"
APPLY=0
[[ "${2:-}" == "--apply" ]] && APPLY=1
LOG="rename-$(date +%Y%m%d-%H%M%S).log"
declare -A planned   # target -> source, needs bash 4+

while IFS= read -r -d '' src; do
    dir=$(dirname "$src"); base=$(basename "$src")
    new=$(printf '%s' "$base" | tr '[:upper:]' '[:lower:]' | tr ' ' '_')
    [[ "$base" == "$new" ]] && continue          # already done: re-runnable
    dest="$dir/$new"

    if [[ -n "${planned[$dest]:-}" ]]; then
        echo "COLLISION  $src and ${planned[$dest]} -> $dest" | tee -a "$LOG"
        continue
    fi
    planned[$dest]="$src"

    if [[ -e "$dest" ]]; then
        echo "EXISTS     $src -> $dest (skipped)" | tee -a "$LOG"
        continue
    fi

    if (( APPLY )); then
        mv -n -- "$src" "$dest"
        if [[ -e "$src" ]]; then
            echo "FAILED     $src -> $dest" | tee -a "$LOG"
        else
            echo "RENAMED    $src -> $dest" | tee -a "$LOG"
        fi
    else
        echo "WOULD      $src -> $dest" | tee -a "$LOG"
    fi
done < <(find "$DIR" -maxdepth 1 -type f -print0)

A few details in there are easy to miss:

  • find -print0 with read -d '' handles spaces, quotes and even newlines in names. Never loop over $(ls).
  • mv -n (no clobber) refuses to overwrite an existing file. I don't trust its exit status to tell me it skipped, because that behaviour differs between platforms and coreutils versions, so the script checks whether the source still exists afterwards.
  • The -- before the filenames stops a file called -rf.txt being read as options.

PowerShell

PowerShell gives you dry runs for free. Declare SupportsShouldProcess, wrap the change in $PSCmdlet.ShouldProcess(), and -WhatIf works just like it does on built-in cmdlets.

# Normalize-Names.ps1 - run with -WhatIf first
[CmdletBinding(SupportsShouldProcess)]
param([Parameter(Mandatory)][string]$Path)

$log = "rename-{0:yyyyMMdd-HHmmss}.csv" -f (Get-Date)

$plan = Get-ChildItem -LiteralPath $Path -File | ForEach-Object {
    $new = $_.Name.ToLower() -replace '\s+', '_'
    [pscustomobject]@{ Source = $_.FullName; NewName = $new; Target = Join-Path $_.DirectoryName $new }
} | Where-Object { (Split-Path $_.Source -Leaf) -cne $_.NewName }   # skip already-normalized

# Two inputs mapping to the same output? Stop before touching anything.
$collisions = $plan | Group-Object Target | Where-Object Count -gt 1
if ($collisions) {
    foreach ($c in $collisions) { Write-Warning "Collision -> $($c.Name): $($c.Group.Source -join ', ')" }
    throw 'Resolve collisions before renaming.'
}

$results = foreach ($item in $plan) {
    if (Test-Path -LiteralPath $item.Target) {
        [pscustomobject]@{ Source = $item.Source; Target = $item.Target; Result = 'SkippedExists' }
        continue
    }
    if ($PSCmdlet.ShouldProcess($item.Source, "Rename to $($item.NewName)")) {
        try {
            Rename-Item -LiteralPath $item.Source -NewName $item.NewName -ErrorAction Stop
            $r = 'Renamed'
        } catch { $r = "Failed: $($_.Exception.Message)" }
        [pscustomobject]@{ Source = $item.Source; Target = $item.Target; Result = $r }
    }
}
if ($results) { $results | Export-Csv -LiteralPath $log -NoTypeInformation -WhatIf:$false }

Run it in two passes:

.\Normalize-Names.ps1 -Path D:\Scans\Incoming -WhatIf
.\Normalize-Names.ps1 -Path D:\Scans\Incoming

Rename-Item already refuses to overwrite an existing file, but we still test first so the skip lands in the log instead of showing up as an error. Keep Move-Item -Force out of rename scripts. -Force is exactly the overwrite you're trying to prevent.

Read the plan before you apply it

When the dry run finishes, I look for any COLLISION or warning lines first. Then any EXISTS lines, since a target that's already there usually means an earlier partial run or a genuine duplicate. Last, I check the count against what I expected. If the plan says it'll rename 4,000 files and you expected 40, stop.

Did anything go missing?

  • Count files before and after. The numbers must match. Renames never reduce the file count unless something got overwritten.
  • Run the script again in dry-run mode. It should report nothing to do.
  • Keep the log with the data. To undo, swap the columns and feed it back through the same no-clobber logic.

Where this has caught us out

  • Case-only renames on case-insensitive filesystems. On NTFS and default macOS APFS, Report.PDF to report.pdf looks like the destination "already exists" because it's the same file, so both scripts above will skip it. If case-only changes matter, go through a temporary name (Report.PDF to Report.PDF.tmp to report.pdf).
  • The in-script map only catches collisions within one run. The destination-exists check is what covers files left by earlier runs.
  • Write logs somewhere other than the target folder, or the next run may try to rename its own log.
  • OneDrive, Dropbox and DFS replication can lag or create conflict copies during a big rename. Pause sync or run it on the server side.

I use the same pattern for other batch jobs too, like bulk AD changes, mailbox moves and clearing out old backups. Show the plan, check for collisions, then act, log it, and make sure a second run won't hurt.

Create an account or sign in to comment

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.