Keaton

Windows Hello for Business (WHfB) lets people sign in with a PIN or biometric tied to the device's TPM instead of a password. In a hybrid environment, the hard part has always been getting on-premises resources like file shares, printers and Kerberos apps to accept that Hello sign-in. Get it wrong and the user signs in with a PIN, then gets a password prompt the moment they open a share. The older key trust and certificate trust models needed either a PKI with certificates on your domain controllers, or a wait while public keys synced back to AD. Cloud Kerberos Trust removes both. Entra ID issues a partial Kerberos ticket that your DCs trust. It's now Microsoft's recommended model for hybrid deployments, and it's what we deploy. Check these before you start Hybrid joined or Entra joined devices on Windows 10 21H2 or later (fully patched) or Windows 11. Domain controllers running Windows Server 2016 or later, fully patched, at every site where users will authenticate. Entra Connect syncing your users. Intune (or Group Policy) to deploy the WHfB settings. A Domain Admin account in the on-premises domain and a Global Administrator (or Hybrid Identity Administrator) account in Entra ID. Users who can complete MFA, since WHfB provisioning requires it. Why we went with cloud trust There's no PKI involved. You don't have DC certificates to issue and renew for Hello, and you don't have to worry about CRL availability. There's also no key sync delay. With key trust, a new user's public key had to sync to AD before on-premises SSO worked, so the instructions for new staff were basically "set up Hello, then wait up to an hour." With cloud trust it works right after provisioning. And if you plan to use FIDO2 security keys for hybrid sign-in, the same Entra Kerberos object supports that too. Create the Entra Kerberos server object This creates a read-only domain controller-style computer object (AzureADKerberos) in your domain and publishes its key to Entra ID, so your DCs can trust the tickets Entra issues. Run it from an elevated PowerShell session on a domain-joined machine that can reach a writable DC. # TLS 1.2 for the PowerShell Gallery on older systems [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 Install-Module -Name AzureADHybridAuthenticationManagement -AllowClobber $domain = $env:USERDNSDOMAIN # e.g. CONTOSO.LOCAL $upn = "admin@contoso.com" # Entra Global Admin or Hybrid Identity Admin $domainCred = Get-Credential # Domain Admin in the on-prem domain Set-AzureADKerberosServer -Domain $domain -UserPrincipalName $upn -DomainCredential $domainCred # Confirm it exists and the key versions match Get-AzureADKerberosServer -Domain $domain -UserPrincipalName $upn -DomainCredential $domainCred If your admin account requires MFA, the module supports an interactive sign-in instead of passing the UPN. Check the module's help for the current parameters. In AD you should now see an AzureADKerberos computer object in the Domain Controllers OU and a krbtgt_AzureAD user. Don't delete or move either one. Repeat this for each domain in the forest where users sign in. The WHfB policy in Intune Create a Settings catalog profile for Windows and search for Windows Hello for Business. At a minimum, set Use Windows Hello For Business to Enabled (with your PIN complexity and biometric preferences) and Use Cloud Trust For On Prem Auth to Enabled. Under the hood, the cloud trust setting is this CSP. ./Device/Vendor/MSFT/PassportForWork/<TenantID>/Policies/UseCloudTrustForOnPremAuth = true Assign it to a pilot device group. If you already have a tenant-wide WHfB configuration under Windows enrollment settings, make sure the two don't conflict. The settings catalog profile is the one we rely on. Without Intune, the equivalent Group Policy setting is "Use cloud Kerberos trust for on-premises authentication" under the Windows Hello for Business administrative templates. Try it with one user Sync policy on the pilot device and have the user sign out and back in. After MFA, they should be prompted to set up Windows Hello. Then lock the machine and sign in with the PIN. Confirming on-prem SSO works Run these as the user, while signed in with Hello. dsregcmd /status klist cloud_debug klist In dsregcmd /status, under SSO State, you want CloudTgt : YES and OnPremTgt : YES. Under the NGC prerequisite check, NgcSet : YES confirms the user has provisioned Hello. klist cloud_debug should report that the cloud primary (hybrid logon) TGT is available and that cloud Kerberos is enabled by policy. klist should show a krbtgt ticket for your on-premises realm, plus service tickets once you browse a file share. The test I care about most is opening \\fileserver\share while signed in with the PIN. No password prompt means on-premises SSO works. What tripped us up Privileged accounts don't work, and that's by design. The Entra Kerberos object behaves like an RODC, so accounts that can't have their credentials cached on an RODC, such as members of Domain Admins and other groups in the "Denied RODC Password Replication Group," won't get on-premises tickets from cloud trust. They'll sign in to Windows fine and then fail to reach on-premises resources. Admins should be using a separate daily account anyway. If any DC a client might talk to is missing the required updates, sign-ins fail intermittently depending on which DC answers. That's a miserable one to chase, so patch them all before the pilot. Devices previously set up for key trust or certificate trust may keep using it. Clean up old GPOs and policies, and have users reprovision if needed. Kerberos fails if the client and DC clocks drift more than about five minutes apart. Check time sync on remote and VPN devices. Getting the on-premises ticket still needs line of sight to a DC when the resource is accessed. Hello sign-in to Windows works off-network, but file shares still need the VPN. One last thing that's easy to forget once everything works. Rotate the Entra Kerberos server key periodically with Set-AzureADKerberosServer -RotateServerKey (using the same domain and credential parameters), and put it on your maintenance calendar so it doesn't depend on somebody remembering.

Every environment has little problems that keep coming back, like a service that keeps getting disabled or a registry setting an app keeps resetting. Each time, somebody calls the help desk and a tech fixes it by hand. Intune Remediations (previously Proactive Remediations) are how we stop doing that. You give Intune a pair of PowerShell scripts that run on a schedule, one to detect the problem and one to fix it. Below is how we write them, how we schedule and check them, and the Graph gotcha that left one of ours detecting problems for weeks without fixing a single one. Licensing and setup Licensing that includes Remediations, which means Windows Enterprise E3/E5 or Education A3/A5 (often through Microsoft 365 E3/E5 or A3/A5), or Windows VDA per user. Devices that are Entra joined or hybrid joined and managed by Intune. The Intune Management Extension is what runs the scripts. A test device group. How the two scripts work together The detection script runs first. exit 0 means everything's fine and there's nothing to do. exit 1 means it found an issue, and Intune runs the remediation script. The remediation script does the fix. After it runs, the detection script runs again so Intune can report whether the fix held. Whatever the scripts write to STDOUT shows up as the detection and remediation output in the portal, and the last line is what you see in reports. Make that line useful to whoever reads the report later, because "ok" doesn't tell them much. The detection script This example makes sure the Windows Time service is set to automatic and running. # Detect-W32Time.ps1 try { $svc = Get-Service -Name W32Time -ErrorAction Stop if ($svc.StartType -ne 'Automatic' -or $svc.Status -ne 'Running') { Write-Output "Issue: W32Time StartType=$($svc.StartType) Status=$($svc.Status)" exit 1 } Write-Output "OK: W32Time running" exit 0 } catch { Write-Output "Issue: $($_.Exception.Message)" exit 1 } The remediation script # Remediate-W32Time.ps1 try { Set-Service -Name W32Time -StartupType Automatic -ErrorAction Stop Start-Service -Name W32Time -ErrorAction Stop w32tm /resync /force | Out-Null Write-Output "Remediated: W32Time set to Automatic and started" exit 0 } catch { Write-Output "Remediation failed: $($_.Exception.Message)" exit 1 } Keep both scripts idempotent, so running the fix twice does no harm. Save them as UTF-8, and keep them small and dependency-free. There's no guarantee any module you rely on exists on the client. Creating the package In the Intune admin center, go to Devices, then Scripts and remediations, then Remediations, and create a script package. Upload both scripts and pick the settings. "Run this script using the logged-on credentials" should be No (so it runs as SYSTEM) for machine settings like services. Only use Yes for per-user fixes. Only turn on "Enforce script signature check" if you sign your scripts. Set "Run script in 64-bit PowerShell" to Yes for most things, so you see the real registry and System32. Assigning and scheduling Assign it to a pilot group first and choose a schedule of once, hourly or daily (with a time). Daily is a sensible default for drift. Hourly is for things that genuinely need it, since every run is another process on every device. You can also run it on a single device on demand from that device's page with the "Run remediation" action, which is handy for testing. Checking that it works Open the remediation and go to the device status view. Add the output columns (pre-remediation detection output, remediation output and post-remediation detection output) so you can see what each script reported. A working package shows devices moving from "With issues" to "Issues fixed." On the device itself, the Intune Management Extension logs under C:\ProgramData\Microsoft\IntuneManagementExtension\Logs\ show each run. Then break it on purpose. Stop the service on a test device, trigger the remediation on demand, and make sure the service comes back. The package that only ever detected We automate a lot of Intune through Microsoft Graph, including creating and assigning remediation packages across tenants. One package showed exactly the right detection results. Devices were listed "with issues," and the output clearly explained the problem. And yet nothing ever got fixed. Weeks later, the same devices were still on the list. The assignment had been created through Graph with the flag that tells Intune to run the remediation script set to true. When we read it back in the portal, the package was assigned in a detection-only state. In our experience that flag on the assignment can be silently ignored. The API call succeeds, no error comes back, and the package runs detection only. Here's what we do now. After any create or assign through Graph, open the package in the portal and read back the assignment. Make sure the remediation script is enabled to run, not just detection. If it isn't, fix it in the portal (or re-check your Graph payload against the current API documentation) and read it back again. Before calling the rollout done, wait until at least one device shows "Issues fixed" in the report. If all you ever see is "With issues," the remediation side isn't running. Other ways these go sideways If detection exits 0 when there's a problem, for example because an error got swallowed, the remediation never runs. Always exit 1 in the catch block. The post-remediation detection runs right away. If your fix only takes effect after a reboot or a service restart, detection reports "failed" even though it worked. Account for that in the logic or the output. Keep scripts and output short. Long output gets truncated in the report. Scripts set to run as the logged-on user fail when nobody is logged on, and they can't change machine settings. I've stopped trusting any automated assignment until I've seen "Issues fixed" in the report myself. It costs a day of waiting, which beats finding out weeks later.

Most real software gets into Intune as a Win32 app. That covers EXE installers, MSIs with transforms, and anything that needs a script to do some work before or after setup. Building the package is the easy part. What actually breaks deployments is detection rules, install context and return codes, and usually you find out when someone opens a ticket saying the app they were promised never showed up. This is the process we follow, including the testing step that saves us the most time. Tools to grab first The Microsoft Win32 Content Prep Tool (IntuneWinAppUtil.exe) from Microsoft's GitHub. The installer and its silent install and uninstall switches. Check the vendor's documentation, or run it with /?. A test VM or spare PC, ideally with a snapshot you can roll back. Sysinternals PsExec, for testing as SYSTEM. Start with a clean source folder Put only what the install needs into one folder, like the installer, any transforms or config files, and an optional wrapper script. Everything in that folder ends up in the package, so don't point the tool at your Downloads folder. C:\Packaging\AcmeApp\Source\ AcmeApp-5.2.msi settings.xml Install.ps1 C:\Packaging\AcmeApp\Output\ Test the silent install as SYSTEM This is the step I'd never skip. Intune installs Win32 apps in SYSTEM context by default, and an installer that works fine when you double-click it can behave completely differently as SYSTEM. There's no user profile, no mapped drives and no HKCU. So test it the way Intune will run it. # Opens a PowerShell window running as SYSTEM (run from an elevated prompt) psexec.exe -s -i powershell.exe # In the SYSTEM window, confirm who you are, then run your exact install line whoami msiexec /i "C:\Packaging\AcmeApp\Source\AcmeApp-5.2.msi" /qn /norestart /l*v C:\Windows\Temp\AcmeApp-install.log $LASTEXITCODE Write down the exit code, then test the uninstall line the same way. If either one needs anybody to click anything, it'll hang in Intune until the timeout. Build the .intunewin .\IntuneWinAppUtil.exe -c "C:\Packaging\AcmeApp\Source" -s "AcmeApp-5.2.msi" -o "C:\Packaging\AcmeApp\Output" -q -c is the source folder, -s is the setup file (relative to the source folder), -o is the output folder, and -q is quiet mode. When the setup file is an MSI, the tool reads its metadata and Intune can prefill the product code for detection. Creating the app in Intune In the Intune admin center, add a Windows app of type "Windows app (Win32)," upload the .intunewin, and fill in the commands and behavior. Install command: msiexec /i "AcmeApp-5.2.msi" /qn /norestart, or for a wrapper, powershell.exe -ExecutionPolicy Bypass -File .\Install.ps1. Uninstall command: for an MSI, msiexec /x {PRODUCT-CODE-GUID} /qn. Install behavior: System or User. Use System unless the app installs per user into the profile. Device restart behavior: usually "Determine behavior based on return codes." The default return codes are 0 (success), 1707 (success), 3010 (soft reboot), 1641 (hard reboot) and 1618 (retry, another install in progress). Add any vendor-specific success codes your installer uses. If you don't, a perfectly good install gets reported as failed, and somebody burns an afternoon chasing it. Requirement rules decide who gets it Requirements decide whether the app even applies to a device. You can check OS architecture, minimum Windows version, disk space and RAM, and optionally add a file, registry or script check. If a requirement isn't met, the device reports "Not applicable" instead of failing. We use a script or file requirement to limit apps to a subset of devices, for example only machines with a marker file dropped by a department provisioning step. Detection rules, where most of the trouble lives Detection is how Intune decides whether the app is installed, right after the install runs and on every check after that. You've got four options. The MSI product code is the most reliable for MSIs, and you can check the version too. A file or folder rule checks that something exists, or compares the version or date on a specific file, for example C:\Program Files\Acme\acme.exe with version greater than or equal to 5.2. A registry rule checks that a key or value exists or matches, like the app's entry under the Uninstall key. Pay attention to the "associated with a 32-bit app on 64-bit clients" option, which controls whether Intune looks in the WOW6432Node view. A custom PowerShell script decides for itself. I reach for it when the other options can't express the condition. # Detection script: detected = exit 0 AND something written to STDOUT $exe = "C:\Program Files\Acme\acme.exe" if (Test-Path $exe) { $ver = [version](Get-Item $exe).VersionInfo.FileVersion if ($ver -ge [version]"5.2.0") { Write-Output "AcmeApp $ver detected" exit 0 } } exit 1 Pilot it Assign it as Required to a small device group, or as Available to a user group for Company Portal. Sync the test device and watch it. How we confirm it landed The app's device install status in Intune shows Installed, not "Failed" or "Not detected." The Intune Management Extension logs on the client tell you what actually ran. They live in C:\ProgramData\Microsoft\IntuneManagementExtension\Logs\, and the ones I open are IntuneManagementExtension.log and AppWorkload.log. CMTrace or any log viewer that follows the file helps. If you included /l*v in the install line, the installer log has the rest. Where packages have gone wrong for us Script detection has to write to STDOUT and exit 0. A script that exits 0 but writes nothing counts as not detected. Write the result with Write-Output, and exit non-zero when the app is missing. This one is easy to forget. "Installed successfully but not detected" means the install worked and the detection rule is wrong. Usually it's the wrong registry view (32-bit vs 64-bit), a version string that doesn't compare the way you expect, or a path that differs on some machines. Depending on how the command is launched, a wrapper can end up running in 32-bit PowerShell, where System32 and the registry are redirected. If your script needs the 64-bit view, launch it explicitly with %windir%\Sysnative\WindowsPowerShell\v1.0\powershell.exe and test both paths. Interactive installers hang until the install timeout. Test in SYSTEM context and make sure nothing prompts. When a new version comes out, use supersedence instead of deploying two versions side by side with overlapping detection rules. When a deployment fails, I open AppWorkload.log before I change anything in the portal. It usually shows exactly what ran and what exit code came back.

If your domain-joined PCs are already hybrid-joined to Entra ID, one Group Policy setting will get them into Intune. The PC enrolls itself in the background with the signed-in user's Entra credentials, and nobody has to click anything. That's the theory. In practice we spent a long time on a handful of machines that refused to enroll, and every one of them was used by someone in IT. It turned out to be a Conditional Access policy. Below is the setup we use, how to check it, and how we tracked that problem down. Confirm these before you start Entra Connect (or Cloud Sync) configured for hybrid Entra join, with the computer OUs in sync scope. Users licensed for Intune, directly or through a suite such as Microsoft 365 Business Premium or E3/E5. The MDM user scope set in Entra ID so those users are allowed to enroll. Windows 10/11 Pro, Enterprise or Education. Group Policy administrative templates recent enough to include the MDM enrollment setting. Make sure hybrid join actually works Auto-enrollment rides on hybrid join. If the device isn't hybrid-joined, nothing else here matters. On a test PC, signed in as a normal domain user, run this. dsregcmd /status Under Device State you want AzureAdJoined : YES and DomainJoined : YES. Under SSO State you want AzureAdPrt : YES. If AzureAdJoined says NO, check that the computer object is in a synced OU, that the service connection point (SCP) is configured, and that the device can reach the Microsoft device registration endpoints. Also look the device up in Entra ID. It should show as "Microsoft Entra hybrid joined" with a registered date, not "Pending." MDM user scope and licensing In the Entra or Intune admin center, find the mobility (MDM and WIP) settings for Microsoft Intune and set the MDM user scope to All, or to Some with a group containing the people who sign in to these PCs. Leave the MAM scope alone unless you know you need it. Confirm those users have an Intune license and that your Windows enrollment restrictions allow Windows (MDM) enrollment. The GPO itself Create a GPO linked to the OU that holds the computers. Start with a test OU. Then set this. Computer Configuration > Policies > Administrative Templates > Windows Components > MDM > Enable automatic MDM enrollment using default Azure AD credentials = Enabled Select Credential Type to Use: User Credential "User Credential" is the standard choice here. "Device Credential" exists for specific co-management and ConfigMgr scenarios, so don't pick it just because it sounds more convenient. When the policy applies, Windows creates a scheduled task under \Microsoft\Windows\EnterpriseMgmt that tries to enroll on a recurring schedule. The next time a licensed, in-scope user signs in, the device enrolls. Kick it and watch gpupdate /force # Confirm the policy landed gpresult /r /scope computer # Confirm the enrollment task exists Get-ScheduledTask -TaskPath "\Microsoft\Windows\EnterpriseMgmt\*" | Select-Object TaskName, State Sign out and back in as a normal user, give it a few minutes, and then check. Signs it enrolled In dsregcmd /status, the Tenant Details section should have MdmUrl filled in. Once enrolled, the device shows up in the Intune device list as co-managed or Intune-managed. In Event Viewer, open Applications and Services Logs > Microsoft > Windows > DeviceManagement-Enterprise-Diagnostics-Provider > Admin. Event ID 75 means auto-enrollment succeeded. Event ID 76 means it failed, and the event includes an error code. In Settings, Accounts > Access work or school shows the work account with an "Info" button that lists the MDM sync status. If you'd rather pull those events with PowerShell: Get-WinEvent -LogName "Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin" -MaxEvents 50 | Where-Object { $_.Id -in 75,76 } | Format-Table TimeCreated, Id, Message -Wrap The machines that never enrolled We proved this one the hard way. Most of the fleet enrolled fine, but a set of machines kept logging event 76. Every one was used by an IT staffer signing in with an account that held an admin role. Regular users on identical hardware, in the same OU with the same GPO, enrolled without any trouble. Background auto-enrollment requests its token non-interactively. There's no prompt, so there's no way to answer an MFA challenge. Our tenant had a Conditional Access policy requiring MFA for users in admin roles, so for those accounts CA blocked the token request, and the enrollment task just failed and retried forever. It took a while to find because the default sign-in logs view in Entra shows interactive sign-ins, and those looked clean. Switch to non-interactive user sign-ins, filter on the affected user, and look for failures from the application Device Management Client. The Conditional Access tab on that sign-in tells you which policy blocked it. What we did about it Here are the options in the order I'd pick them. Stop signing in to workstations with admin accounts. Give IT staff a normal daily account for their PC and keep privileged roles on separate accounts. The device then enrolls under the normal user, and your admin accounts keep their MFA requirement. This is the one I push for. If you really have to, make a narrow exclusion. Adjust the CA policy so only the enrollment resource shown in the failed sign-in is excluded, and nothing broader. Don't exclude the admins from MFA entirely. Review the change with whoever owns your CA policies and test it with the What If tool. Enroll the affected devices once with a non-admin user, then change the primary user later if you need to. If you manage several tenants, check each one for a similar admin-MFA policy before you roll this out. I'd assume it's there until you've looked. A few smaller ones Users with a non-routable UPN like @contoso.local can't authenticate to Entra properly. The UPN has to match a verified domain. A device that's already MDM-enrolled somewhere else (another tenant or a previous MDM) will fail until you clean that up. A device reimaged with the same name can have a stale Entra object. Delete the stale object and let it re-register. Sync cycles, the scheduled task interval and policy refresh all add delay. On a big rollout, give it a day before you start chasing ghosts.

A lot of the small offices we pick up have a firewall with a pile of port forwards on it. 8443 goes to one server, 8080 to another, 9000 to a third, and nobody remembers which is which. Each of those services faces the internet with its own TLS settings, or none. A reverse proxy cleans that up. You get one public IP, only ports 80 and 443 open, routing by hostname, and certificates managed in one place. We use Nginx Proxy Manager (NPM) for a lot of this because it puts a usable GUI on top of plain nginx, which helps when the next tech to touch it isn't an nginx person. Below is how we set it up, plus the client IP and redirect loop problems that catch everyone. What needs to be in place A small Linux VM or container host with Docker, on a network segment that can reach your internal apps. One public IP with ports 80 and 443 forwarded (or 1:1 NATed) to the proxy, and nothing else. DNS records for each hostname (for example app.example.com) pointing at the public IP. For DNS-01 certificates, an API token from your DNS provider, scoped to the zones you need. Getting NPM running services: npm: image: jc21/nginx-proxy-manager:latest restart: unless-stopped ports: - "80:80" - "443:443" - "81:81" # admin UI - do NOT expose this publicly volumes: - ./data:/data - ./letsencrypt:/etc/letsencrypt Once it's in production, pin a specific image version so an update doesn't surprise you. Port 81 is the admin UI, so restrict it with the host firewall to your management network or VPN. Log in, change the default credentials right away, and back up the data and letsencrypt folders. One proxy host per app For each app, create a proxy host with the public hostname and the internal scheme, IP and port (for example http://10.10.50.20:8080), and turn on "Block common exploits." Turn on WebSockets support for anything with live updates, like dashboards, remote consoles or chat. NPM routes on the Host header, so dozens of apps can share the same IP and port 443. Lock the backends down as well. The app servers should only accept connections from the proxy's IP. Otherwise anyone on the internal network can walk right around whatever the proxy enforces. Certificates, HTTP-01 or DNS-01 With HTTP-01, Let's Encrypt connects to http://hostname/.well-known/acme-challenge/. Port 80 has to be reachable from the internet and the DNS record has to point at the proxy. It's simple, but you get no wildcards, and it won't work for hostnames that are internal-only. With DNS-01, NPM creates a TXT record through your DNS provider's API. That works for wildcards (*.example.com), for internal-only hosts, and when port 80 is blocked. The cost is an API token sitting on the proxy, so scope it to DNS edit on the specific zone. Then turn on "Force SSL," and once you're sure everything works over HTTPS, turn on HSTS too. Getting the real client IP to the backend Skip this and every request in the app's logs comes from the proxy's IP. That wrecks your auditing and rate limiting, and tools like fail2ban will end up banning the proxy, which means banning everyone behind it. NPM sends X-Forwarded-For, X-Real-IP and X-Forwarded-Proto by default. The backend has to be told to trust those headers only from the proxy. If the backend runs nginx, it looks like this. # On the backend server, in the http or server block set_real_ip_from 10.10.50.5; # the NPM host's IP real_ip_header X-Forwarded-For; real_ip_recursive on; If something else sits in front of NPM, such as a CDN or cloud proxy, NPM also needs set_real_ip_from entries for that provider's published ranges and the provider's client IP header. Add them in the proxy host's Advanced tab or a custom include. Never trust X-Forwarded-For from just anywhere, because clients can set it to whatever they like. Apps that aren't behind nginx have their own setting, usually called "trusted proxies" (Laravel, Nextcloud, and many Node and Python frameworks). Set it to the proxy IP, not to "everything." The "too many redirects" loop The browser talks HTTPS to NPM, and NPM usually talks plain HTTP to the backend. An app that enforces HTTPS sees a plain HTTP request and redirects to HTTPS. The browser comes back through the proxy, the app sees HTTP again, and around it goes until the browser gives up with "too many redirects." You fix it by making the app honor X-Forwarded-Proto: https from the trusted proxy. A few examples: In Nextcloud, set overwriteprotocol to https and fill in trusted_proxies in config.php. In Django, set SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https"). In Laravel, configure the TrustProxies middleware with the proxy IP. For anything else, set the app's "base URL" or "external URL" to https://app.example.com. Testing it from outside # Certificate and redirect chain from outside your network curl -sI http://app.example.com | head -n 5 curl -svI https://app.example.com 2>&1 | grep -Ei "subject|expire|HTTP/" # Confirm only 80/443 are open on the public IP (run from outside) nmap -Pn -p 1-10000 192.0.2.10 Run these from outside your network. Then load the app, sign in, and check that its access log shows your real public IP and not the proxy's. Where this has bitten us A CDN in "flexible" mode (HTTPS to the browser, HTTP to the origin) in front of an NPM host set to Force SSL gives you a redirect loop before the app is even involved. Use full or strict mode at the CDN. If the backend is HTTPS with a self-signed cert, set the scheme to https in NPM. Some apps also insist on a matching Host header or reject the proxy. When uploads fail, it's usually nginx's default body size limit, which is small. Add client_max_body_size in the Advanced tab for file-heavy apps. Repeatedly failing HTTP-01 validations while you're debugging DNS can hit Let's Encrypt rate limits. Fix DNS first, then request the cert. Putting an admin login page behind a proxy doesn't make it safe. Add NPM access lists, an SSO layer or VPN-only access for admin interfaces. Once it's set up right, the proxy is about the most boring box we run, and I mean that as a compliment.

This one has cost us more hours than I'd like to admit. A remote site or a patrol car connects back to the main network through a cellular router. The tunnel is up and you can ping the domain controllers by IP, but names won't resolve. The person in the car gets slow or failed logons, Group Policy doesn't apply, and mapped drives break. Often the cause is the router's own DNS forwarder answering for your internal domain before the question ever reaches your DCs. Here's how we diagnose it and the fixes that have worked for us. Gather this first Admin access to the cellular or edge router's LAN and DNS settings, either locally or through its cloud management portal. The internal AD DNS domain (we'll use contoso.local) and the DC IPs (for example 10.10.1.10 and 10.10.1.11). The remote site's subnet (for example 10.10.40.0/24). Domain admin rights for AD Sites and Services and DNS changes. Why the router gets in the way Lots of small routers, including some cellular ones, run a dnsmasq-style DNS forwarder for their LAN. That forwarder has a "local domain" configured for its own DHCP clients, and it's authoritative for that domain. Any name in that domain it doesn't know gets an immediate NXDOMAIN instead of being forwarded upstream. Some devices also treat .local specially. If the router's local domain is set to your AD domain, or covers it, every lookup for dc01.contoso.local or the _ldap._tcp.dc._msdcs.contoso.local SRV records gets "does not exist" straight from the router. The pattern that gives it away is that public names resolve, internal hostnames fail instantly (not a timeout), and querying the DC directly works. Prove it from a client From a machine at the remote site, compare what the router says with what the DC says. # Which DNS servers did DHCP hand out? Get-DnsClientServerAddress -AddressFamily IPv4 # Ask the default resolver (probably the router) Resolve-DnsName dc01.contoso.local # Ask the DC directly across the tunnel Resolve-DnsName dc01.contoso.local -Server 10.10.1.10 # The record domain members actually need to find a DC Resolve-DnsName _ldap._tcp.dc._msdcs.contoso.local -Type SRV -Server 10.10.1.10 If the first query fails with "DNS name does not exist" and the direct query to the DC succeeds, the local forwarder is your problem. If the direct query fails too, stop and fix the tunnel, routing or firewall first (UDP and TCP 53 to the DCs). Fixing the router There are three ways to go. I've listed them in the order I'd try them. Change the router's local domain Set the router's LAN local domain to something that can't collide with AD, such as lan. The router stops claiming authority over contoso.local and forwards those queries upstream. This is what fixed our case. It was a single setting, pushed to every router in the group. Forward the internal zone to the DCs If the router supports per-domain forwarding (split DNS), send the AD zone and the relevant reverse zones to the DCs and leave everything else on the normal upstream. In raw dnsmasq terms it looks like this. # Forward the AD zone and its reverse zone to the domain controllers server=/contoso.local/10.10.1.10 server=/contoso.local/10.10.1.11 server=/40.10.10.in-addr.arpa/10.10.1.10 # Do not answer authoritatively for the AD domain # (make sure no local=/contoso.local/ or domain=contoso.local line exists) Vendor GUIs usually call this something like "DNS forwarding rules," "split DNS" or "custom DNS servers per domain." Hand out the DCs as DNS servers You can also have the router's DHCP give clients the DCs as their DNS servers. It's the cleanest option for domain members, but name resolution then depends entirely on the tunnel. If the cellular link or VPN drops, clients lose all DNS, public names included. For vehicles and kiosks that still need the internet when the tunnel is down, I'd stick with one of the first two options. Tell AD about the remote subnet Even with DNS fixed, domain members on a subnet AD doesn't know about can't work out which site they're in. They may pick a DC across the country, and logons and Group Policy get slower. Add the subnet and tie it to a site. New-ADReplicationSite -Name "Branch-North" New-ADReplicationSubnet -Name "10.10.40.0/24" -Site "Branch-North" If the remote site has no DC of its own, associating the subnet with the site that has the nearest DCs is fine. Add the reverse zone while you're in there A missing reverse zone won't break logons, but it makes troubleshooting misleading. nslookup complains about the server name, logs show IPs instead of names, and some applications care about it. Create one per remote subnet. Add-DnsServerPrimaryZone -NetworkId "10.10.40.0/24" -ReplicationScope "Forest" Then make sure DHCP or the clients register PTR records. Is it actually fixed? Run ipconfig /flushdns first so you aren't looking at a cached NXDOMAIN, then run these. Resolve-DnsName dc01.contoso.local Resolve-DnsName _ldap._tcp.dc._msdcs.contoso.local -Type SRV nltest /dsgetdc:contoso.local nltest /dsgetsite gpupdate /force nltest /dsgetsite should return the site you created, and /dsgetdc should find a DC without a long delay. Things that tripped us up Windows caches negative answers. After we fixed the router, clients kept failing for a while because they'd cached the NXDOMAIN. Flush DNS on the client before you decide the fix didn't work. If the router is managed from a cloud portal, a local change can be overwritten at the next sync. Make the change in the group or template, not on one box. We've seen the same problem at more than one site. If one fleet of routers had the setting wrong, assume every fleet configured from the same template does too, and check them all. .local is reserved for multicast DNS, which is one reason some devices handle it specially. It isn't worth renaming a domain over, but keep it in mind when a Mac or a consumer device acts strangely. If the remote subnet overlaps something else you route, DNS will look broken when it's really routing. Run the direct DC query first. These days, when internal names fail instantly but the DC answers fine when asked directly, the router's local domain is the first thing I look at.

Site-to-site IPsec is easy when you control both ends and they're the same vendor. It's a lot less fun when the other end belongs to a different team, runs a different vendor's gear from a different decade, and you're trading screenshots over email. Most of the failures we see aren't exotic. They come down to overlapping subnets, mismatched proposals, a missing NAT exemption or an MTU problem. Here's the planning sheet we send out, and the order I troubleshoot in when the tunnel shows "up" and nothing passes. Before anyone touches config Admin access to both firewalls or routers, or a contact on the other side who can read their config to you. Public IPs for both peers (for example 192.0.2.10 and 198.51.100.20). The local and remote subnets that need to talk. A test host on each side that will answer a ping. Agree on the worksheet first Before anybody configures anything, we send the remote admin a short worksheet and both sides fill in the same values. It feels like paperwork, but it saves a lot of back-and-forth calls later. Peer public IP (A / B): 192.0.2.10 / 198.51.100.20 IKE version: IKEv2 Authentication: PSK (exchanged out of band) or certificate Phase 1 encryption/integrity: AES-256 / SHA-256 Phase 1 DH group: 14 (or 19/20) Phase 1 lifetime: 28800 s Phase 2 protocol: ESP Phase 2 encryption/integrity: AES-256 / SHA-256 (or AES-GCM) PFS group: 14 (or none, but both must agree) Phase 2 lifetime: 3600 s Local subnets (A): 10.10.20.0/24 Remote subnets (B): 10.40.0.0/24 NAT-T: enabled (UDP 4500) DPD / keepalive: enabled Subnets can't overlap If both offices use 192.168.1.0/24, routing has no way to tell which side a packet belongs to. You can re-address one side, which is the best long-term answer, or NAT inside the tunnel, which works but makes every future rule harder. Also check for overlap with everything else you route, like other tunnels, SD-WAN and remote-access VPN pools. Overlapping a remote-access or mesh VPN range is a classic way to break a site for just one group of users, and those tickets are confusing because it works fine for everyone else. Proposals have to match exactly Phase 1 (the IKE SA) and phase 2 (the IPsec or child SA) are negotiated separately. A phase 1 mismatch means nothing comes up. A phase 2 mismatch means phase 1 is up and phase 2 never establishes, and some GUIs still show that as a green tunnel. Stay away from legacy options (DES, 3DES, MD5, DH groups 1, 2 and 5) unless the other side truly can't do better, and if you have to use them, write it down. PSK or certificates A long random pre-shared key is fine for a handful of tunnels. Send it over a different channel than the rest of the config. Certificates scale better, avoid shared secrets, and are the right answer when one peer has a dynamic IP. The catch is that you now own a CA and certificate expiry, so put the expiry date on your calendar. NAT-T If either peer sits behind NAT, like an ISP modem or a cloud NAT gateway, IKE moves from UDP 500 to UDP 4500 and ESP gets encapsulated in UDP 4500. Allow UDP 500, UDP 4500 and IP protocol 50 (ESP) from the peer's address. Behind NAT, the peer's IKE identity is often its private IP, so you may need to set the local and remote ID explicitly. Bring it up with real traffic Policy-based tunnels often won't negotiate phase 2 until interesting traffic shows up. Start a continuous ping from a host on one side to a host on the other and watch the logs while it runs. Ping a host, not the remote firewall's inside interface, unless you know that firewall answers pings sourced from the tunnel. Up but no traffic, in the order I check it Each of these rules out a whole layer, so I go in order even when I think I already know the answer. Start with the SA counters. Look at the phase 2 SA and its encrypt and decrypt (encaps/decaps) counters. On Cisco IOS that's show crypto ipsec sa, and most firewalls have an IPsec monitor page. If encrypts are climbing and decrypts sit at zero, your packets are leaving and nothing is coming back, so the problem is on the far side. If neither is moving, traffic isn't reaching the tunnel on your side. Check the proxy IDs, also called traffic selectors. For policy-based VPNs, the local and remote subnets have to be mirror images on both ends. If side A says 10.10.20.0/24 to 10.40.0.0/24, side B must say 10.40.0.0/24 to 10.10.20.0/24. A /16 on one side and a /24 on the other is a mismatch for many vendors. When a route-based device (VTI) talks to a policy-based one, set explicit proxy IDs on the route-based side instead of leaving 0.0.0.0/0. Look at routing. On route-based tunnels you need a route for the remote subnet pointing at the tunnel interface. Then check the return path. The remote host's default gateway has to be the device that terminates the tunnel. If the remote server's gateway is a different router, its replies head out to the internet instead. Confirm the NAT exemption. If outbound NAT translates your source before IPsec sees it, the traffic no longer matches the selectors. Add a no-NAT or identity NAT rule for local-to-remote subnets above your general outbound NAT rule. Check firewall policy. A tunnel being up doesn't mean traffic is allowed. Most firewalls need explicit rules for traffic entering from the VPN zone or interface, in both directions. Check host firewalls too, because Windows will happily block ICMP from a "foreign" subnet. Last, suspect MTU and MSS. If ping works but SMB, RDP or web pages hang, it's probably fragmentation. IPsec overhead, plus NAT-T and PPPoE, shrinks the usable MTU. Clamp TCP MSS on the tunnel. Around 1360 is a safe place to start. ! Cisco IOS on a tunnel interface (route-based VTI) interface Tunnel10 ip mtu 1400 ip tcp adjust-mss 1360 To test the path MTU, ping with the don't-fragment bit set and walk the size down until it passes. # Windows: 1372 bytes of payload + 28 bytes of headers = 1400 ping 10.40.0.10 -f -l 1372 # Linux ping -M do -s 1372 10.40.0.10 What working looks like Phase 1 and phase 2 SAs are both established, and the encrypt and decrypt counters are climbing on both ends. Ping works in both directions, started from each side. A real application test holds up, like an SMB file copy or an RDP session that stays connected for several minutes. That catches MTU problems that ping misses. The tunnel survives a rekey. Wait out the phase 2 lifetime or force a rekey, and confirm traffic keeps flowing. I don't call a tunnel done until the rekey test passes. Otherwise it tends to come back as a ticket the next morning. Failures we keep running into It works for an hour, then dies. That's a lifetime or PFS mismatch. The first negotiation succeeds and the rekey fails, so match lifetimes and PFS exactly. Only one side can start the tunnel. One peer is set to respond-only or has a dynamic IP. That's fine if it's intended, but write it down so the next person doesn't chase it. Only the first subnet works when there are several behind one side. Some vendors build one child SA per subnet pair and others want them combined, so look at how each side is building SAs. One tunnel worked for everyone except staff connected from home. Their remote-access VPN pool used the same range as the remote office. The counters tell you which side to look at, and that usually settles the "your end or mine" conversation before it gets awkward.

Half the networks we inherit have VLANs that someone started and never finished. Usually I find out because the office manager calls to say the desk phones are dead again, or because I'm standing in a closet with a console cable after a trunk change cut me off. So this is how we plan VLANs now, what trunks and access ports actually do, and the config we use on Cisco IOS and on HP/Aruba ProCurve-style switches. Have these ready first A managed switch you can reach by console or SSH. Bring the console cable anyway, because you will lock yourself out at least once. The list of networks you need, like data, voice, guest, cameras, printers and management. A gateway on the router or firewall (SVI or subinterface) for each VLAN. Somewhere to write it all down, whether that's a spreadsheet, NetBox or your wiki. Write the table before you touch a switch Every VLAN gets an ID, a name, a subnet and a purpose. We keep the same scheme at every site we manage, so whoever walks into a building next already knows VLAN 20 is voice. Often that person is me, six months later. A simple plan looks like this. VLAN Name Subnet Gateway Purpose 10 DATA 10.10.10.0/24 10.10.10.1 Staff PCs 20 VOICE 10.10.20.0/24 10.10.20.1 Desk phones 30 GUEST 10.10.30.0/24 10.10.30.1 Guest Wi-Fi (internet only) 99 MGMT 10.10.99.0/24 10.10.99.1 Switch/AP/UPS management 999 BLACKHOLE (none) (none) Unused native VLAN on trunks Two rules I don't bend on. Management gets its own VLAN, never VLAN 1 and never the data VLAN. And VLAN 1 carries nothing. Using an unused VLAN as the native VLAN on trunks closes off VLAN-hopping tricks and keeps untagged junk from landing somewhere useful. Access ports, trunks and the native VLAN An access port belongs to one VLAN and sends frames untagged. That's what a PC or printer plugs into. A trunk carries many VLANs, with each frame tagged using 802.1Q. You use trunks switch to switch, switch to firewall, and usually switch to AP. The native VLAN is the one VLAN on a trunk that goes out untagged. Both ends have to agree on it. If they don't, traffic leaks between VLANs, and on Cisco you'll see native VLAN mismatch messages from CDP. An access port can also carry a tagged voice VLAN, so the phone and the PC plugged into the back of it share one cable. The phone learns which VLAN to use through LLDP-MED, CDP or DHCP options. Cisco calls these "access" and "trunk." HP/Aruba ProCurve doesn't use those words. Each port is untagged in one VLAN and tagged in any others. An HP port untagged in VLAN 10 and tagged in VLAN 20 is the same idea as a Cisco access port with a voice VLAN. The Cisco IOS side vlan 10 name DATA vlan 20 name VOICE vlan 99 name MGMT vlan 999 name BLACKHOLE ! interface range GigabitEthernet1/0/1 - 46 switchport mode access switchport access vlan 10 switchport voice vlan 20 spanning-tree portfast ! interface GigabitEthernet1/0/48 description UPLINK-TO-CORE switchport mode trunk switchport trunk native vlan 999 switchport trunk allowed vlan 10,20,99 ! interface Vlan99 ip address 10.10.99.11 255.255.255.0 no shutdown ! ip default-gateway 10.10.99.1 On some older or multilayer platforms, switchport mode trunk won't be accepted until you add switchport trunk encapsulation dot1q first. Prune your trunks with allowed vlan instead of letting every VLAN go everywhere. This next part is where I see people trip most. When you add a VLAN to an existing trunk later, use switchport trunk allowed vlan add 30. Leave off the word add and you replace the whole list, which drops everything else on that trunk, including the management session you're typing into. Same thing on HP/Aruba vlan 10 name "DATA" untagged 1-46 tagged 48 exit vlan 20 name "VOICE" tagged 1-46,48 voice exit vlan 99 name "MGMT" tagged 48 ip address 10.10.99.12 255.255.255.0 exit ip default-gateway 10.10.99.1 Making a port untagged in VLAN 10 pulls it out of VLAN 1 automatically. The voice flag marks VLAN 20 so LLDP-MED advertises it to phones. On the uplink (port 48), the native VLAN is just whichever VLAN the port is untagged in. If the port is tagged-only for everything, there's no native VLAN at all. I prefer that when the other end allows it. Move management last Build the new management SVI and make sure you can reach it over the trunk before you remove the old management IP. On Cisco, run reload in 10 before anything risky. If you cut yourself off, the switch rolls back on its own and you're not driving out to the site. Once you're safely back in, cancel it with reload cancel. Then save with write memory, which works on both platforms. After that, write it down. Record the VLAN table, which ports are trunks and what's on the other end, the native VLAN on each trunk, and each switch's management IP. Port descriptions on uplinks cost nothing, and the next tech who inherits the site will save hours because of them. Checking your work ! Cisco show vlan brief show interfaces trunk show interfaces GigabitEthernet1/0/5 switchport show cdp neighbors detail ! HP/Aruba show vlans show vlans ports 48 detail show lldp info remote-device Each trunk should list the VLANs you expect as allowed and forwarding. Plug in a test PC and make sure it gets a DHCP address in the right subnet, then do the same with a phone and check that it pulls an address from the voice scope. The time the phones went dead We put a new switch in at an outlying building for one of our municipal clients. PCs were fine, but every desk phone sat at "network unavailable." The access ports had the voice VLAN set correctly, and the phones were even learning the VLAN ID over LLDP-MED. The problem was upstream. The voice VLAN had never been created on the new switch's uplink path, so the uplink wasn't carrying it tagged, and the phones were tagging their traffic into a VLAN that went nowhere. We fixed it by creating the VLAN, tagging it on the uplink, and confirming the upstream switch was tagging it toward us too. Now when phones fail and PCs work, I check the trunk allowed list on every hop between the phone and the call server before I look at anything else. Other things that have bitten us A native VLAN mismatch between two vendors quietly bridges two VLANs together. Make both ends match, or tag everything. Forgetting add on Cisco and overwriting the allowed list. For remote work, set reload in first. The VLAN exists but clients get no DHCP. Usually the firewall or router has no interface or DHCP relay for the new VLAN yet. APs on trunks. If the AP's management is on the native VLAN and you change the native VLAN, the AP drops off the controller. When something on one VLAN breaks and everything else works, I walk the trunks hop by hop. It's tedious, but it finds the problem faster than guessing.