Respond: The First Four Hours
az managedservices assignment list per subscription, or Azure portal › Service providers), its RMM agent on your hosts, and apps it authorised in Salesforce, Google Workspace, Slack or GitHub, which Entra never lists. Package: lockfiles across all repositories, the uses: lines of every workflow, CI build logs and caches, container images and their base-image digests, OS packages on hosts and images (xz-utils was a distro package, not a lockfile entry), developer machines, production hosts. Why: you can only cut and rotate what you have listed. Lockfiles show what should be installed; CI logs and SBOMs of built images show what actually was, and when; a pinned action's own dependencies are not pinned by your SHA.# lockfiles and workflows in a checkout (repeat per repo, or use code search across the org)
grep -rn --include=package-lock.json --include=yarn.lock --include=pnpm-lock.yaml '<package>' .
grep -rn 'uses:' .github/workflows/ # every action, with its tag or SHA
npm ls <package> --all
pip show <package>
# OS packages and built images
dpkg -l '<package>*'; rpm -q <package>
syft <image> | grep -i <package>
docker image inspect <image> --format '{{index .RepoDigests 0}}'
# GitHub: audit log (Enterprise Cloud API) and the run logs of every job that installed the package
gh api --paginate '/orgs/<org>/audit-log?phrase=created:>=<window start>' > audit-log.json
gh run list -R <org>/<repo> --created '>=<window start>' --json databaseId,name,createdAt > runs.json
gh api /repos/<org>/<repo>/actions/runs/<run id>/logs > run-<run id>.zip
# npm: publish history, who published, integrity and provenance of the bad version; then the cached tarball
npm view <package> time --json > time.json
npm view <package>@<bad version> _npmUser dist.integrity dist.attestations --json > version.json
grep -rl '<package>' ~/.npm/_cacache/index-v5 | xargs -I{} cp --parents {} ./evidence/
tar -czf npm-cache-<host>.tgz ~/.npm/_cacache # or the registry proxy's storage for that package
npm audit signatures # registry signatures and provenance of what is installed now
gh attestation verify <artifact> --owner <org> # for artifacts with GitHub provenance
# AWS: event history (90 d) and the trail
aws cloudtrail lookup-events --start-time <window start> --lookup-attributes AttributeKey=EventName,AttributeValue=AssumeRoleWithWebIdentity > sts.json
# Entra and M365: see the Export act on the Defender tab. Hash everything.
sha256sum ./evidence/* audit-log.json runs.json *.zip *.json *.tgz > manifest.sha256
# GitHub audit log: what a stolen token did (fine-grained PATs log request_created/access_granted; classic PAT creation is not logged, only its use)
gh api --paginate '/orgs/<org>/audit-log?phrase=created:>=<window start>' | jq -r '.[] | select(.action | test("^(repo\\.create|repo\\.access|workflows\\.created_workflow_run|git\\.push|personal_access_token\\.|org\\.(create|update)_actions_secret|repo\\.(create|update)_actions_secret|environment\\.(create|update)_actions_secret|hook\\.create|integration_installation\\.create|public_key\\.create|protected_branch\\.destroy|org\\.add_outside_collaborator|repo\\.add_member)")) | [.created_at, .action, .actor, .repo // .org, .user_agent] | @tsv'
# Workflows and branches the worm added; private repositories flipped public (repo.access) and "-migration" copies
gh search repos --owner <org> --created '>=<window start>' --json name,visibility,description,createdAt
# Registry: did our packages get republished, and by whom
npm view <our package> time --json | jq 'to_entries | map(select(.value >= "<window start>"))'
npm owner ls <our package>; npm dist-tag ls <our package>
npm view <our package>@<version> _npmUser dist.attestations
Update-MgServicePrincipal flips none of them — end the GDAP or DAP relationship (admin center › Partner relationships › Remove roles), delete every Lighthouse assignment, block the partner tenant inbound in cross-tenant access settings (guests and direct connect only; it does not govern GDAP), disable the RMM agent service fleet-wide and revoke the RMM's API keys. SaaS-to-SaaS: revoke the vendor's connected app in each platform it was authorised in; the Entra cut does not reach a Salesforce or Workspace grant. Package: block the bad versions in your registry proxy and pause the CI workflows that pull it; network-contain the hosts and runners where it ran — they get rebuilt from clean later, not cleaned in place (see Rebuild from clean below). Trojanised update: block the vendor's update host until it confirms a clean release, and block the stolen certificate, not only the hashes. Credentials, in the same pass. Vendor: API keys, OAuth tokens and app consents, service accounts and VPN accounts it holds; reset any password shared with it. A multi-tenant vendor app's secret lives on the app object in the vendor's tenant: you cannot rotate it, so send the vendor the key id from the stage 3 hunt and get written confirmation. Package: everything readable from where it ran, listed below. Revoke first, then reissue. Pins. Pin to the last known-good version only after checking it: its hash or signature matches the registry's record, it was published before the compromise window, and the advisory doesn't list it. For a GitHub Action, pin to a full commit SHA and verify that SHA against the upstream commit history, not against the tag: in the tj-actions/changed-files incident (March 2025) every version tag was moved to the malicious commit. A SHA pin covers that action only; the actions it calls internally are pinned however its author pinned them. Why: a path left open while another is cut is a warning, not a containment; most malicious packages are credential stealers, and self-spreading ones use stolen publish tokens to trojanise the victim's own packages next. The next build reinstalls whatever the pin points at, and tags can be moved by whoever controls the repository.# Package containment: no lifecycle scripts, nothing published after the window, a cooling-off period, an allowlist
npm ci --ignore-scripts # or ignore-scripts=true in .npmrc for CI
npm install --before=<date before the compromise> # resolves only versions published before that date
pnpm config set minimumReleaseAge 10080 # minutes: no version younger than 7 days (pnpm 10.16+)
# registry proxy (Artifactory, Nexus, Verdaccio): block <package>@<bad versions>, or allowlist what CI may fetch
# Rotation scope for every host and runner where it ran, in this order
# 1. cloud: the IMDS role the runner could reach, then every long-lived key on it
aws iam put-role-policy --role-name <runner role> --policy-name AWSRevokeOlderSessions --policy-document \
'{"Version":"2012-10-17","Statement":[{"Effect":"Deny","Action":"*","Resource":"*","Condition":{"DateLessThan":{"aws:TokenIssueTime":"<now, ISO 8601>"}}}]}'
aws iam update-access-key --user-name <user> --access-key-id <key> --status Inactive # then delete after reissue
# review every OIDC trust policy: sub/aud conditions that let a pull_request or fork token assume a deploy role
aws iam get-role --role-name <runner role> --query Role.AssumeRolePolicyDocument
# 2. platform tokens the payload could read: ~/.npmrc, ~/.pypirc, ~/.docker/config.json, ~/.kube/config,
# ~/.aws/credentials and ~/.aws/sso/cache, ~/.azure/msal_token_cache.json, ~/.config/gcloud/, ~/.config/gh/hosts.yml,
# mounted Kubernetes service-account tokens, Vault tokens
npm token revoke <id>; npm profile enable-2fa auth-and-writes # npm: revoke, then 2FA for publish; move CI to trusted publishing (OIDC)
vault token revoke -accessor <accessor>; vault lease revoke -prefix <mount>/
kubectl -n <ns> delete secret <sa token secret>; kubectl -n <ns> rollout restart deployment/<runner>
# 3. GitHub: org, repository, environment and Dependabot secrets; deploy keys; the App's private key and installation tokens;
# runner registration tokens; self-hosted runner state; and the Actions cache, which is persistence if it was poisoned
gh secret list --org <org>; gh secret list -R <org>/<repo>; gh secret list -R <org>/<repo> -e <env>; gh secret list -R <org>/<repo> --app dependabot
gh api /repos/<org>/<repo>/keys; gh api -X DELETE /repos/<org>/<repo>/keys/<id>
gh cache delete --all -R <org>/<repo>
gh api /orgs/<org>/actions/runners; gh api -X DELETE /orgs/<org>/actions/runners/<id> # re-register from a rebuilt image
# 4. registry side, if our own packages were touched
npm dist-tag add <our package>@<good version> latest; npm deprecate <our package>@<bad version> "compromised, see advisory"
# report to [email protected] (npm) or [email protected] (PyPI) with the version list; PyPI yanks via the project page
Hour 4 – Day 7: Scope, Rebuild
d315c63d1ad4b89e03c7688b29169e977c2abc4559c23b9f6b2282f3c91a6c7d, run elevated from removable media or pushed with your EDR), then a persistence snapshot to diff against a clean build (Windows: Get-PersistenceSnapshot.ps1 a9534865f9e8e7b1f5077b4e1cfd8c29e7e32f6e9e49a4918ce4f8fb713e64df, read-only, -Zip, -CompareTo; Linux runners and hosts: get-persistence-snapshot.sh 8fe386a7d6ebd50225b04ad83581ee3e5d504a10a4263253b5a11b8a61eeca67, --compare-to; both cover the places an install script writes to: cron, systemd, shell rc files, SSH keys, scheduled tasks, Run keys). Then purge package and build caches, delete the Actions cache (gh cache delete --all: a poisoned cache re-infects the next run of a clean workflow), rebuild images from pinned base-image digests with the verified pins, re-image CI runners and the developer machines where the payload executed, and re-register self-hosted runners from the rebuilt image. Hosts where the payload or the vendor's tooling ran are rebuilt, not cleaned — deleting the package folder removes what you know about, not what it dropped. Why: caches and long-lived runners reinstall the bad version or keep what it dropped, and the rebuild destroys the evidence the notification decision needs.Day 1 – Day 30: Notify, Decide, Learn
How Far Has It Got?
Vendor or MSP breach
| # | Stage | ATT&CK | Question | Where to look |
|---|---|---|---|---|
| 1 | Disclosure | T1195 | What exactly does the vendor say happened, from what earliest date, and is the notice genuine? | Vendor security contact, reached on a known number, not the notice; the vendor's IR firm's report |
| 2 | Our connection | T1199 | What access does the vendor have to us: accounts and guests, API keys, OAuth apps, GDAP or DAP, Lighthouse, VPN, RMM, software? | Vendor register; Entra sign-ins by home tenant and path; Partner relationships page; Lighthouse delegations; connected apps in each SaaS; firewall rules; software inventory |
| 3 | Credentials | T1528 T1098.001 | Which credentials or tokens tied to us are in scope, which tenant holds them, and was one added? | Vendor statement; Entra audit log; service-principal sign-ins (key id, owner tenant); our secret inventory |
| 4 | Software | T1195.002 T1553.002 | Did we install an affected update or version, and what else is signed with the same certificate? | Software inventory; SBOM; package versions; file certificate telemetry; module loads |
| 5 | Activity | T1078.004 | Is there anomalous activity from vendor accounts, IPs, integrations or stolen CI tokens in our environment? | Sign-in logs; CloudTrail and cloud audit logs; GitHub audit log; API logs; firewall logs |
| 6 | Our spread | T1219 T1195.001 | Did the attacker move from the vendor's access, its RMM agent or the package's install script into our systems? | EDR; RMM command history; lateral-movement hunts from vendor-connected hosts and runners |
| 7 | Data held | Which of our data does the vendor hold, and whose is it? | Contract and DPA; data map | |
| 8 | Data exposed | T1567 | Is our data confirmed in the stolen set, or dumped somewhere public? | Vendor confirmation; Have I Been Pwned; leak-site monitoring; public repositories and workflow logs |
Compromised package or software update
| # | Stage | ATT&CK | Question | Where to look |
|---|---|---|---|---|
| 1 | Exposure | T1195.001 | Is a bad version in any of our lockfiles, workflows, images, base images or OS packages? | Lockfiles across repos; uses: lines; SBOMs of built images (Syft); image digests; dpkg -l / rpm -q; registry proxy logs |
| 2 | Installed | T1195.001 | Was it actually installed — where and when: developer machines, CI runners, production? | CI build logs; package-manager caches (~/.npm/_cacache); EDR file events; Velociraptor FileFinder |
| 3 | Executed | T1059.007 | Did its install script or code run? | CI job logs; EDR process tree (node spawning cmd.exe /d /s /c or sh -c, trufflehog, gh, AI CLIs); node's DNS and connections |
| 4 | Secrets reachable | T1552.001 T1552.005 | Which secrets could it read where it ran? | CI secret and variable inventory (org, repo, environment, Dependabot); IMDS roles and OIDC tokens on runners; mounted service-account and Vault tokens; ~/.npmrc, ~/.pypirc, ~/.docker/config.json, cloud CLI caches, .env and SSH keys on developer machines |
| 5 | Exfiltration | T1567.001 | Did it send data out? | Egress logs from runners and developer machines; the advisory's domains and IPs; new public repositories and workflow logs in our org |
| 6 | Credential use | T1550.001 | Have the stolen tokens been used? | CloudTrail (AssumeRoleWithWebIdentity subjects, leaked key ids); GitHub audit log; registry publish history |
| 7 | Propagation | T1195.001 | Were our own packages, repositories, workflows or runners modified? | Registry publish history, owners and dist-tags; GitHub audit log (repo.create, repo.access, workflows.*, git.push, secrets, runners); Actions cache |
| 8 | Production | T1195.002 | Did the compromised code reach production builds? | Deploy history; image digests; release records; provenance attestations |
| 9 | Downstream | T1195.002 | Did we ship it to customers or other teams? | Release records; customer deliverables; contractual notification SLAs |
Hunt & Act by Platform
Hunt
Stage 2 · Our connectionEvery external tenant with a path into ours: guests, direct connect, GDAP and DAP
serviceProvider is an MSP acting through a Partner Center GDAP or DAP relationship, b2bCollaboration a guest, b2bDirectConnect a Teams shared channel. Each path has a different cut, so the path matters as much as the account.// Which external tenants act in ours, and by which path (serviceProvider = GDAP/DAP)
union SigninLogs, AADNonInteractiveUserSignInLogs
| where TimeGenerated > ago(90d)
| where CrossTenantAccessType != "none" and HomeTenantId != AADTenantId
| summarize SignIns=count(), LastSeen=max(TimeGenerated), Users=dcount(UserPrincipalName),
Apps=make_set(AppDisplayName, 20) by HomeTenantId, HomeTenantName, CrossTenantAccessType
// The named vendor: every account, what it reached, from where
union SigninLogs, AADNonInteractiveUserSignInLogs
| where TimeGenerated > ago(90d) and HomeTenantId == "<vendor tenant id>"
| summarize LastSeen=max(TimeGenerated), Apps=make_set(AppDisplayName), IPs=make_set(IPAddress, 20)
by UserPrincipalName, UserType, CrossTenantAccessType
Stage 2 · Our connectionVendor-owned apps, their permissions, and Azure Lighthouse delegations
AppOwnerTenantId tells you which apps those are and therefore which credentials you cannot rotate yourself. Lighthouse delegations give the MSP ARM access to subscriptions without any sign-in to your tenant, so the sign-in logs never show them.Non-Microsoft SaaS-to-SaaS grants (a vendor app authorised in Salesforce, Google Workspace, Slack or GitHub) do not appear in Entra at all. List them in each platform's connected-apps page: the Salesloft Drift tokens used against Salesforce in August 2025 were invisible to an Entra-only inventory.
// Service principals the vendor owns (app object, and its secret, live in their tenant)
AADServicePrincipalSignInLogs
| where TimeGenerated > ago(90d)
| where AppOwnerTenantId == "<vendor tenant id>" or AppId == "<vendor app id>"
| summarize LastSeen=max(TimeGenerated), Resources=make_set(ResourceDisplayName), IPs=make_set(IPAddress, 20)
by ServicePrincipalName, AppId, AppOwnerTenantId
// App governance: what those apps are allowed to do, and whether an admin consented
OAuthAppInfo
| where AppOwnerTenantId == "<vendor tenant id>" or AppId == "<vendor app id>"
| project AppName, AppId, PrivilegeLevel, Permissions, IsAdminConsented, AssignedRoles, LastUsedTime
// Azure Lighthouse: delegations written in the window. The current list is
// `az managedservices assignment list` per subscription, or Azure portal > Service providers.
AzureActivity
| where TimeGenerated > ago(90d)
| where OperationNameValue =~ "Microsoft.ManagedServices/registrationAssignments/write"
| project TimeGenerated, Caller, CallerIpAddress, ResourceId, ActivityStatusValue
Stage 3 · CredentialsWhich vendor-app credential is in use, was one added, and whose tenant holds it
AuditLogs
| where TimeGenerated > ago(30d)
| where OperationName in ("Add service principal credentials", "Consent to application",
"Add app role assignment to service principal", "Add delegated permission grant")
or OperationName has "Certificates and secrets management"
| where tostring(TargetResources) has "<vendor app name>"
| project TimeGenerated, OperationName, Result, InitiatedBy, TargetResources
// Anything the vendor's own identities changed in our directory (GDAP technicians, guests, the app)
AuditLogs
| where TimeGenerated > ago(30d)
| where tostring(InitiatedBy) has "<vendor upn domain>" or tostring(InitiatedBy) has "<vendor app id>"
| summarize Count=count(), Examples=make_set(tostring(TargetResources), 10) by OperationName, Category
// Which key the app actually signs in with: KeyId on OUR service principal only if
// AppOwnerTenantId is our tenant; otherwise it is the vendor's app-object key (send them the id)
AADServicePrincipalSignInLogs
| where TimeGenerated > ago(30d) and AppId == "<vendor app id>"
| summarize FirstSeen=min(TimeGenerated), LastSeen=max(TimeGenerated), IPs=make_set(IPAddress)
by AppOwnerTenantId, ServicePrincipalCredentialKeyId, ServicePrincipalCredentialThumbprint
Stage 4 · SoftwareWhere the package, the vendor binary or the sideloaded payload exists, ran or was loaded
ffmpeg.dll loaded by the signed desktop app), so the installer hash alone misses the hosts that matter.// The trojanised binary by hash: written, executed, or loaded as a module
union DeviceFileEvents, DeviceProcessEvents, DeviceImageLoadEvents
| where Timestamp > ago(30d)
| where SHA256 in ("<sha256 of installer>", "<sha256 of payload dll>")
| summarize FirstSeen=min(Timestamp), LastSeen=max(Timestamp), Types=make_set(ActionType) by DeviceName, FolderPath, FileName
// The sideloaded payload by name and parent, hash-agnostic (catches variants the advisory missed)
DeviceImageLoadEvents
| where Timestamp > ago(30d)
| where FileName =~ "<sideloaded dll>" and InitiatingProcessFileName =~ "<vendor exe>"
| summarize Hosts=dcount(DeviceName), Hashes=make_set(SHA256) by FolderPath
// The npm package on disk: laptops and runner workspaces (PyPI: FolderPath has "site-packages")
DeviceFileEvents
| where Timestamp > ago(30d)
| where FolderPath has "node_modules" and FolderPath has "<package>" and FileName =~ "package.json"
| summarize FirstSeen=min(Timestamp) by DeviceName, FolderPath, InitiatingProcessFileName
// Vendor desktop and server software by version. DeviceTvmSoftwareInventory indexes installed
// programs only: it does not see npm, pip, container or base-image contents.
DeviceTvmSoftwareInventory
| where SoftwareVendor has "<vendor>" | project DeviceName, SoftwareName, SoftwareVersion
Stage 4 · SoftwareEverything signed with the vendor's stolen certificate
// Signer, serial and countersignature (timestamp) time come from DeviceFileCertificateInfo, joined on SHA1
DeviceFileCertificateInfo
| where Timestamp > ago(30d)
| where Signer has "<vendor signer name>" or CertificateSerialNumber =~ "<serial from the advisory>"
| join kind=inner (
union DeviceProcessEvents, DeviceImageLoadEvents
| where Timestamp > ago(30d)
| project DeviceName, SHA1, FileName, FolderPath, Timestamp) on SHA1
| summarize Hosts=dcount(DeviceName), FirstSeen=min(Timestamp1), Files=make_set(FileName, 20)
by Signer, CertificateSerialNumber, CertificateCountersignatureTime, IsTrusted
// IsTrusted stays true until the CA revokes and the device fetches the CRL. A countersignature
// dated before the revocation keeps the signature valid even then: trust the serial and the
// countersignature time inside the compromise window, not the trust flag.
Stage 5 · ActivityVendor accounts, apps and MSP technicians signing in from outside the vendor's ranges
union SigninLogs, AADNonInteractiveUserSignInLogs
| where TimeGenerated > ago(30d)
| where HomeTenantId == "<vendor tenant id>"
| where not(ipv4_is_in_any_range(IPAddress, "<vendor cidr>"))
| summarize SignIns=count(), Apps=make_set(AppDisplayName), FirstSeen=min(TimeGenerated)
by UserPrincipalName, CrossTenantAccessType, IPAddress, ResultType
AADServicePrincipalSignInLogs
| where TimeGenerated > ago(30d) and AppId == "<vendor app id>"
| where not(ipv4_is_in_any_range(IPAddress, "<vendor cidr>"))
| summarize SignIns=count(), FirstSeen=min(TimeGenerated)
by ServicePrincipalName, IPAddress, ResourceDisplayName, ServicePrincipalCredentialKeyId
Stage 5 · ActivityConnections and logons into our hosts from the vendor's ranges
DeviceNetworkEvents
| where Timestamp > ago(30d)
| where ipv4_is_in_any_range(RemoteIP, "<vendor cidr>")
| summarize Connections=count(), LastSeen=max(Timestamp) by DeviceName, LocalPort, ActionType, InitiatingProcessFileName
DeviceLogonEvents
| where Timestamp > ago(30d) and ipv4_is_in_any_range(RemoteIP, "<vendor cidr>")
| summarize Logons=count() by DeviceName, AccountName, LogonType, ActionType
Stage 5 · ActivityStolen CI credentials used in the cloud: OIDC role assumption from an unexpected subject
UserIdentityUserName on AssumeRoleWithWebIdentity is the token's sub claim (GitHub: repo:org/repo:ref:refs/heads/main); a subject or source address you cannot map to a workflow is the stolen token in use. CloudTrail event history keeps 90 d; a trail to S3 keeps what you configured.AWSCloudTrail
| where TimeGenerated > ago(30d)
| where EventName == "AssumeRoleWithWebIdentity"
| extend RoleArn = tostring(parse_json(RequestParameters).roleArn)
| where UserIdentityUserName !startswith "repo:<org>/"
or UserIdentityUserName has "pull_request"
or not(ipv4_is_in_any_range(SourceIpAddress, dynamic(["<runner egress cidr>"])))
| summarize Count=count(), FirstSeen=min(TimeGenerated), IPs=make_set(SourceIpAddress, 20) by UserIdentityUserName, RoleArn
// Long-lived keys that lived on runners or laptops: everything they did from a new address
AWSCloudTrail
| where TimeGenerated > ago(30d) and UserIdentityAccessKeyId in ("<leaked access key ids>")
| summarize Calls=count(), Events=make_set(EventName, 30) by UserIdentityAccessKeyId, SourceIpAddress, UserAgent
Stage 6 · Our spreadShells and commands launched by the MSP's remote-management agent
DeviceProcessEvents
| where Timestamp > ago(30d)
// Kaseya AgentMon.exe, ScreenConnect.ClientService.exe, AteraAgent.exe, NinjaRMMAgent.exe, CagService.exe (Datto), BASupSrvc.exe (N-able)
| where InitiatingProcessFileName in~ ("<rmm agent exe>")
| where FileName in~ ("cmd.exe", "powershell.exe", "pwsh.exe", "rundll32.exe", "certutil.exe", "msiexec.exe", "sh", "bash")
| extend Suspicious = ProcessCommandLine has_any ("-EncodedCommand", "-enc ", "certutil", "-decode", "DisableRealtimeMonitoring", "Set-MpPreference", "vssadmin", "bcdedit")
| summarize Hosts=dcount(DeviceName), Examples=make_set(ProcessCommandLine, 25), FirstSeen=min(Timestamp)
by InitiatingProcessFileName, FileName, AccountName, Suspicious
Stage 6 · Our spreadInstall-time execution: lifecycle scripts, in-process credential theft and the 2025 worm pattern
cmd.exe /d /s /c "<script>" on Windows, sh -c on Linux and macOS, with node as the parent. In-process payloads never spawn curl: Shai-Hulud (September and November 2025) ran inside node, fetched trufflehog, read ~/.npmrc and cloud credential files, talked to api.github.com, created repositories and workflows and published with the stolen npm token. Nx s1ngularity (August 2025) used installed claude, gemini and q CLIs as the secret finder. Hunt node's children, its files and its connections, not a download tool.// 1. Lifecycle scripts in the window: every package with an install script matches, so filter to the package
DeviceProcessEvents
| where Timestamp > ago(30d)
| where InitiatingProcessFileName in~ ("node.exe", "node")
| where (FileName =~ "cmd.exe" and ProcessCommandLine has "/d /s /c")
or (FileName in~ ("sh", "bash", "dash") and ProcessCommandLine has "-c")
| where ProcessCommandLine has_any ("<package>", "preinstall", "postinstall")
| summarize Examples=make_set(ProcessCommandLine, 25), FirstSeen=min(Timestamp) by DeviceName, AccountName
// 2. Node or bun spawning the tools the 2025 worms used
DeviceProcessEvents
| where Timestamp > ago(30d)
| where InitiatingProcessFileName in~ ("node.exe", "node", "bun", "bun.exe")
| where FileName in~ ("trufflehog", "trufflehog.exe", "gh", "gh.exe", "claude", "gemini", "q", "q.exe")
or ProcessCommandLine has_any ("--dangerously-skip-permissions", "--yolo", "--trust-all-tools", "npm publish", "gh repo create", "gh auth", "/tmp/inventory.txt")
| project Timestamp, DeviceName, AccountName, FileName, ProcessCommandLine, InitiatingProcessCommandLine
// 3. Node, bun or trufflehog talking to GitHub, a webhook sink, the registry or the metadata service
DeviceNetworkEvents
| where Timestamp > ago(30d)
| where InitiatingProcessFileName in~ ("node.exe", "node", "bun", "bun.exe", "trufflehog", "trufflehog.exe")
| where RemoteUrl has_any ("api.github.com", "webhook.site", "registry.npmjs.org") or RemoteIP in ("169.254.169.254")
| summarize Count=count(), FirstSeen=min(Timestamp) by DeviceName, InitiatingProcessFileName, RemoteUrl, RemoteIP
// 4. Files the worms wrote (Shai-Hulud: bundle.js, setup_bun.js, bun_environment.js, the *.json dumps, migrate-repos.sh)
DeviceFileEvents
| where Timestamp > ago(30d)
| where FileName in~ ("setup_bun.js", "bun_environment.js", "cloud.json", "contents.json", "environment.json", "truffleSecrets.json", "actionsSecrets.json", "migrate-repos.sh", "shai-hulud-workflow.yml")
or (FileName =~ "bundle.js" and FolderPath has "node_modules" and FileSize > 2000000)
| summarize Hosts=dcount(DeviceName), Paths=make_set(FolderPath, 20) by FileName
Act
Broad first, surgical once the scope is known. Export before you cut; collect before you stop anything; compromised hosts and runners are rebuilt, not cleaned.
T+15–60 · ExportExport the identity evidence before it ages out
Export to a case folder with a hash manifest. Ask the vendor's IR firm for its report and send the vendor a litigation-hold letter on day one so their logs are preserved too.
# Entra sign-ins and directory audit (Microsoft Graph PowerShell)
Get-MgAuditLogSignIn -Filter "createdDateTime ge <window start>T00:00:00Z" -All |
Where-Object { $_.HomeTenantId -eq '<vendor tenant id>' -or $_.AppId -eq '<vendor app id>' } |
Export-Csv vendor-signins.csv -NoTypeInformation
Get-MgAuditLogDirectoryAudit -Filter "activityDateTime ge <window start>T00:00:00Z" -All | Export-Csv directory-audit.csv -NoTypeInformation
# Unified audit log (Exchange Online PowerShell): what the vendor's accounts and app touched in M365
Search-UnifiedAuditLog -StartDate <window start> -EndDate (Get-Date) -UserIds <vendor accounts> -ResultSize 5000 -SessionCommand ReturnLargeSet | Export-Csv ual-vendor.csv -NoTypeInformation
# Device timeline for every host in scope (Defender XDR API, 30 d)
GET https://api.security.microsoft.com/api/machines/<machine id>/alerts
Get-ChildItem <case folder> -File | Get-FileHash -Algorithm SHA256 | Export-Csv manifest.csv -NoTypeInformation
T+60–120 · CutCut the vendor's cloud access — each path with its own primitive
Update-MgServicePrincipal cuts the app and nothing else. An MSP reaches you through a GDAP or DAP relationship, Lighthouse delegations and guests, each with its own switch; a path left open while another is cut tells the attacker they have been found.GDAP and DAP: Microsoft 365 admin center › Settings › Partner relationships › select the partner › Remove roles. That ends the relationship from your side; the partner sees it terminated in Partner Center. Cross-tenant access settings govern guests and direct connect only — they do not apply to GDAP sign-ins, which is why the relationship itself has to go. Non-Microsoft OAuth (a vendor app authorised in Salesforce, Google Workspace, Slack, GitHub): revoke it in that platform — Salesforce Setup › Connected Apps OAuth Usage › Block — the Entra cut does not reach it. Ban the vendor's app in Defender for Cloud Apps › OAuth apps as well, so re-consent is blocked.
# Vendor service principal: disable, then strip what it was granted. Issued access tokens stay valid up to ~1 h.
Update-MgServicePrincipal -ServicePrincipalId <sp object id> -AccountEnabled:$false
Get-MgServicePrincipalOauth2PermissionGrant -ServicePrincipalId <sp object id> |
ForEach-Object { Remove-MgOauth2PermissionGrant -OAuth2PermissionGrantId $_.Id }
Get-MgServicePrincipalAppRoleAssignment -ServicePrincipalId <sp object id> |
ForEach-Object { Remove-MgServicePrincipalAppRoleAssignment -ServicePrincipalId <sp object id> -AppRoleAssignmentId $_.Id }
# Credentials: only when AppOwnerTenantId (stage 2) is OUR tenant does the secret live on an object we control.
# A multi-tenant vendor app's secret is on the app object in the vendor's tenant: you cannot rotate it.
# Send the vendor the ServicePrincipalCredentialKeyId from stage 3 and get written confirmation it was rotated.
Remove-MgServicePrincipalPassword -ServicePrincipalId <sp object id> -KeyId <key id> # our-tenant app only
# Guests: disable first, then revoke (revoke alone leaves access tokens valid until they expire)
Update-MgUser -UserId <guest object id> -AccountEnabled:$false
Revoke-MgUserSignInSession -UserId <guest object id>
# Cross-tenant access settings: block the partner tenant inbound (B2B collaboration; not GDAP)
New-MgPolicyCrossTenantAccessPolicyPartner -TenantId <vendor tenant id> -B2BCollaborationInbound @{
usersAndGroups = @{ accessType = "blocked"; targets = @(@{ target = "AllUsers"; targetType = "user" }) }
applications = @{ accessType = "blocked"; targets = @(@{ target = "AllApplications"; targetType = "application" }) } }
# Azure Lighthouse: per delegated subscription
az account set -s <subscription id>
az managedservices assignment list
az managedservices assignment delete --assignment <assignment id>
T+60–120 · CutBlock the payload, the stolen certificate and the update channel fleet-wide
Settings › Endpoints › Indicators: File hashes, Certificates (upload the .CER or .PEM, or the thumbprint via the API) and URLs/Domains. IP indicators take single IPs only: block the vendor's ranges at the firewall or VPN instead. Server-side update compromises (the vendor's build or update server) also need the vendor's service accounts rotated on their side; ask for it in writing.
POST https://api.security.microsoft.com/api/indicators # one call per indicator
{"indicatorValue": "<sha256 of sideloaded dll>", "indicatorType": "FileSha256", "action": "BlockAndRemediate",
"title": "IR <case>: <vendor> payload", "severity": "High", "description": "Supply-chain incident <case>"}
{"indicatorValue": "<certificate thumbprint>", "indicatorType": "CertificateThumbprint", "action": "Block",
"title": "IR <case>: <vendor> stolen signing certificate", "description": "Supply-chain incident <case>"}
{"indicatorValue": "<update host>", "indicatorType": "DomainName", "action": "Block",
"title": "IR <case>: <vendor> update channel until a clean release is confirmed", "description": "Supply-chain incident <case>"}
T+60–120 · CutIsolate every device where the package or vendor tooling ran — in one pass
Live Response keeps working on an isolated device, so collection continues.
$ids | ForEach-Object {
Invoke-RestMethod -Method Post -Headers $h -ContentType 'application/json' `
-Uri "https://api.security.microsoft.com/api/machines/$_/isolate" `
-Body '{"Comment": "IR <case>: supply chain", "IsolationType": "Full"}'
}
T+60–120 · CollectMemory and persistence first, then the file
Upload the tools to the Live Response library once (Settings › Endpoints › Response › Library). Windows memory: Magnet RESPONSE 1.7.2 d315c63d1ad4b89e03c7688b29169e977c2abc4559c23b9f6b2282f3c91a6c7d. Windows persistence: Get-PersistenceSnapshot.ps1 a9534865f9e8e7b1f5077b4e1cfd8c29e7e32f6e9e49a4918ce4f8fb713e64df (read-only; -Zip for chain of custody, -CompareTo against a clean build's snapshot). Linux runners and hosts: get-persistence-snapshot.sh 8fe386a7d6ebd50225b04ad83581ee3e5d504a10a4263253b5a11b8a61eeca67 (cron, systemd, SSH keys, PAM, ld.so.preload, SUID, package verification, --compare-to). Then Device page › Collect investigation package, and File page › Stop and quarantine file (up to 1000 devices).
# Live Response on the isolated device
putfile MagnetRESPONSEv172_Self_Extracting_Archive.exe
run MagnetRESPONSEv172_Self_Extracting_Archive.exe # elevated; RAM, pagefile, running processes, triage files
putfile Get-PersistenceSnapshot.ps1
run Get-PersistenceSnapshot.ps1 -parameters "-Zip"
getfile "<snapshot zip path from the script output>"
getfile "<path to package folder, vendor binary or sideloaded dll>"
# Linux host or runner (Live Response on Linux, or your SSH): run read-only, hash, then pull
run get-persistence-snapshot.sh
After the gate · ScopedRelease a device only after it is rebuilt and the gate is met
Device page › Release from isolation.
POST https://api.security.microsoft.com/api/machines/{id}/unisolate
{"Comment": "IR <case>: rebuilt, eradication gate confirmed"}
Hunt
Stage 2 · Our connectionEvery external tenant with a path into ours: guests, direct connect, GDAP and DAP
serviceProvider is an MSP acting through Partner Center (GDAP or DAP), b2bCollaboration a guest. Each path has a different cut.#Vendor=microsoft @sourcetype="microsoft-entra-id"
| Vendor.properties.crossTenantAccessType!="none"
| groupBy([Vendor.properties.homeTenantId, Vendor.properties.crossTenantAccessType],
function=[max(@timestamp, as=LastSeen), count(as=SignIns), count(Vendor.properties.userPrincipalName, distinct=true, as=Users)])
// The named vendor: accounts, apps and the path used
#Vendor=microsoft @sourcetype="microsoft-entra-id"
| Vendor.properties.homeTenantId="<vendor tenant id>" or Vendor.properties.appId="<vendor app id>"
| groupBy([Vendor.category, Vendor.properties.userPrincipalName, Vendor.properties.crossTenantAccessType, Vendor.properties.appDisplayName],
function=[max(@timestamp, as=LastSeen), count(as=SignIns)])
// Vendor.properties.* mirrors Entra's JSON keys. Confirm homeTenantId, crossTenantAccessType and
// appOwnerTenantId exist in one parsed event before trusting an empty result.
Stage 3 · CredentialsWhich vendor-app credential is in use, was one added, and whose tenant holds it
#Vendor=microsoft @sourcetype="microsoft-entra-id" Vendor.category=AuditLogs
| Vendor.operationName=/Add service principal credentials|Certificates and secrets management|Consent to application|Add app role assignment to service principal|Add delegated permission grant/i
| table([@timestamp, Vendor.operationName, Vendor.properties.initiatedBy.user.userPrincipalName, Vendor.properties.targetResources[0].displayName])
// Filter the table on <vendor app name>. Then the key in use:
#Vendor=microsoft @sourcetype="microsoft-entra-id" Vendor.category=ServicePrincipalSignInLogs
| Vendor.properties.appId="<vendor app id>"
| groupBy([Vendor.properties.appOwnerTenantId, Vendor.properties.servicePrincipalCredentialKeyId, Vendor.properties.servicePrincipalCredentialThumbprint],
function=[min(@timestamp, as=FirstSeen), max(@timestamp, as=LastSeen), count(as=SignIns)])
Stage 4 · SoftwareWhere the package, the vendor binary or the sideloaded payload exists, ran or was loaded
ffmpeg.dll loaded by the signed desktop app), so the installer hash alone misses the hosts that matter.#event_simpleName=ProcessRollup2
| SHA256HashData=~in(values=["<sha256 of installer>", "<sha256 of payload dll>"]) or CommandLine=/<package>/i
| groupBy([ComputerName, ImageFileName, SHA256HashData], function=[min(@timestamp, as=FirstSeen), count(as=Runs)])
// The sideloaded payload as a module load (confirm ClassifiedModuleLoad is populated in your tenant)
#event_simpleName=ClassifiedModuleLoad
| ImageFileName=/\\<sideloaded dll>$/i or SHA256HashData="<sha256 of payload dll>"
| groupBy([ComputerName, ImageFileName, SHA256HashData, TargetProcessId], function=count())
// Execution and loads only. A package that is installed but never ran needs a disk check, RTR on the runner or laptop:
// runscript -Raw=```Get-ChildItem C:\Users, C:\actions-runner, C:\agent -Recurse -Filter package.json -ErrorAction SilentlyContinue | ? FullName -like '*\node_modules\<package>\package.json' | % { $_.FullName; (Get-Content $_.FullName | ConvertFrom-Json).version }```
// runscript -Raw=```find /home /root /opt /srv /builds -path '*/node_modules/<package>/package.json' -exec grep -H '"version"' {} +```
Stage 4 · SoftwareEverything signed with the vendor's stolen certificate
runscript -Raw=```Get-ChildItem 'C:\Program Files', 'C:\Program Files (x86)', 'C:\ProgramData', 'C:\Users' -Recurse -Include *.exe,*.dll,*.msi -ErrorAction SilentlyContinue |
Get-AuthenticodeSignature | Where-Object { $_.SignerCertificate.Thumbprint -eq '<thumbprint>' -or $_.SignerCertificate.SerialNumber -eq '<serial>' } |
Select-Object Path, Status, @{n='Countersigned';e={$_.TimeStamperCertificate.NotBefore}}, @{n='SHA256';e={(Get-FileHash $_.Path).Hash}}```
// Status stays Valid until the CA revokes and the host fetches the CRL; a countersignature dated before the
// revocation keeps it Valid even then. The serial and a signing time inside the window are the evidence.
Stage 5 · ActivityVendor accounts, apps and MSP technicians signing in from outside the vendor's ranges
#Vendor=microsoft @sourcetype="microsoft-entra-id"
| Vendor.properties.homeTenantId="<vendor tenant id>" or Vendor.properties.appId="<vendor app id>"
| !cidr(source.ip, subnet=["<vendor cidr>"])
| groupBy([user.email, Vendor.properties.crossTenantAccessType, Vendor.properties.appDisplayName, source.ip, #event.outcome],
function=[min(@timestamp, as=FirstSeen), count(as=SignIns)])
Stage 5 · ActivityConnections and logons into our hosts from the vendor's ranges
#event_simpleName=NetworkReceiveAcceptIP4
| cidr(RemoteAddressIP4, subnet=["<vendor cidr>"])
| groupBy([aid, ComputerName, LocalPort, RemoteAddressIP4], function=count(as=Connections))
#event_simpleName=UserLogon
| cidr(RemoteAddressIP4, subnet=["<vendor cidr>"])
| groupBy([ComputerName, UserName, LogonType])
Stage 5 · ActivityStolen CI credentials used in the cloud: OIDC role assumption from an unexpected subject
AssumeRoleWithWebIdentity the user name is the token's sub claim (GitHub: repo:org/repo:ref:refs/heads/main); a subject or source address you cannot map to a workflow is the stolen token in use. CloudTrail event history keeps 90 d.// AWS CloudTrail connector, ECS-normalised. Confirm the field names in one parsed AssumeRoleWithWebIdentity event first.
#Vendor=aws event.action="AssumeRoleWithWebIdentity"
| user.name!=/^repo:<org>\// or user.name=/pull_request/ or !cidr(source.ip, subnet=["<runner egress cidr>"])
| groupBy([user.name, source.ip], function=[min(@timestamp, as=FirstSeen), count()])
Stage 6 · Our spreadShells and commands launched by the MSP's remote-management agent
#event_simpleName=ProcessRollup2
// AgentMon.exe (Kaseya), ScreenConnect.ClientService.exe, AteraAgent.exe, NinjaRMMAgent.exe, CagService.exe (Datto), BASupSrvc.exe (N-able)
| ParentBaseFileName=/^<rmm agent exe>$/i
| ImageFileName=/[\\\/](cmd|powershell|pwsh|rundll32|certutil|msiexec)\.exe$|\/(ba)?sh$/i
| Suspicious := if(CommandLine=/-enc|-EncodedCommand|certutil|-decode|DisableRealtimeMonitoring|vssadmin|bcdedit/i, then="yes", else="no")
| groupBy([ParentBaseFileName, ImageFileName, UserName, Suspicious, CommandLine], function=count(ComputerName, distinct=true, as=Hosts))
Stage 6 · Our spreadInstall-time execution: lifecycle scripts, in-process credential theft and the 2025 worm pattern
cmd.exe /d /s /c on Windows and sh -c elsewhere, with node as the parent. In-process payloads never spawn curl: Shai-Hulud (2025) ran inside node, fetched trufflehog, read ~/.npmrc and cloud credential files, talked to api.github.com and published with the stolen token; Nx s1ngularity used installed claude, gemini and q CLIs as the secret finder. Hunt node's children and its DNS, not a download tool.// 1. Lifecycle scripts in the window, scoped to the package
#event_simpleName=ProcessRollup2
| ParentBaseFileName=/^node(\.exe)?$/i
| (ImageFileName=/\\cmd\.exe$/i and CommandLine=/\/d \/s \/c/) or (ImageFileName=/\/(ba|da)?sh$/ and CommandLine=/ -c /)
| CommandLine=/<package>|preinstall|postinstall/i
| groupBy([ComputerName, UserName, CommandLine], function=min(@timestamp, as=FirstSeen))
// 2. Node or bun spawning the tools the worms used
#event_simpleName=ProcessRollup2
| ParentBaseFileName=/^(node|bun)(\.exe)?$/i
| ImageFileName=/[\\\/](trufflehog|gh|claude|gemini|q)(\.exe)?$/i or CommandLine=/--dangerously-skip-permissions|--yolo|--trust-all-tools|npm publish|gh repo create|gh auth|\/tmp\/inventory\.txt/i
| table([@timestamp, ComputerName, UserName, ImageFileName, CommandLine, ParentCommandLine])
// 3. Node, bun or trufflehog resolving GitHub, a webhook sink or the registry (confirm DnsRequest field names in one event)
#event_simpleName=DnsRequest
| ContextBaseFileName=/^(node|bun|trufflehog)(\.exe)?$/i
| DomainName=/api\.github\.com|webhook\.site|registry\.npmjs\.org/i
| groupBy([ComputerName, ContextBaseFileName, DomainName], function=[min(@timestamp, as=FirstSeen), count()])
Act
Broad first, surgical once the scope is known. Export before you cut; collect before you stop anything; compromised hosts and runners are rebuilt, not cleaned.
T+15–60 · ExportExport the identity evidence before it ages out
Export the stage 2, 3 and 5 results from Advanced Event Search (CSV) to a hashed case folder, and run the Graph and Exchange Online exports on the Defender tab for the sources Falcon does not ingest. Litigation-hold letter to the vendor on day one.
T+60–120 · CutCut the vendor's cloud access (Entra, Partner Center, Lighthouse, SaaS)
Not a Falcon action: terminate the GDAP relationship (Microsoft 365 admin center › Partner relationships › Remove roles), remove Lighthouse assignments, block the partner tenant inbound, disable the service principal and revoke guest sessions with the commands on the Defender tab. Non-Microsoft OAuth grants are revoked in each SaaS platform.
T+60–120 · CutBlock the payload and the sideloaded library on every host (custom IOC)
Endpoint security › IOC management › Add indicator: type sha256, action prevent, all hosts. Domains and IPs accept only detect or no_action, so the update host and the vendor's ranges are blocked at the firewall, proxy or DNS.
POST /iocs/entities/indicators/v1
{"indicators": [
{"type": "sha256", "value": "<sha256 of installer>", "action": "prevent", "platforms": ["windows", "mac", "linux"], "applied_globally": true, "severity": "high", "description": "IR <case>: trojanised <vendor> installer"},
{"type": "sha256", "value": "<sha256 of payload dll>", "action": "prevent", "platforms": ["windows"], "applied_globally": true, "severity": "high", "description": "IR <case>: <vendor> sideloaded payload"}
]}
T+60–120 · CutContain every host and runner in scope in one call
Host management › select the hosts › Network contain. RTR keeps working on a contained host.
POST /devices/entities/devices-actions/v2?action_name=contain
{"ids": ["<AID 1>", "<AID 2>", "<AID 3>"]}
T+60–120 · CollectMemory and persistence first, then the file, then kill (RTR)
Put the tools on the host with put (upload them once to Host setup and management › Response scripts and files). Windows memory: Magnet RESPONSE 1.7.2 d315c63d1ad4b89e03c7688b29169e977c2abc4559c23b9f6b2282f3c91a6c7d. Windows persistence: Get-PersistenceSnapshot.ps1 a9534865f9e8e7b1f5077b4e1cfd8c29e7e32f6e9e49a4918ce4f8fb713e64df. Linux runners and hosts: get-persistence-snapshot.sh 8fe386a7d6ebd50225b04ad83581ee3e5d504a10a4263253b5a11b8a61eeca67.
put MagnetRESPONSEv172_Self_Extracting_Archive.exe
runscript -Raw=```Start-Process .\MagnetRESPONSEv172_Self_Extracting_Archive.exe -Wait```
put Get-PersistenceSnapshot.ps1
runscript -Raw=```.\Get-PersistenceSnapshot.ps1 -Zip```
get <snapshot zip path from the script output>
get <path to package folder, vendor binary or sideloaded dll>
# Linux
put get-persistence-snapshot.sh
runscript -Raw=```bash ./get-persistence-snapshot.sh```
ps
kill <pid>
After the gate · ScopedLift containment per host only after rebuild and the gate
POST /devices/entities/devices-actions/v2?action_name=lift_containment
{"ids": ["<AID>"]}
Hunt
Stage 2 · Our connectionWhich of our hosts the vendor reaches, and whether Entra data is in the lake
// Identity half: only if Entra logs are ingested. Discover the parser's fields, then filter on the vendor tenant id:
dataSource.name = '<entra source>' | limit 5
// Endpoint half: hosts that accept logons from the vendor's addresses
event.category = 'logins' AND net_ipsubnet(src.endpoint.ip.address, '<vendor cidr>')
| group Logons = count() by endpoint.name, event.login.userName, event.login.type
Stage 3 · CredentialsCopies of the vendor's key or token on our hosts
event.type = 'Process Creation' AND tgt.process.cmdline contains '<key prefix>'
| columns event.time, endpoint.name, src.process.user, tgt.process.cmdline
// Files that embed the key: RemoteOps Fetch Files on the hosts above, or a Remote Shell grep:
// grep -rl '<key prefix>' /etc /opt /home /srv 2>/dev/null
Stage 4 · SoftwareWhere the package, the vendor binary or the sideloaded payload exists, ran or was loaded
ffmpeg.dll loaded by the signed desktop app), so the installer hash alone misses the hosts that matter.// By hash across every event type (#sha256 expands to every SHA256 field; explicit: tgt.file.sha256, module.sha256)
#sha256 in ('<sha256 of installer>', '<sha256 of payload dll>')
| group Events = count(), Hosts = estimate_distinct(endpoint.name) by event.type, tgt.file.path, module.path
// The sideloaded payload by name and parent, hash-agnostic
event.type = 'Module Load' AND module.path contains '<sideloaded dll>' AND src.process.name = '<vendor exe>'
| group Hosts = estimate_distinct(endpoint.name), Hashes = estimate_distinct(module.sha256) by module.path
// The npm package on disk (PyPI: site-packages)
event.type = 'File Creation' AND tgt.file.path contains 'node_modules' AND tgt.file.path contains '<package>'
| group Files = count() by endpoint.name, src.process.name
// Installed vendor applications by version: Application Inventory in the console (installed programs only;
// it does not see npm, pip or container contents)
Stage 4 · SoftwareEverything signed with the vendor's stolen certificate
src.process.publisher contains '<vendor signer name>' OR tgt.process.publisher contains '<vendor signer name>'
| group Hosts = estimate_distinct(endpoint.name), Hashes = estimate_distinct(tgt.process.image.sha256)
by tgt.process.publisher, tgt.process.name, tgt.process.signedStatus
// Serial number and countersignature time are not in the event: check them on the sample with
// Get-AuthenticodeSignature <file> | Select SignerCertificate, TimeStamperCertificate
// A countersignature dated before the revocation keeps the signature valid; the serial is the evidence.
Stage 5 · ActivityVendor accounts signing in to our hosts from outside the vendor's ranges
event.category = 'logins' AND event.login.userName contains '<vendor account prefix>'
AND NOT net_ipsubnet(src.endpoint.ip.address, '<vendor cidr>')
| group Logons = count() by endpoint.name, event.login.userName, src.endpoint.ip.address
Stage 5 · ActivityConnections into our hosts from the vendor's ranges
event.type = 'IP Connect' AND event.network.direction = 'INCOMING'
AND (net_ipsubnet(src.ip.address, '<vendor cidr>') OR net_ipsubnet(dst.ip.address, '<vendor cidr>'))
| group Connections = count() by endpoint.name, src.process.name, dst.port.number
Stage 6 · Our spreadShells and commands launched by the MSP's remote-management agent
// AgentMon.exe (Kaseya), ScreenConnect.ClientService.exe, AteraAgent.exe, NinjaRMMAgent.exe, CagService.exe (Datto), BASupSrvc.exe (N-able)
event.type = 'Process Creation' AND src.process.name in:anycase ('<rmm agent exe>')
AND tgt.process.name in:anycase ('cmd.exe', 'powershell.exe', 'pwsh.exe', 'rundll32.exe', 'certutil.exe', 'msiexec.exe', 'sh', 'bash')
| group Hosts = estimate_distinct(endpoint.name), First = min(event.time) by src.process.name, tgt.process.cmdline
// Narrow to the suspicious set:
// AND tgt.process.cmdline contains ('-enc', '-EncodedCommand', 'certutil', '-decode', 'DisableRealtimeMonitoring', 'vssadmin', 'bcdedit')
Stage 6 · Our spreadInstall-time execution: lifecycle scripts, in-process credential theft and the 2025 worm pattern
cmd.exe /d /s /c on Windows and sh -c elsewhere, with node as the parent. In-process payloads never spawn curl: Shai-Hulud (2025) ran inside node, fetched trufflehog, read ~/.npmrc and cloud credential files, talked to api.github.com and published with the stolen token; Nx s1ngularity used installed claude, gemini and q CLIs as the secret finder. Hunt node's children, its files and its connections, not a download tool.// 1. Lifecycle scripts in the window, scoped to the package
event.type = 'Process Creation' AND src.process.name in:anycase ('node', 'node.exe')
AND ((tgt.process.name in:anycase ('cmd.exe') AND tgt.process.cmdline contains '/d /s /c')
OR (tgt.process.name in:anycase ('sh', 'bash', 'dash') AND tgt.process.cmdline contains ' -c '))
AND tgt.process.cmdline contains ('<package>', 'preinstall', 'postinstall')
| group First = min(event.time), Cmds = count() by endpoint.name, src.process.user, tgt.process.cmdline
// 2. Node or bun spawning the tools the worms used
event.type = 'Process Creation' AND src.process.name in:anycase ('node', 'node.exe', 'bun', 'bun.exe')
AND (tgt.process.name in:anycase ('trufflehog', 'trufflehog.exe', 'gh', 'gh.exe', 'claude', 'gemini', 'q', 'q.exe')
OR tgt.process.cmdline contains ('--dangerously-skip-permissions', '--yolo', '--trust-all-tools', 'npm publish', 'gh repo create', 'gh auth', '/tmp/inventory.txt'))
| columns event.time, endpoint.name, src.process.user, tgt.process.name, tgt.process.cmdline, src.process.cmdline
// 3. Node, bun or trufflehog connecting to GitHub, a webhook sink, the registry or the metadata service
(event.type = 'IP Connect' OR event.type = 'DNS Resolved')
AND src.process.name in:anycase ('node', 'node.exe', 'bun', 'bun.exe', 'trufflehog', 'trufflehog.exe')
AND (dns.request contains ('api.github.com', 'webhook.site', 'registry.npmjs.org') OR dst.ip.address = '169.254.169.254')
| group Count = count(), First = min(event.time) by endpoint.name, src.process.name, dns.request, dst.ip.address
// 4. Files the worms wrote
event.type = 'File Creation'
AND (tgt.file.path contains ('setup_bun.js', 'bun_environment.js', 'truffleSecrets.json', 'actionsSecrets.json', 'migrate-repos.sh', 'shai-hulud-workflow.yml', 'cloud.json', 'environment.json')
OR (tgt.file.path contains 'node_modules' AND tgt.file.path contains 'bundle.js' AND tgt.file.size > 2000000))
| group Hosts = estimate_distinct(endpoint.name) by tgt.file.path
Act
Broad first, surgical once the scope is known. Export before you cut; collect before you stop anything; compromised hosts and runners are rebuilt, not cleaned.
T+15–60 · ExportExport the identity evidence before it ages out
Export the Event Search results (CSV) to a hashed case folder and run the Graph and Exchange Online exports on the Defender tab for the sources SentinelOne does not hold. Litigation-hold letter to the vendor on day one.
T+60–120 · CutCut the vendor's cloud access (Entra, Partner Center, Lighthouse, SaaS)
Not a SentinelOne action: terminate the GDAP relationship (Microsoft 365 admin center › Partner relationships › Remove roles), remove Lighthouse assignments, block the partner tenant inbound, disable the service principal and revoke guest sessions with the commands on the Defender tab. Non-Microsoft OAuth grants are revoked in each SaaS platform.
T+60–120 · CutBlocklist the payload and the sideloaded library tenant-wide, and block the vendor's ranges and update host
Blocklist › Add, scope tenant (the value field takes the SHA1). For the vendor's ranges and update host: a Firewall Control rule denying <vendor cidr> and <update host> (agents with Firewall Control enabled), POST /web/api/v2.1/firewall-control.
POST /web/api/v2.1/restrictions
{"filter": {"tenant": true},
"data": {"type": "black_hash", "value": "<sha1 of payload dll>", "osType": "windows",
"description": "IR <case>: <vendor> sideloaded payload"}}
T+60–120 · CutDisconnect every agent in scope from the network in one call
Endpoint › Actions › Disconnect from network, or select several endpoints and apply the action to all.
POST /web/api/v2.1/agents/actions/disconnect
{"filter": {"ids": ["<agent id 1>", "<agent id 2>"]}, "data": {}}
T+60–120 · CollectMemory and persistence first, then the files, then kill and quarantine
RemoteOps › Run Script with the tools uploaded to the script library once. Windows memory: Magnet RESPONSE 1.7.2 d315c63d1ad4b89e03c7688b29169e977c2abc4559c23b9f6b2282f3c91a6c7d. Windows persistence: Get-PersistenceSnapshot.ps1 a9534865f9e8e7b1f5077b4e1cfd8c29e7e32f6e9e49a4918ce4f8fb713e64df with -Zip. Linux runners and hosts: get-persistence-snapshot.sh 8fe386a7d6ebd50225b04ad83581ee3e5d504a10a4263253b5a11b8a61eeca67. Then Fetch Files: the snapshot zip, the package folder, the vendor binary and the sideloaded library, shell history. Then Threat › Mitigation: Kill, Quarantine.
After the gate · ScopedReconnect per endpoint only after rebuild and the gate
Endpoint › Actions › Reconnect to network.
Hunt
Stage 2 · Our connectionThe endpoint half of the vendor's connection — its software and its agents on our hosts
Hunt: Windows.Sys.Programs (installed programs; filter the results on the vendor's publisher)
Hunt: Linux.Debian.Packages (dpkg and snap packages)
Hunt: Linux.RHEL.Packages (rpm packages)
Hunt: Generic.System.Pstree (the vendor's agent running right now)
CallChainFilter = (?i)<rmm agent or vendor process>
Stage 3 · CredentialsCopies of the vendor's key or token on our hosts
Hunt: Windows.Search.FileFinder
SearchFilesGlob = C:\ProgramData\**\*.{json,config,ini,xml,env}
YaraRule = rule k { strings: $a = "<key prefix>" condition: $a }
Hunt: Linux.Search.FileFinder
SearchFilesGlob = /{etc,opt,home/*,root}/**/*.{json,yml,yaml,env,conf,npmrc,pypirc}
YaraRule = (same rule)
Hunt: Linux.Sys.BashHistory
SearchRegex = <key prefix>
Stage 4 · SoftwareWhere the package, the vendor binary or the sideloaded payload exists or is loaded
ffmpeg.dll loaded by the signed desktop app), so the installer hash alone misses the hosts that matter.Hunt: Generic.Detection.HashHunter
TargetGlob = C:/**/*.{exe,dll,msi} (narrow to the vendor's install folder)
SHA256List = <sha256 of installer>
<sha256 of payload dll>
Hunt: Windows.System.DLLs (the sideloaded payload loaded right now, with its certificate)
ProcessRegex = (?i)^<vendor exe>$
DllRegex = (?i)<sideloaded dll>$
Calculate_Hash = Y
CertificateInfo = Y
Hunt: Windows.Search.FileFinder
SearchFilesGlob = C:\{Users\*,actions-runner,agent}\**\node_modules\<package>\package.json
Upload_File = Y (the version is inside the file)
Hunt: Linux.Search.FileFinder
SearchFilesGlob = /{home/*,root,opt,srv,builds}/**/node_modules/<package>/package.json
Upload_File = Y
(PyPI: .../site-packages/<package>-*.dist-info/METADATA)
Stage 4 · SoftwareEverything signed with the vendor's stolen certificate
authenticode(); the serial and the signing time inside the window are the evidence, the trust flag is not.-- Notebook or custom artifact; narrow the glob to the vendor's folders on a large fleet
SELECT OSPath, Sig.SubjectName AS Signer, Sig.SerialNumber AS Serial, Sig.Timestamp AS Countersigned, Sig.Trusted AS Trusted,
hash(path=OSPath).SHA256 AS SHA256
FROM foreach(row={
SELECT OSPath FROM glob(globs=['C:/Program Files*/**/*.{exe,dll}', 'C:/ProgramData/**/*.{exe,dll}', 'C:/Users/*/AppData/**/*.{exe,dll}'])
}, query={ SELECT OSPath, authenticode(filename=OSPath) AS Sig FROM scope() })
WHERE Signer =~ "<vendor signer name>" OR Serial =~ "<serial from the advisory>"
-- Trusted stays true until the CA revokes and the host fetches the CRL; a countersignature dated
-- before the revocation keeps it true even then.
Stage 5 · ActivityLogons by the vendor's accounts, and from the vendor's ranges, on our hosts
Hunt: Windows.EventLogs.RDPAuth
UserNameRegex = (?i)<vendor account prefix>
DateAfter = <start of the exposure window>
(compare the source IPs in the results with the vendor's ranges)
Hunt: Windows.EventLogs.RDPAuth
SourceIPRegex = <vendor ip regex> (regex, not CIDR: e.g. ^203\.0\.113\.)
DateAfter = <start of the exposure window>
Hunt: Windows.Network.NetstatEnriched (live connections)
IPRegex = <vendor ip regex>
Hunt: Linux.Network.NetstatEnriched
IPRegex = <vendor ip regex>
Stage 6 · Our spreadShells and commands launched by the MSP's remote-management agent
Hunt: Generic.System.Pstree (live process chains)
CallChainFilter = (?i)<rmm agent exe>.*(cmd|powershell|pwsh|certutil|rundll32|bash)
Hunt: Windows.EventLogs.EvtxHunter (history, if Sysmon or 4688 with command line is logged)
IocRegex = (?i)<rmm agent exe>|-EncodedCommand|certutil.*-decode
DateAfter = <start of the exposure window>
Hunt: Linux.Sys.BashHistory (what the agent's shells ran, if interactive)
SearchRegex = (?i)curl|wget|base64|chmod \+x|crontab|systemctl enable
Stage 6 · Our spreadInstall-time execution: what the package left behind, and what node is running now
bundle.js, setup_bun.js, bun_environment.js, the credential dumps, migrate-repos.sh), the shell history of the install and of trufflehog, the .bashrc sabotage from Nx s1ngularity, and the credential files it read (~/.npmrc, cloud SDK caches).Hunt: Linux.Search.FileFinder
SearchFilesGlob = /{home/*,root,opt,srv,builds,tmp}/**/{setup_bun.js,bun_environment.js,truffleSecrets.json,actionsSecrets.json,cloud.json,environment.json,contents.json,migrate-repos.sh,inventory.txt}
Upload_File = Y
Hunt: Windows.Search.FileFinder
SearchFilesGlob = C:\{Users\*,actions-runner,agent}\**\{setup_bun.js,bun_environment.js,truffleSecrets.json,actionsSecrets.json,migrate-repos.sh}
Hunt: Linux.Sys.BashHistory
SearchRegex = (?i)(npm|pnpm|yarn|bun|pip3?) (install|i|add|ci) .*<package>|trufflehog|gh repo create|npm publish|claude .*--dangerously|gemini .*--yolo
Hunt: Linux.Sys.BashHistory (Nx s1ngularity appended a shutdown to shell rc files)
TargetGlob = /{root,home/*}/.{bashrc,zshrc}
SearchRegex = shutdown
Hunt: Generic.System.Pstree (live, while a build runs)
CallChainFilter = (?i)(node|bun).*(sh|bash|trufflehog|gh|claude|gemini|q)
Act
Broad first, surgical once the scope is known. Export before you cut; collect before you stop anything; compromised hosts and runners are rebuilt, not cleaned.
T+15–60 · ExportExport the identity evidence before it ages out
Run the Graph, Exchange Online and CloudTrail exports from the Defender and Splunk tabs into a hashed case folder. Litigation-hold letter to the vendor on day one.
T+60–120 · CutCut the vendor's cloud access (Entra, Partner Center, Lighthouse, SaaS)
Not a Velociraptor action: terminate the GDAP relationship (Microsoft 365 admin center › Partner relationships › Remove roles), remove Lighthouse assignments, block the partner tenant inbound, disable the service principal and revoke guest sessions with the commands on the Defender tab. Non-Microsoft OAuth grants are revoked in each SaaS platform.
T+60–120 · CutLabel every client in scope, then quarantine them in one hunt
Label Windows and Linux clients separately and start one hunt per label: Windows.Remediation.Quarantine for Windows, Linux.Remediation.Quarantine (nftables) for Linux. The Velociraptor connection stays up. Velociraptor has no hash or certificate block: those land in your EDR or AppLocker/WDAC, and the update host at the proxy or DNS.
SELECT label(client_id=client_id, labels=['sc-win'], op='set')
FROM clients() WHERE client_id IN ('C.<id 1>', 'C.<id 2>')
SELECT hunt(description='IR <case>: quarantine',
artifacts=['Windows.Remediation.Quarantine'],
include_labels=['sc-win'])
FROM scope()
T+60–120 · CollectMemory and persistence first, then the package, the binary and what was run
Windows memory: Windows.Memory.Acquisition, or Magnet RESPONSE 1.7.2 d315c63d1ad4b89e03c7688b29169e977c2abc4559c23b9f6b2282f3c91a6c7d pushed by your EDR or run from removable media. Persistence: paste the body of Get-PersistenceSnapshot.ps1 a9534865f9e8e7b1f5077b4e1cfd8c29e7e32f6e9e49a4918ce4f8fb713e64df into Windows.System.PowerShell's Command (with -Zip), and of get-persistence-snapshot.sh 8fe386a7d6ebd50225b04ad83581ee3e5d504a10a4263253b5a11b8a61eeca67 into Linux.Sys.BashShell's Command on runners and hosts; both are read-only. Then Windows.Search.FileFinder / Linux.Search.FileFinder with Upload_File = Y for the snapshot zip, the package folder, the vendor binary and the sideloaded library; Linux.Sys.BashHistory for what was installed and run.
T+60–120 · StopStop execution
Windows.System.PowerShell on Windows, Linux.Sys.BashShell on Linux, with:
Stop-Process -Id <pid> -Force
kill -9 <pid>
After the gate · ScopedLift quarantine per client only after rebuild and the gate
Re-run the quarantine artifact on that client with RemovePolicy = true.
Hunt
Stage 2 · Our connectionEvery external tenant with a path into ours: guests, direct connect, GDAP and DAP
serviceProvider is an MSP acting through Partner Center (GDAP or DAP), b2bCollaboration a guest. Each path has a different cut.sourcetype=azure:monitor:aad category IN (SignInLogs, NonInteractiveUserSignInLogs) properties.crossTenantAccessType!=none
| stats count as signins max(_time) as last_seen dc(properties.userPrincipalName) as users
by properties.homeTenantId properties.crossTenantAccessType
| convert ctime(last_seen)
`comment("the named vendor: accounts, apps, and its service principals")`
sourcetype=azure:monitor:aad category IN (SignInLogs, NonInteractiveUserSignInLogs, ServicePrincipalSignInLogs)
(properties.homeTenantId="<vendor tenant id>" OR properties.appOwnerTenantId="<vendor tenant id>" OR properties.appId="<vendor app id>")
| stats max(_time) as last_seen values(properties.appDisplayName) as apps values(properties.ipAddress) as ips
by category properties.userPrincipalName properties.crossTenantAccessType properties.servicePrincipalName properties.appOwnerTenantId
| convert ctime(last_seen)
Stage 3 · CredentialsWhich vendor-app credential is in use, was one added, and whose tenant holds it
appOwnerTenantId is the vendor's, the secret is on their app object: only they can rotate it, and the key id from the sign-in is what you send them.sourcetype=azure:monitor:aad category=AuditLogs
(operationName="Add service principal credentials" OR operationName="Update application*Certificates and secrets management*"
OR operationName="Consent to application" OR operationName="Add app role assignment to service principal" OR operationName="Add delegated permission grant")
"<vendor app name>"
| table _time operationName properties.initiatedBy.user.userPrincipalName properties.targetResources{}.displayName
`comment("the key the app signs in with")`
sourcetype=azure:monitor:aad category=ServicePrincipalSignInLogs properties.appId="<vendor app id>"
| stats min(_time) as first_seen max(_time) as last_seen count by properties.appOwnerTenantId properties.servicePrincipalCredentialKeyId properties.servicePrincipalCredentialThumbprint
| convert ctime(first_seen) ctime(last_seen)
Stage 4 · SoftwareWhere the package, the vendor binary or the sideloaded payload exists, ran or was loaded
ffmpeg.dll loaded by the signed desktop app) and every other file signed with the vendor's certificate.index=windows source="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational"
((EventCode=1 (Hashes="*<sha256 of installer>*" OR Hashes="*<sha256 of payload dll>*"))
OR (EventCode=7 (Hashes="*<sha256 of payload dll>*" OR (ImageLoaded="*\\<sideloaded dll>" Image="*\\<vendor exe>")))
OR (EventCode=11 TargetFilename="*node_modules*<package>*package.json"))
| stats min(_time) as first_seen by host EventCode Image ImageLoaded TargetFilename Signature SignatureStatus
| convert ctime(first_seen)
`comment("everything signed with the vendor's certificate: Sysmon 7 Signature is the signer name; the serial is not logged, check it on the sample with Get-AuthenticodeSignature")`
index=windows source="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational" EventCode=7 Signed=true Signature="<vendor signer name>"
| stats dc(host) as hosts values(Hashes) as hashes min(_time) as first_seen by ImageLoaded Image SignatureStatus
| convert ctime(first_seen)
`comment("Linux runners with Sysmon for Linux: source=\"Syslog:Linux-Sysmon/Operational\" (sourcetype=sysmon:linux)")`
Stage 5 · ActivityVendor accounts, apps and MSP technicians signing in from outside the vendor's ranges
sourcetype=azure:monitor:aad category IN (SignInLogs, NonInteractiveUserSignInLogs, ServicePrincipalSignInLogs)
(properties.homeTenantId="<vendor tenant id>" OR properties.appId="<vendor app id>")
| where NOT cidrmatch("<vendor cidr>", 'properties.ipAddress')
| stats count min(_time) as first_seen by properties.userPrincipalName properties.crossTenantAccessType properties.servicePrincipalId properties.ipAddress properties.appDisplayName properties.servicePrincipalCredentialKeyId
| convert ctime(first_seen)
Stage 5 · ActivityConnections and logons into our hosts from the vendor's ranges
index=windows ((source="XmlWinEventLog:Security" EventCode=4624)
OR (source="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational" EventCode=3 Initiated="false"))
| eval src=coalesce(IpAddress, SourceIp)
| where cidrmatch("<vendor cidr>", src)
| stats count by host EventCode TargetUserName Image DestinationPort
`comment("firewall and VPN logs: the same cidrmatch on your firewall index's source-IP field")`
Stage 5 · ActivityStolen CI credentials used in the cloud: OIDC role assumption from an unexpected subject
AssumeRoleWithWebIdentity, userIdentity.userName is the token's sub claim (GitHub: repo:org/repo:ref:refs/heads/main); a subject or source address you cannot map to a workflow is the stolen token in use. CloudTrail event history keeps 90 d; the add-on keeps what your index does.sourcetype=aws:cloudtrail eventName=AssumeRoleWithWebIdentity
| rename userIdentity.userName as sub, requestParameters.roleArn as role
| where NOT like(sub, "repo:<org>/%") OR like(sub, "%pull_request%") OR NOT cidrmatch("<runner egress cidr>", sourceIPAddress)
| stats count min(_time) as first_seen values(sourceIPAddress) as ips by sub role
| convert ctime(first_seen)
`comment("long-lived keys that lived on runners or laptops: everything they did from a new address")`
sourcetype=aws:cloudtrail userIdentity.accessKeyId IN ("<leaked access key ids>")
| stats count values(eventName) as events by userIdentity.accessKeyId sourceIPAddress userAgent
Stage 6 · Our spreadShells and commands launched by the MSP's remote-management agent
index=windows source="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational" EventCode=1
ParentImage IN ("*\\AgentMon.exe", "*\\ScreenConnect.ClientService.exe", "*\\AteraAgent.exe", "*\\NinjaRMMAgent.exe", "*\\CagService.exe", "*\\BASupSrvc.exe", "*\\<rmm agent exe>")
(Image="*\\cmd.exe" OR Image="*\\powershell.exe" OR Image="*\\pwsh.exe" OR Image="*\\rundll32.exe" OR Image="*\\certutil.exe" OR Image="*\\msiexec.exe")
| eval suspicious=if(match(CommandLine, "(?i)-enc|-EncodedCommand|certutil|-decode|DisableRealtimeMonitoring|vssadmin|bcdedit"), "yes", "no")
| stats dc(host) as hosts values(host) as host_list min(_time) as first_seen by ParentImage Image User suspicious CommandLine
| convert ctime(first_seen)
Stage 6 · Our spreadInstall-time execution: lifecycle scripts, in-process credential theft and the 2025 worm pattern
cmd.exe /d /s /c on Windows and sh -c elsewhere, with node as the parent. In-process payloads never spawn curl: Shai-Hulud (2025) ran inside node, fetched trufflehog, read ~/.npmrc and cloud credential files, talked to api.github.com and published with the stolen token; Nx s1ngularity used installed claude, gemini and q CLIs as the secret finder. Hunt node's children, its DNS and its file writes, not a download tool.`comment("1. lifecycle scripts in the window, scoped to the package")`
index=* (source="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational" OR source="Syslog:Linux-Sysmon/Operational") EventCode=1
(ParentImage="*\\node.exe" OR ParentImage="*/node")
((Image="*\\cmd.exe" CommandLine="*/d /s /c*") OR (Image IN ("*/sh", "*/bash", "*/dash") CommandLine="* -c *"))
(CommandLine="*<package>*" OR CommandLine="*preinstall*" OR CommandLine="*postinstall*")
| stats min(_time) as first_seen values(CommandLine) as cmds by host User
`comment("2. node or bun spawning the tools the worms used")`
index=* (source="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational" OR source="Syslog:Linux-Sysmon/Operational") EventCode=1
(ParentImage IN ("*\\node.exe", "*/node", "*\\bun.exe", "*/bun"))
(Image IN ("*/trufflehog", "*\\trufflehog.exe", "*/gh", "*\\gh.exe", "*/claude", "*/gemini", "*/q", "*\\q.exe")
OR CommandLine IN ("*--dangerously-skip-permissions*", "*--yolo*", "*--trust-all-tools*", "*npm publish*", "*gh repo create*", "*gh auth*", "*/tmp/inventory.txt*"))
| table _time host User Image CommandLine ParentCommandLine
`comment("3. node, bun or trufflehog resolving GitHub, a webhook sink or the registry (Sysmon 22), or hitting the metadata service (Sysmon 3)")`
index=* (source="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational" OR source="Syslog:Linux-Sysmon/Operational")
(Image IN ("*\\node.exe", "*/node", "*\\bun.exe", "*/bun", "*/trufflehog", "*\\trufflehog.exe"))
((EventCode=22 QueryName IN ("api.github.com", "webhook.site", "registry.npmjs.org")) OR (EventCode=3 DestinationIp="169.254.169.254"))
| stats count min(_time) as first_seen by host Image QueryName DestinationIp
`comment("4. files the worms wrote (Sysmon 11)")`
index=* (source="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational" OR source="Syslog:Linux-Sysmon/Operational") EventCode=11
TargetFilename IN ("*setup_bun.js", "*bun_environment.js", "*truffleSecrets.json", "*actionsSecrets.json", "*cloud.json", "*environment.json", "*migrate-repos.sh", "*shai-hulud-workflow.yml")
| stats dc(host) as hosts values(TargetFilename) as paths by Image
Act
Broad first, surgical once the scope is known. Export before you cut; collect before you stop anything; compromised hosts and runners are rebuilt, not cleaned.
T+15–60 · ExportPin the evidence to a case index before the window rolls
Create an index with long retention for the case, write each hunt's results into it, and export the same results to a hashed case folder. Run the Graph and Exchange Online exports on the Defender tab for the sources Splunk does not ingest. Litigation-hold letter to the vendor on day one.
<each hunt above> | collect index=ir_<case> marker="hunt=stage2-vendor-identities"
T+60–120 · CutAct from the EDR, Entra and the admin centres, not from Splunk
Terminate the GDAP relationship, remove Lighthouse assignments, block the partner tenant inbound, disable the vendor app and revoke guest sessions (Defender tab); block the hashes and the certificate fleet-wide and isolate every host in scope with your EDR (see its tab). With Splunk SOAR, the EDR's block and isolate actions run from the playbook — pass the full host list from the hunts, not one host.
T+60–120 · CollectMemory and persistence first, through the EDR or by hand
Windows memory: Magnet RESPONSE 1.7.2 d315c63d1ad4b89e03c7688b29169e977c2abc4559c23b9f6b2282f3c91a6c7d, pushed with your EDR or run elevated from removable media. Windows persistence: Get-PersistenceSnapshot.ps1 a9534865f9e8e7b1f5077b4e1cfd8c29e7e32f6e9e49a4918ce4f8fb713e64df with -Zip. Linux runners and hosts: get-persistence-snapshot.sh 8fe386a7d6ebd50225b04ad83581ee3e5d504a10a4263253b5a11b8a61eeca67. Index the snapshot JSON into the case index so the diff against a clean build is a search.
Hunt
Stage 2 · Our connectionEvery external tenant with a path into ours: guests, direct connect, GDAP and DAP
serviceProvider is an MSP acting through Partner Center (GDAP or DAP), b2bCollaboration a guest. Each path has a different cut.FROM logs-azure.signinlogs-*
| WHERE @timestamp > NOW() - 90 days AND azure.signinlogs.properties.cross_tenant_access_type != "none"
| STATS signins = COUNT(*), last_seen = MAX(@timestamp), users = COUNT_DISTINCT(azure.signinlogs.properties.user_principal_name)
BY azure.signinlogs.properties.home_tenant_id, azure.signinlogs.properties.cross_tenant_access_type
// The named vendor: accounts, apps and the path used
FROM logs-azure.signinlogs-*
| WHERE @timestamp > NOW() - 90 days
| WHERE azure.signinlogs.properties.home_tenant_id == "<vendor tenant id>"
OR azure.signinlogs.properties.app_id == "<vendor app id>"
| STATS last_seen = MAX(@timestamp), signins = COUNT(*)
BY azure.signinlogs.category, azure.signinlogs.properties.user_principal_name,
azure.signinlogs.properties.cross_tenant_access_type, azure.signinlogs.properties.service_principal_name
Stage 3 · CredentialsWhich vendor-app credential is in use, was one added, and whose tenant holds it
appOwnerOrganizationId).FROM logs-azure.auditlogs-*
| WHERE @timestamp > NOW() - 30 days
| WHERE azure.auditlogs.operation_name IN ("Add service principal credentials", "Consent to application",
"Add app role assignment to service principal", "Add delegated permission grant")
OR azure.auditlogs.operation_name LIKE "*Certificates and secrets management*"
| KEEP @timestamp, azure.auditlogs.operation_name, azure.auditlogs.properties.initiated_by.user.userPrincipalName,
azure.auditlogs.properties.target_resources.*
// Filter on <vendor app name> in target_resources. The key the app signs in with is in the
// service-principal sign-in properties (same integration, category ServicePrincipalSignInLogs).
Stage 4 · SoftwareWhere the package, the vendor binary or the sideloaded payload exists, ran or was loaded
ffmpeg.dll loaded by the signed desktop app), so the installer hash alone misses the hosts that matter.any where
process.hash.sha256 in ("<sha256 of installer>", "<sha256 of payload dll>") or
file.hash.sha256 in ("<sha256 of installer>", "<sha256 of payload dll>") or
dll.hash.sha256 == "<sha256 of payload dll>" or
(event.category == "file" and file.path : ("*/node_modules/<package>/package.json",
"*\\node_modules\\<package>\\package.json"))
// The sideloaded payload by name and parent, hash-agnostic
library where dll.name : "<sideloaded dll>" and process.name : "<vendor exe>"
Stage 4 · SoftwareEverything signed with the vendor's stolen certificate
any where
process.code_signature.subject_name : "<vendor signer name>" or
dll.code_signature.subject_name : "<vendor signer name>"
// ECS carries subject_name, trusted, status and timestamp, not the serial: check the serial on the sample
// with Get-AuthenticodeSignature. trusted stays true until the CA revokes and the host fetches the CRL;
// a countersignature dated before the revocation keeps it true even then.
Stage 5 · ActivityVendor accounts, apps and MSP technicians signing in from outside the vendor's ranges
FROM logs-azure.signinlogs-*
| WHERE @timestamp > NOW() - 30 days
| WHERE azure.signinlogs.properties.home_tenant_id == "<vendor tenant id>"
OR azure.signinlogs.properties.app_id == "<vendor app id>"
| WHERE NOT CIDR_MATCH(source.ip, "<vendor cidr>")
| STATS signins = COUNT(*), first_seen = MIN(@timestamp)
BY azure.signinlogs.properties.user_principal_name, azure.signinlogs.properties.cross_tenant_access_type,
azure.signinlogs.properties.service_principal_name, source.ip
Stage 5 · ActivityConnections and logons into our hosts from the vendor's ranges
FROM logs-*
| WHERE @timestamp > NOW() - 30 days
| WHERE CIDR_MATCH(source.ip, "<vendor cidr>")
| STATS events = COUNT(*) BY host.name, event.dataset, destination.port, user.name
Stage 5 · ActivityStolen CI credentials used in the cloud: OIDC role assumption from an unexpected subject
AssumeRoleWithWebIdentity the user name is the token's sub claim (GitHub: repo:org/repo:ref:refs/heads/main); a subject or source address you cannot map to a workflow is the stolen token in use. CloudTrail event history keeps 90 d; the integration keeps what your ILM policy does.FROM logs-aws.cloudtrail-*
| WHERE @timestamp > NOW() - 30 days AND event.action == "AssumeRoleWithWebIdentity"
| WHERE NOT user.name LIKE "repo:<org>/*" OR user.name LIKE "*pull_request*" OR NOT CIDR_MATCH(source.ip, "<runner egress cidr>")
| STATS count = COUNT(*), first_seen = MIN(@timestamp), ips = VALUES(source.ip) BY user.name, aws.cloudtrail.user_identity.type
// Long-lived keys that lived on runners or laptops: everything they did from a new address
FROM logs-aws.cloudtrail-*
| WHERE @timestamp > NOW() - 30 days AND aws.cloudtrail.user_identity.access_key_id IN ("<leaked access key ids>")
| STATS calls = COUNT(*), events = VALUES(event.action) BY aws.cloudtrail.user_identity.access_key_id, source.ip, user_agent.original
Stage 6 · Our spreadShells and commands launched by the MSP's remote-management agent
process where event.type == "start" and
process.parent.name : ("AgentMon.exe", "ScreenConnect.ClientService.exe", "AteraAgent.exe", "NinjaRMMAgent.exe", "CagService.exe", "BASupSrvc.exe", "<rmm agent exe>") and
process.name : ("cmd.exe", "powershell.exe", "pwsh.exe", "rundll32.exe", "certutil.exe", "msiexec.exe", "sh", "bash")
// Narrow to the suspicious set: and process.command_line : ("*-enc*", "*-EncodedCommand*", "*certutil*", "*-decode*", "*DisableRealtimeMonitoring*", "*vssadmin*", "*bcdedit*")
Stage 6 · Our spreadInstall-time execution: lifecycle scripts, in-process credential theft and the 2025 worm pattern
cmd.exe /d /s /c on Windows and sh -c elsewhere, with node as the parent. In-process payloads never spawn curl: Shai-Hulud (2025) ran inside node, fetched trufflehog, read ~/.npmrc and cloud credential files, talked to api.github.com and published with the stolen token; Nx s1ngularity used installed claude, gemini and q CLIs as the secret finder. Hunt node's children, its DNS and its file writes, not a download tool.// 1. Lifecycle scripts in the window, scoped to the package
process where event.type == "start" and process.parent.name : ("node", "node.exe") and
((process.name : "cmd.exe" and process.command_line : "*/d /s /c*") or
(process.name : ("sh", "bash", "dash") and process.args : "-c")) and
process.command_line : ("*<package>*", "*preinstall*", "*postinstall*")
// 2. Node or bun spawning the tools the worms used
process where event.type == "start" and process.parent.name : ("node", "node.exe", "bun", "bun.exe") and
(process.name : ("trufflehog", "trufflehog.exe", "gh", "gh.exe", "claude", "gemini", "q", "q.exe") or
process.command_line : ("*--dangerously-skip-permissions*", "*--yolo*", "*--trust-all-tools*", "*npm publish*", "*gh repo create*", "*gh auth*", "*/tmp/inventory.txt*"))
// 3. Node, bun or trufflehog resolving GitHub, a webhook sink or the registry, or hitting the metadata service
any where process.name : ("node", "node.exe", "bun", "bun.exe", "trufflehog", "trufflehog.exe") and
(dns.question.name : ("api.github.com", "webhook.site", "registry.npmjs.org") or destination.ip == "169.254.169.254")
// 4. Files the worms wrote
file where event.type : ("creation", "change") and
(file.name : ("setup_bun.js", "bun_environment.js", "truffleSecrets.json", "actionsSecrets.json", "cloud.json", "environment.json", "migrate-repos.sh", "shai-hulud-workflow.yml") or
(file.name : "bundle.js" and file.path : "*node_modules*" and file.size > 2000000))
Act
Broad first, surgical once the scope is known. Export before you cut; collect before you stop anything; compromised hosts and runners are rebuilt, not cleaned.
T+15–60 · ExportPin the evidence before ILM rolls it
Snapshot the indices in scope to a repository with no delete phase, export each hunt's results to a hashed case folder, and run the Graph and Exchange Online exports on the Defender tab for the sources Elastic does not ingest. Litigation-hold letter to the vendor on day one.
PUT _snapshot/<repo>/ir-<case>-<date>
{"indices": "logs-azure.signinlogs-*,logs-azure.auditlogs-*,logs-aws.cloudtrail-*,logs-endpoint.events.*", "include_global_state": false}
T+60–120 · CutCut the vendor's cloud access (Entra, Partner Center, Lighthouse, SaaS)
Not an Elastic action: terminate the GDAP relationship (Microsoft 365 admin center › Partner relationships › Remove roles), remove Lighthouse assignments, block the partner tenant inbound, disable the service principal and revoke guest sessions with the commands on the Defender tab. Non-Microsoft OAuth grants are revoked in each SaaS platform.
T+60–120 · CutBlocklist the payload by hash and the stolen certificate by signer
Elastic Defend: Blocklist page (or Artifacts › Blocklist, by version) › Add blocklist entry: one entry with the hashes of the installer and the sideloaded payload, and a Windows entry with the signer name.
T+60–120 · CutIsolate every host in scope in one request
Wazuh: isolation is an active-response script run through PUT /active-response, and Wazuh ships none — write and test one before you need it.
POST /api/endpoint/action/isolate
{"comment": "IR <case>: supply chain",
"endpoint_ids": ["<endpoint id 1>", "<endpoint id 2>"]}
T+60–120 · CollectMemory and persistence first, then the file, then kill (response console)
Windows memory: Magnet RESPONSE 1.7.2 d315c63d1ad4b89e03c7688b29169e977c2abc4559c23b9f6b2282f3c91a6c7d. Windows persistence: Get-PersistenceSnapshot.ps1 a9534865f9e8e7b1f5077b4e1cfd8c29e7e32f6e9e49a4918ce4f8fb713e64df. Linux runners and hosts: get-persistence-snapshot.sh 8fe386a7d6ebd50225b04ad83581ee3e5d504a10a4263253b5a11b8a61eeca67.
upload --file MagnetRESPONSEv172_Self_Extracting_Archive.exe # the response shows where it landed
execute --command "<uploaded path>\MagnetRESPONSEv172_Self_Extracting_Archive.exe" --timeout 30m
upload --file Get-PersistenceSnapshot.ps1
execute --command "powershell -ExecutionPolicy Bypass -File <uploaded path>\Get-PersistenceSnapshot.ps1 -Zip"
get-file --path "<snapshot zip path from the script output>"
get-file --path "<path to package folder, vendor binary or sideloaded dll>"
# Linux
upload --file get-persistence-snapshot.sh
execute --command "bash <uploaded path>/get-persistence-snapshot.sh"
processes
kill-process --pid <pid>
After the gate · ScopedRelease per host only after rebuild and the gate
release --comment "IR <case>: rebuilt, eradication gate confirmed"
By Scenario: What Changes
1. Vendor or MSP Breach
Variation: The attacker is in the vendor's environment, not yours — yet. Verify the notice first, then cut data flows and revoke every credential the vendor holds; the question is whether their access into you was used, and the answer is in your logs from the vendor's earliest access date, not from the notice. An MSP is a different incident: its RMM agent is command-and-control as SYSTEM on every host it manages, and in the Kaseya VSA incident (July 2021) the attacker pushed ransomware to downstream organisations (fewer than 1,500 by Kaseya's own count) as a “hot-fix” procedure through the agent. Disable the agent service fleet-wide, revoke the RMM's API keys, end the GDAP or DAP relationship and the Lighthouse delegations, then review every script, policy and procedure the RMM pushed in the window — those are the attacker's actions, not a patch run. The multi-tenant vendor app's secret lives in the vendor's tenant: you cannot rotate it, so get written confirmation that they did. A vendor app authorised in Salesforce, Google Workspace or Slack is cut in that platform; the Salesloft Drift tokens used against Salesforce tenants (August 2025) were untouched by any Entra action. Template: supply-chain-response.
2. Compromised Package or GitHub Action
Variation: Install is execution: a bad version that only reached a CI runner or a laptop still had that machine's secrets. Scope by “where was it installed?”, not “did it reach production?”. The 2025 worms ran inside node with no curl to catch: they harvested ~/.npmrc, cloud and GitHub credentials, published them to public repositories and workflow logs, and used the npm token to trojanise every package the victim maintained — so stage 7 (propagation) is minutes behind stage 3, and your own packages' publish history is a first-hour check, not a day-two one. xz-utils was a distro package carried into base images, not a lockfile entry: pin base images by digest and inventory OS packages in images with Syft, not only application dependencies. Do not pin Actions to tags (every tj-actions tag was moved), do not rebuild on the same caches or runners, and install with --ignore-scripts and a cooling-off period until the registry proxy allowlist is in place. Template: supply-chain-package-compromise.
3. Trojanised Vendor Update
Variation: Treat it as an endpoint compromise, not a patching task: the update ran with the application's privileges on every host that installed it. The signature is the dimension that changes the hunt: the installer is signed with the vendor's real certificate, so a hash block covers one file while the attacker holds the key. Hunt by signer and serial (DeviceFileCertificateInfo, Sysmon 7, code_signature.subject_name) for everything signed with that certificate, block the certificate itself, and read the trust flag with care — it stays valid until the CA revokes and the device fetches the CRL, and a countersignature dated before the revocation keeps the file valid afterwards. In the 3CX case the signed desktop app sideloaded a trojanised ffmpeg.dll: the main executable's hash never identified the payload, the module load did. Block the vendor's update host until it confirms a clean release, because the update channel is the delivery path; if the compromise was on the vendor's build or update server rather than a developer's key, its service accounts are what need rotating, on their side. Isolate the hosts, collect memory and a persistence snapshot, then uninstall or roll back — the pre-encryption situation. Template: supply-chain-package-compromise.