Respond: The First Hour
Check that the organisation trail is logging, GuardDuty / Defender for Cloud / Security Command Center are enabled, and the log sinks and diagnostic settings are intact. Turn back on anything that is off, and write down when it went off.
Why: stopping the trail, deleting the detector and removing log sinks are among the first things a cloud attacker does. Every minute without logs is a minute you can't scope later.
aws cloudtrail get-trail-status --name <org-trail-arn> # IsLogging must be true
aws cloudtrail get-event-selectors --trail-name <org-trail-arn> # are S3 data events on? decides stages 8-9
aws guardduty list-detectors --region <region> # empty = detector deleted
az monitor diagnostic-settings subscription list --subscription <sub-id>
gcloud logging sinks list --organization=<org-id>
gcloud logging buckets describe _Default --location=global --project=<project-id> # retentionDays: 30 unless changed
- Which principal: a long-lived access key, a role or service-account session, a service-principal secret, a console user, the root / Global Admin?
- Which accounts, subscriptions and projects?
- What is happening: discovery, resource abuse (mining), data access, destruction?
Why: the kind of credential decides how it is revoked, and the stage reached decides how wide to cut. Answer How Far Has It Got?, then pick the card in By Scenario.
Deactivating a key or resetting a password does not end sessions already issued from it, and the three clouds differ in what does.
- AWS:
AWSDenyAllon the user denies the user's own sessions on each call, but not sessions of roles the user assumed — each of those roles needs its ownaws:TokenIssueTimedeny, and that deny also breaks every workload sharing the role until it fetches a new session. - Azure: disabling a service principal or running
Revoke-MgUserSignInSessionstops new tokens and kills refresh tokens; access tokens already issued stay valid until they expire (60–90 minutes by default), and Azure Resource Manager does not enforce the revocation early, so removing the role assignments is what actually ends ARM access. A managed identity's tokens are cached for about 24 hours and cannot be forced to refresh. - GCP: disabling the service account rejects its existing access tokens (IAM API reference); disabling only a key does not say so, so disable the account.
Why: a "disabled" key whose temporary credentials keep working for an hour is the most common way cloud containment silently fails. The full commands, per cloud, are under Hunt & Act.
# AWS: the user, then every role it assumed (one revoke per role; expect shared workloads to break)
aws iam update-access-key --user-name <user> --access-key-id <AKIA...> --status Inactive
aws iam attach-user-policy --user-name <user> --policy-arn arn:aws:iam::aws:policy/AWSDenyAll
aws iam put-role-policy --role-name <role> --policy-name AWSRevokeOlderSessions --policy-document file://revoke-older-sessions.json # Deny * where aws:TokenIssueTime < now
# Azure: identities off, every credential gone (SP and app), roles stripped — roles are the real cut
Update-MgServicePrincipal -ServicePrincipalId <sp-objectId> -AccountEnabled:$false
az ad sp credential list --id <sp-objectId>; az ad app credential list --id <appId>; az ad app federated-credential list --id <appId>
Update-MgUser -UserId <upn> -AccountEnabled:$false; Revoke-MgUserSignInSession -UserId <upn>
az role assignment list --assignee <objectId> --all --include-inherited -o table # then delete at every scope
# GCP: the account, not just the key
gcloud iam service-accounts disable <sa-email>
gcloud iam service-accounts keys disable <key-id> --iam-account=<sa-email>
AWS: move the account into the quarantine OU, whose deny-all SCP exempts only your IR role. GCP: attach an IAM deny policy at the project or folder — it applies to tokens already minted, but only covers the permissions on Google's supported list, so when in doubt replace the project's IAM policy with an IR-only one (the T0 dump is what you restore from). Azure has no deny-all equivalent you can apply ad hoc: Azure Policy deny only blocks creating or changing resources through ARM, denyAction only blocks deletes, Deployment Stacks' deny settings only cover the stack's own resources, and none of them touch data-plane reads or Entra — so strip role assignments at subscription and management-group scope, disable the identities, and use a denyAction assignment to pin resources against deletion while you work.
Why: this is the wide cut: one move that stops every credential in the boundary, including the ones you haven't found yet. Two gaps to know: SCPs never apply to the management account or to service-linked roles, and they don't stop principals from other organisations using access a resource policy already granted — that needs a resource control policy or the resource's own policy.
aws organizations move-account --account-id <id> \
--source-parent-id <current-ou> --destination-parent-id <quarantine-ou>
gcloud iam policies create ir-quarantine --kind=denypolicies \
--attachment-point=cloudresourcemanager.googleapis.com%2Fprojects%2F<project-id> \
--policy-file=deny-all-except-ir.json
Confirm backups, snapshots, object versions and keys still exist and can't be deleted by the compromised principal: AWS Backup vault lock, S3 versioning and Object Lock, Azure Recovery Services soft delete (14 days) and multi-user authorisation through a Resource Guard, GCS soft delete (7 days by default, 0–90 configurable) and retention locks. Keys have windows too: a KMS key scheduled for deletion waits 7–30 days and can be cancelled, a Key Vault with soft delete keeps a deleted vault 7–90 days unless it was purged, a Cloud KMS version in DESTROY_SCHEDULED waits at least 24 hours and can be restored. Check each one now, then copy critical snapshots to an account the attacker has no path to.
Why: deleting snapshots, backups and keys is how a cloud attacker turns a breach into an outage; once they see containment start, it is the next move, and a lifecycle rule that expires everything tomorrow is the quiet version.
aws s3api get-bucket-versioning --bucket <bucket>; aws s3api get-bucket-lifecycle-configuration --bucket <bucket>
aws s3api get-object-lock-configuration --bucket <bucket>
aws backup describe-backup-vault --backup-vault-name <vault> # Locked: true
aws kms list-keys --query 'Keys[].KeyId' --output text | xargs -n1 aws kms describe-key --key-id --query 'KeyMetadata.[KeyId,KeyState,DeletionDate]' --output text | grep PendingDeletion
aws kms cancel-key-deletion --key-id <key-id>
az backup vault backup-properties show -g <rg> -n <vault> # softDeleteFeatureState: Enabled
az keyvault list-deleted -o table; az keyvault show -n <vault> --query 'properties.enablePurgeProtection'
gcloud storage buckets describe gs://<bucket> --format='value(soft_delete_policy,retention_policy)'
gcloud kms keys versions list --key=<key> --keyring=<ring> --location=<loc> --filter='state=DESTROY_SCHEDULED'
Only where a VM, container host or function is the foothold, and in that order. Memory first, while the agent channel still works: SSM Run Command, az vm run-command or gcloud compute ssh running AVML (Linux) or a Windows memory tool staged in the evidence bucket, uploaded straight back to it. Then snapshot the disks; then cut the network and remove the workload's identity. Three traps: an isolation security group kills SSM unless the VPC has interface endpoints for ssm, ssmmessages and ec2messages and the group allows 443 to them; an EBS snapshot under the default aws/ebs key cannot be shared to the evidence account — copy it to a customer-managed key first; security-group and NSG rule changes don't cut connections already tracked — add a subnet NACL deny, or an Azure route table with next hop None, if the cut must be immediate. Don't terminate or deallocate.
Why: terminating destroys disk and memory evidence and often the ephemeral logs on the host; detaching the instance profile stops new credentials, but ones already fetched from the metadata service stay valid until the role's sessions are revoked (step above). The per-cloud sequences are under Hunt & Act.
aws ssm send-command --instance-ids <i-id> --document-name AWS-RunShellScript \
--parameters 'commands=["aws s3 cp s3://<evidence-bucket>/tools/avml /tmp/avml","chmod +x /tmp/avml","/tmp/avml --compress /tmp/<i-id>.lime.compressed","aws s3 cp /tmp/<i-id>.lime.compressed s3://<evidence-bucket>/<case>/"]'
aws ec2 create-snapshot --volume-id <vol-id> --description "IR case <n>"
aws ec2 copy-snapshot --source-region <region> --source-snapshot-id <snap-id> --encrypted --kms-key-id <evidence-cmk-arn>
aws ec2 modify-instance-attribute --instance-id <i-id> --groups <isolation-sg>
aws ec2 modify-instance-attribute --instance-id <i-id> --disable-api-termination
aws ec2 disassociate-iam-instance-profile --association-id <iip-assoc-id>
Hour 1–24: Evict, Scope, Preserve
New users, access keys and console passwords; MFA devices added or deactivated and the password policy weakened; new roles and changed trust policies (especially trust to accounts you don't own); new policy versions and removed permissions boundaries; identity providers, Roles Anywhere trust anchors, Identity Center assignments; Lambda code, EventBridge rules, SSM associations and instance user-data that run code on a schedule. Azure: service-principal secrets, certificates and federated credentials (logged as a plain Update application), VM extensions and run commands, Automation runbooks, Logic Apps, federated credentials on managed identities, subscriptions created on your billing account. GCP: service-account keys and iam.serviceAccountTokenCreator grants, ssh-keys in instance or project metadata, OS Login turned off, Cloud Scheduler and Functions, workload identity pool providers without an attribute condition, projects created on your billing account.
Why: removing the entry credential while a backdoor role trusts the attacker's own account is not eviction. The queries are under Hunt & Act.
CloudTrail event history keeps 90 days of management events per region (IAM events land in us-east-1); the Azure Activity Log keeps 90 days; Entra sign-in and audit logs keep 7 days on the free tier and 30 on P1/P2; GCP Admin Activity logs keep 400 days but Data Access and flow logs sit in _Default for 30. Export into an evidence account with write-once storage, hash each export with SHA-256 and record who collected it; the T0 dumps (IAM authorization details and credential report, Resource Graph role assignments and app credentials, search-all-iam-policies) are in the Preserve step of each Hunt & Act tab.
Why: every one of those windows is shorter than a typical investigation, and evidence you can't prove is unaltered is weak evidence for the insurer and the regulator.
aws cloudtrail validate-logs --trail-arn <org-trail-arn> --start-time <window-start>
aws cloudtrail lookup-events --region <region> --start-time <window-start> --max-results 50 --output json > events-<region>.json # one region at a time, 90 d
Get-MgAuditLogSignIn -All -Filter "createdDateTime ge <window-start>" | Export-Csv signins.csv
Get-MgAuditLogDirectoryAudit -All -Filter "activityDateTime ge <window-start>" | Export-Csv directory-audit.csv
az monitor activity-log list --start-time <window-start> --subscription <sub-id> -o json > activity-<sub-id>.json
gcloud logging copy _Default gs://<evidence-bucket>/<case>/logs-<project-id> --location=global --project=<project-id>
sha256sum * | tee SHA256SUMS
Anchor on the attacker's first use (stage 1), then order every stage 3–9 hit by time and principal. CloudTrail Insights, if it was enabled, has already flagged the API-rate spikes; gcloud asset get-history gives the IAM policy of any resource at any point in the last 35 days without depending on logs; Resource Graph's createdOn on role assignments and timeCreated on resources date the things the attacker made.
Why: the status line says how far it got (the ladder); the timeline says in what order, which is what decides scope, notification and whether the eviction is complete.
Look for storage made public, snapshots and images shared to other accounts, storage keys listed, disk SAS issued, secrets read, presigned URLs and SAS tokens issued. Object-level reads (S3 data events, StorageBlobLogs and AZKVAuditLogs, GCP Data Access logs) are off by default; BigQuery is the one exception. If they were off, you cannot show what was read — record that, and scope the notification on what the principal could read. For a workload foothold, VPC, VNet or VPC flow logs (also off by default) give the bytes out.
Why: "no evidence of exfiltration" and "no logs that could show exfiltration" are different statements, and regulators treat them differently.
The usual ones: a key committed to a repository or baked into an image; a credential stolen from a developer laptop; SSRF to the instance metadata service (IMDSv1); a CI/CD pipeline identity with a trust policy that is too broad; an OAuth app or third-party integration.
Why: restoring before the entry point is closed means doing this again next week — with an attacker who now knows your environment.
trufflehog git https://github.com/<org>/<repo> --only-verified
aws ec2 describe-instances --query "Reservations[].Instances[?MetadataOptions.HttpTokens=='optional'].InstanceId"
Not only its own keys: everything in Secrets Manager, Parameter Store, Key Vault and Secret Manager it had read access to, function environment variables, database passwords, and third-party API keys stored in the account.
Why: an attacker who listed the secret store leaves with credentials to systems outside the cloud account, and those don't show up in any cloud log.
Day 1–7: Notify, Rebuild, Recover
GDPR: supervisory authority within 72 h of becoming aware of a personal-data breach, data subjects without undue delay if the risk is high. NIS2: early warning within 24 h, notification within 72 h, final report within one month. DORA (financial entities): initial notification within 4 h of classifying the incident as major, and no later than 24 h after becoming aware.
Why: the clocks start before scoping ends; notify with what you know and update.
Redeploy compromised workloads from infrastructure-as-code and known-good images, and bring data back from snapshots or versions taken before the earliest attacker activity. Where the whole account is suspect, deploy into a new account and migrate data rather than scrubbing the old one.
Gate: reconnect only after the incident lead declares eradication complete — the entry point closed, every attacker-created identity and resource removed, every readable secret rotated, and a verification hunt clean since the eviction time.
Why: one missed role trust or scheduled function puts the attacker straight back in, and the cloud makes rebuilding cheaper than proving a resource is clean.
Cost Explorer or billing export for the incident window, per region and per service, and the list of resources the attacker created.
Why: attacker-run compute can reach five figures in days; the billing case with the provider and the insurance claim both need the figure and the evidence behind it.
After Day 7: Feed the Slow Loops
- Entry route → Cloud Security posture: long-lived keys replaced by roles and workload identity, IMDSv2 enforced, pipeline trust policies narrowed — everywhere the same weakness exists.
- What slowed containment → structure: no pre-built quarantine OU, no read-only IR role across the estate, a region lock without a responder exemption, logs you had to find before you could read them.
- What detection missed → the detection backlog: each stage they reached without an alert (logging tampering, new keys, cross-account trust, launches in unused regions) becomes a tested detection with an owner.
- Guardrail gaps they used → the management account, service-linked roles, external principals, self-granting admin roles: close the ones this incident proved real.
- Indicators become intelligence only after an analyst checks them and gives each one a date, a source, an expiry and a confidence level. Attacker IPs and access-key IDs rotate; an unexpired block list becomes noise.
How Far Has It Got?
| # | Stage | ATT&CK | Question | Where to look |
|---|---|---|---|---|
| 1 | Initial access | T1078.004 | Which credential, from where, and when was its first use by the attacker? | Per-key (IP, user agent) pairs against the previous 90 days in Athena; GuardDuty credential-exfiltration findings; the four Entra sign-in tables (user, non-interactive, service principal, managed identity); GCP callerIp, user agent and serviceAccountKeyName |
| 2 | Discovery | T1087.004 T1580 | Have they enumerated who they are and what exists? | AWS: readonly = true bursts and AccessDenied counts per principal (GetCallerIdentity, ListBuckets, GetAccountAuthorizationDetails). Azure: MicrosoftGraphActivityLogs — the Activity Log never records reads. GCP: Data Access logs, off by default; AzureHound, ROADtools, Pacu user agents |
| 3 | Privilege escalation | T1098.003 T1548.005 | Did they give themselves more permissions? | Policy versions, attachments, permissions boundaries, trust policies, iam:PassRole, Identity Center assignments; Azure role assignments and Entra elevateAccess (service “Azure RBAC (Elevated Access)” in AuditLogs); GCP SetIamPolicy with bindingDeltas |
| 4 | Persistence | T1098.001 T1136.003 | Did they create a way back in? | Users, keys, login profiles, MFA devices, password policy, roles, identity providers, Lambda code, EventBridge rules, SSM associations, Roles Anywhere; service-principal secrets and federated credentials, VM extensions, run commands, runbooks, new subscriptions; service-account keys, ssh-keys metadata, OS Login off, Scheduler and Functions, identity pools, new projects |
| 5 | Defence evasion | T1562.008 | Did they blind you? | Trails stopped, narrowed or deleted, event selectors and Insights changed, DeleteEventDataStore, GuardDuty filters and IP sets, trail-bucket lifecycle and policy; diagnostic settings, workspace retention, Defender plans, Sentinel connectors, locks; sinks, exclusions, bucket retention, auditConfigDeltas, SCC mutes; activity in regions nobody uses |
| 6 | Lateral movement | T1550.001 T1021.007 | Did they reach other accounts, subscriptions or projects? | AssumeRole chains via sessionContext.sessionIssuer and cross-account hops joined on sharedEventID; a principal in subscriptions it never used, Lighthouse and management-group writes, Entra elevateAccess; GCP GenerateAccessToken / SignBlob on iamcredentials (Data Access logs) with the delegation chain |
| 7 | Resource abuse | T1496.001 T1496.004 | Are they running compute on your bill? | RunInstances, fleets, SageMaker, ECS/EKS, Lightsail, Batch, Bedrock entitlements, quota increases; scale sets, AKS, ML compute, Azure OpenAI deployments; GKE, Dataproc, Vertex AI, Cloud Run jobs — in every region and every project; cost anomaly alerts |
| 8 | Data access | T1530 T1537 | What could they read, and what did they read? | Could: storage made public, snapshots shared out, keys listed, SAS and beginGetAccess, GetSecretValue. Did: S3 data events, AZKVAuditLogs, StorageBlobLogs, GCP Data Access logs — all off by default; flow logs for a workload foothold |
| 9 | Impact | T1485 T1486 T1490 | Are they deleting, re-encrypting or holding data hostage? | DeleteObjects, bucket lifecycle and replication rules, SSE-C writes (SSEApplied), ScheduleKeyDeletion, DeleteBackupVault, CloseAccount; backupconfig/write, Resource Guard and vault purge; DestroyCryptoKeyVersion, billing detached, soft delete removed; ransom notes in buckets |
| 10 | Extortion | T1657 | Have they made contact, or is data already published? | Emails to staff, the root or security contact, support cases you did not open; leak-site monitoring. No cloud log records this stage |
Hunt & Act by Cloud
Primary: Athena over the organisation trail's S3 bucket (table cloudtrail_logs from the CloudTrail docs' partition-projection DDL; column names are lower-case, useridentity is a struct, requestparameters and additionaleventdata are JSON strings).
The same SQL runs in CloudTrail Lake with camel-case fields.
Fallback: aws cloudtrail lookup-events — no setup, but 90 days of management events only, one region and one lookup attribute per call, 2 requests per second, and the IP and user agent are inside the CloudTrailEvent JSON string (hence the jq).
IAM, STS, Organizations and Account Management events land in us-east-1.
Data events (S3 objects, Lambda invokes, DynamoDB items, Bedrock invocations) exist only if the trail's event selectors include them; where a hunt needs them it says so.
Hunt
Stage 1 · Initial accessEverything one access key did, from where, with which client
No trail at all? aws iam get-access-key-last-used --access-key-id <AKIA...> still gives the last service, region and time. A key AWS found in public is also announced as an AWS Health event AWS_RISK_CREDENTIALS_EXPOSED and a quarantine policy on the user.
-- first and last use per source: which (IP, user agent) pairs are the owner, which are the attacker
SELECT sourceipaddress, useragent, min(eventtime) AS first_seen, max(eventtime) AS last_seen,
count(*) AS calls, count_if(errorcode IS NOT NULL) AS errors,
array_join(array_agg(DISTINCT eventsource), ',') AS services
FROM cloudtrail_logs
WHERE useridentity.accesskeyid = '<AKIA...>'
AND eventtime > '<window-start ISO8601>'
GROUP BY 1, 2 ORDER BY first_seen;
-- then the full timeline for the attacker's pairs
SELECT eventtime, awsregion, eventsource, eventname, sourceipaddress, useragent, errorcode, requestparameters
FROM cloudtrail_logs
WHERE useridentity.accesskeyid = '<AKIA...>' AND sourceipaddress IN ('<attacker-ip>')
ORDER BY eventtime;
# no Athena: one region, 90 days, IP and UA pulled out of the CloudTrailEvent string
aws cloudtrail lookup-events --region <region> --start-time <window-start> \
--lookup-attributes AttributeKey=AccessKeyId,AttributeValue=<AKIA...> --output json \
| jq -r '.Events[].CloudTrailEvent | fromjson
| [.eventTime, .awsRegion, .eventSource, .eventName, .sourceIPAddress, .userAgent, (.errorCode // "")] | @tsv'
Stage 2 · DiscoveryEnumeration bursts and permission probing
Why: an attacker's first minutes are GetCallerIdentity, ListBuckets, GetAccountAuthorizationDetails, ListSecrets and a spray of Describe/List calls, many of them denied.
A principal that touches forty distinct read APIs in an hour, or collects twenty AccessDenieds, is enumerating.
GetCallerIdentity is never denied and never needs a permission, so it is the one call every stolen key makes.
SELECT useridentity.arn AS principal, sourceipaddress, useragent,
date_trunc('hour', from_iso8601_timestamp(eventtime)) AS hour,
count(DISTINCT eventname) AS distinct_calls,
count_if(errorcode = 'AccessDenied') AS denied,
count_if(eventname IN ('GetCallerIdentity','ListBuckets','GetAccountAuthorizationDetails','ListSecrets',
'ListUsers','ListRoles','ListAccessKeys','GetAccountSummary','DescribeRegions')) AS hallmark_calls,
array_join(array_agg(DISTINCT eventsource), ',') AS services
FROM cloudtrail_logs
WHERE eventtime > '<window-start ISO8601>' AND readonly = 'true'
GROUP BY 1, 2, 3, 4
HAVING count(DISTINCT eventname) > 40 OR count_if(errorcode = 'AccessDenied') > 20
ORDER BY denied DESC, distinct_calls DESC;
Stage 3 · Privilege escalationPolicies, boundaries and trust changed; a role handed to compute
iam:PassRole handing an admin role to an instance or function the attacker controls. Identity Center assignments are the same thing one level up.The before/after of a policy version: aws iam get-policy-version --policy-arn <arn> --version-id v<n> for the attacker's version and the previous one. The T0 dump from the Preserve step is the authoritative “before”.
SELECT eventtime, eventsource, eventname, useridentity.arn AS actor, sourceipaddress, requestparameters, errorcode
FROM cloudtrail_logs
WHERE eventtime > '<window-start ISO8601>' AND (
(eventsource = 'iam.amazonaws.com' AND eventname IN (
'CreatePolicyVersion','SetDefaultPolicyVersion','AttachUserPolicy','AttachRolePolicy','AttachGroupPolicy',
'PutUserPolicy','PutRolePolicy','PutGroupPolicy','AddUserToGroup','UpdateAssumeRolePolicy',
'PutUserPermissionsBoundary','DeleteUserPermissionsBoundary','PutRolePermissionsBoundary','DeleteRolePermissionsBoundary',
'CreateLoginProfile','UpdateLoginProfile'))
OR (eventsource = 'sso.amazonaws.com' AND eventname IN (
'CreateAccountAssignment','CreatePermissionSet','AttachManagedPolicyToPermissionSet',
'PutInlinePolicyToPermissionSet','AttachCustomerManagedPolicyReferenceToPermissionSet'))
-- iam:PassRole: a privileged role attached to compute at launch
OR (eventname IN ('RunInstances','CreateFunction20150331','CreateStack','CreateAutoScalingGroup','RegisterTaskDefinition')
AND regexp_like(requestparameters, 'iamInstanceProfile|"role"|roleArn|RoleARN')))
ORDER BY eventtime;
Stage 4 · PersistenceUsers, keys, MFA, identity providers, functions, rules, SSM, Roles Anywhere
Lambda event names carry an API-version suffix (CreateFunction20150331, UpdateFunctionCode…, AddPermission20150331v2), hence the LIKE. For every hit, the T0 dump answers whether the object existed before the incident.
SELECT eventtime, awsregion, eventsource, eventname, useridentity.arn AS actor, sourceipaddress, requestparameters
FROM cloudtrail_logs
WHERE eventtime > '<window-start ISO8601>' AND errorcode IS NULL AND (
(eventsource = 'iam.amazonaws.com' AND eventname IN (
'CreateUser','CreateAccessKey','CreateLoginProfile','UpdateLoginProfile','CreateRole','UpdateAssumeRolePolicy',
'CreateSAMLProvider','UpdateSAMLProvider','CreateOpenIDConnectProvider','AddClientIDToOpenIDConnectProvider',
'UpdateOpenIDConnectProviderThumbprint','CreateVirtualMFADevice','EnableMFADevice','DeactivateMFADevice',
'DeleteVirtualMFADevice','UpdateAccountPasswordPolicy','DeleteAccountPasswordPolicy',
'CreateServiceSpecificCredential','UploadSSHPublicKey','UploadSigningCertificate'))
OR (eventsource = 'lambda.amazonaws.com' AND (eventname LIKE 'CreateFunction%' OR eventname LIKE 'UpdateFunctionCode%'
OR eventname LIKE 'UpdateFunctionConfiguration%' OR eventname LIKE 'AddPermission%' OR eventname LIKE 'CreateEventSourceMapping%'
OR eventname = 'CreateFunctionUrlConfig'))
OR (eventsource = 'events.amazonaws.com' AND eventname IN ('PutRule','PutTargets'))
OR (eventsource = 'ssm.amazonaws.com' AND eventname IN ('CreateAssociation','UpdateAssociation','SendCommand','StartSession','CreateDocument','UpdateDocument'))
OR (eventsource = 'rolesanywhere.amazonaws.com' AND eventname IN ('CreateTrustAnchor','UpdateTrustAnchor','CreateProfile','UpdateProfile'))
OR (eventsource = 'sso.amazonaws.com' AND eventname IN ('CreateAccountAssignment','CreatePermissionSet','CreateInstance'))
OR (eventsource = 'ec2.amazonaws.com' AND eventname IN ('ModifyInstanceAttribute','CreateLaunchTemplateVersion') AND requestparameters LIKE '%userData%'))
ORDER BY eventtime;
Stage 5 · Defence evasionLogging and detection switched off, narrowed or starved
StopLogging or DeleteDetector; the quiet ones are UpdateTrail dropping multi-region or global events, PutEventSelectors removing data events, a GuardDuty filter that archives findings, an IP set that trusts the attacker's address, a lifecycle rule or policy change on the trail bucket, and DeleteEventDataStore on CloudTrail Lake.SELECT eventtime, awsregion, eventsource, eventname, useridentity.arn AS actor, sourceipaddress, requestparameters
FROM cloudtrail_logs
WHERE eventtime > '<window-start ISO8601>' AND errorcode IS NULL AND (
(eventsource = 'cloudtrail.amazonaws.com' AND eventname IN (
'StopLogging','DeleteTrail','UpdateTrail','PutEventSelectors','PutInsightSelectors',
'DeleteEventDataStore','UpdateEventDataStore','StopEventDataStoreIngestion'))
OR (eventsource = 'guardduty.amazonaws.com' AND eventname IN (
'DeleteDetector','UpdateDetector','CreateFilter','UpdateFilter','CreateIPSet','UpdateIPSet',
'CreateThreatIntelSet','UpdateThreatIntelSet','DeleteMembers','DisassociateFromAdministratorAccount',
'DeletePublishingDestination','UpdateMalwareScanSettings'))
OR (eventsource = 'config.amazonaws.com' AND eventname IN ('StopConfigurationRecorder','DeleteConfigurationRecorder','DeleteDeliveryChannel','DeleteConfigRule'))
OR (eventsource = 'securityhub.amazonaws.com' AND eventname IN ('DisableSecurityHub','BatchDisableStandards','DisableImportFindingsForProduct'))
OR (eventsource = 'ec2.amazonaws.com' AND eventname = 'DeleteFlowLogs')
OR (eventsource = 'macie2.amazonaws.com' AND eventname = 'DisableMacie')
OR (eventsource = 'detective.amazonaws.com' AND eventname = 'DeleteGraph')
OR (eventsource = 'logs.amazonaws.com' AND eventname IN ('DeleteLogGroup','PutRetentionPolicy','DeleteSubscriptionFilter','DeleteResourcePolicy'))
OR (eventsource = 's3.amazonaws.com'
AND eventname IN ('PutBucketLifecycle','PutBucketPolicy','DeleteBucketPolicy','PutBucketVersioning','DeleteBucket','PutBucketAcl')
AND json_extract_scalar(requestparameters, '$.bucketName') IN ('<trail-bucket>', '<flow-log-bucket>')))
ORDER BY eventtime;
Stage 5 · Defence evasionGuardDuty findings you haven't read, log-file integrity, Insights
validate-logs proves the trail files in S3 were not altered or deleted after delivery (needs log-file validation on the trail); CloudTrail Insights, if enabled, has already flagged the API-rate spike that the attacker's enumeration or mass-delete produced.aws guardduty list-findings --region <region> --detector-id <detector-id> --finding-criteria '{"Criterion":{"type":{"Eq":[
"Stealth:IAMUser/CloudTrailLoggingDisabled","Stealth:IAMUser/PasswordPolicyChange","Policy:IAMUser/RootCredentialUsage",
"UnauthorizedAccess:IAMUser/TorIPCaller","UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration.OutsideAWS",
"Discovery:IAMUser/AnomalousBehavior","Persistence:IAMUser/AnomalousBehavior","Exfiltration:IAMUser/AnomalousBehavior",
"CredentialAccess:IAMUser/AnomalousBehavior","DefenseEvasion:IAMUser/AnomalousBehavior"]}}}' \
| jq -r '.FindingIds[]' | xargs -r aws guardduty get-findings --region <region> --detector-id <detector-id> --finding-ids \
| jq -r '.Findings[] | [.UpdatedAt, .Type, .Severity, .Resource.AccessKeyDetails.UserName // "", .Title] | @tsv'
# were the delivered log files tampered with after the fact?
aws cloudtrail validate-logs --region <trail-home-region> --trail-arn <org-trail-arn> --start-time <window-start>
# Insights (only if PutInsightSelectors was ever applied to the trail)
aws cloudtrail lookup-events --region <region> --event-category insight --start-time <window-start> \
--query 'Events[].[EventTime,EventName,Resources[0].ResourceName]' --output table
Stage 6 · Lateral movementRole chains and cross-account assumption
Why: cloud lateral movement is AssumeRole.
A session assuming another role is a chain; a target account that differs from the caller's account is a hop into another account.
useridentity.sessioncontext.sessionissuer.arn names the role the caller already held, requestparameters.roleArn the one it took, and sharedeventid ties the caller-side record to the record in the target account on the organisation trail.
Console role switches are SwitchRole under signin.amazonaws.com. The same principal or IP in a partner's or customer's trail is the next question — ask them.
SELECT eventtime, useridentity.type AS caller_type, useridentity.arn AS caller,
useridentity.sessioncontext.sessionissuer.arn AS caller_role,
json_extract_scalar(requestparameters, '$.roleArn') AS target_role,
json_extract_scalar(requestparameters, '$.roleSessionName') AS session_name,
useridentity.accountid AS from_account, recipientaccountid AS to_account,
sharedeventid, sourceipaddress, useragent, errorcode
FROM cloudtrail_logs
WHERE eventsource = 'sts.amazonaws.com'
AND eventname IN ('AssumeRole','AssumeRoleWithSAML','AssumeRoleWithWebIdentity','GetFederationToken','AssumeRoot')
AND eventtime > '<window-start ISO8601>'
AND useridentity.invokedby IS NULL -- drop AWS services assuming roles on your behalf
AND (useridentity.accountid <> recipientaccountid -- cross-account hop
OR useridentity.type = 'AssumedRole') -- role assuming a role: a chain
ORDER BY eventtime;
Stage 7 · Resource abuseCompute, fleets, GPUs, model access and quota increases in every region
Why: miners launch where you don't look and in the shapes you don't watch: spot fleets, SageMaker notebooks and training jobs, ECS tasks, EKS node groups, Lightsail, Batch.
A RequestServiceQuotaIncrease for GPU families is the attacker preparing.
Since 2024 the same stolen key is used to enable and invoke Bedrock models and resell the access.
What is live right now: loop aws ec2 describe-instances --region $r --filters Name=instance-state-name,Values=running --query 'Reservations[].Instances[].[InstanceId,InstanceType,LaunchTime]' over describe-regions. Bedrock InvokeModel is a data event; the entitlement calls below are management events.
SELECT eventtime, awsregion, eventsource, eventname, useridentity.arn AS actor, sourceipaddress,
json_extract_scalar(requestparameters, '$.instanceType') AS instance_type, errorcode
FROM cloudtrail_logs
WHERE eventtime > '<window-start ISO8601>' AND useridentity.invokedby IS NULL AND (
(eventsource = 'ec2.amazonaws.com' AND eventname IN ('RunInstances','CreateFleet','RequestSpotFleet','RequestSpotInstances','CreateLaunchTemplate'))
OR (eventsource = 'sagemaker.amazonaws.com' AND eventname IN ('CreateNotebookInstance','CreateTrainingJob','CreateProcessingJob','CreateEndpoint'))
OR (eventsource = 'ecs.amazonaws.com' AND eventname IN ('RunTask','CreateService','RegisterTaskDefinition'))
OR (eventsource = 'eks.amazonaws.com' AND eventname IN ('CreateCluster','CreateNodegroup','CreateFargateProfile'))
OR (eventsource = 'lightsail.amazonaws.com' AND eventname IN ('CreateInstances','CreateContainerService'))
OR (eventsource = 'batch.amazonaws.com' AND eventname IN ('CreateComputeEnvironment','SubmitJob'))
OR (eventsource = 'servicequotas.amazonaws.com' AND eventname = 'RequestServiceQuotaIncrease')
OR (eventsource = 'bedrock.amazonaws.com' AND eventname IN ('PutFoundationModelEntitlement','PutUseCaseForModelAccess','CreateModelInvocationJob','InvokeModel')))
ORDER BY eventtime;
Stage 8 · Data accessStorage opened, snapshots shared, secrets read, disks exported
GetSecretValue and a decrypted GetParameter are management events, so the secret reads are visible even without data events.SELECT eventtime, awsregion, eventsource, eventname, useridentity.arn AS actor, sourceipaddress, requestparameters
FROM cloudtrail_logs
WHERE eventtime > '<window-start ISO8601>' AND errorcode IS NULL AND (
(eventsource = 's3.amazonaws.com' AND (eventname IN ('PutBucketPolicy','PutBucketAcl','PutAccessPointPolicy','PutBucketCors')
OR eventname LIKE '%PublicAccessBlock%'))
OR (eventsource = 'ec2.amazonaws.com' AND eventname IN ('ModifySnapshotAttribute','ModifyImageAttribute','CopySnapshot','CreateSnapshot','CreateSnapshots','CreateImage'))
OR (eventsource = 'rds.amazonaws.com' AND eventname IN ('ModifyDBSnapshotAttribute','ModifyDBClusterSnapshotAttribute','CopyDBSnapshot','CreateDBSnapshot','StartExportTask'))
OR (eventsource = 'secretsmanager.amazonaws.com' AND eventname IN ('GetSecretValue','BatchGetSecretValue','ListSecrets','PutResourcePolicy'))
OR (eventsource = 'ssm.amazonaws.com' AND eventname IN ('GetParameter','GetParameters','GetParametersByPath') AND requestparameters LIKE '%"withDecryption":true%')
OR (eventsource = 'kms.amazonaws.com' AND eventname IN ('PutKeyPolicy','CreateGrant','ReplicateKey'))
OR (eventsource = 'dynamodb.amazonaws.com' AND eventname = 'ExportTableToPointInTime')
OR (eventsource = 'sts.amazonaws.com' AND eventname = 'GetFederationToken'))
ORDER BY eventtime;
Stage 8 · Data accessWhat was read — object-level S3 data events
Why: the only record of an object read is an S3 data event, and the trail only has them if its event selectors included S3 objects before the incident.
If they were off, write that down — “no evidence of exfiltration” and “no logs that could show exfiltration” are different statements — and scope on what the principal could read.
GuardDuty S3 Protection (Exfiltration:S3/AnomalousBehavior) and bucket server-access logs are the only other sources.
-- requires S3 data events on the trail (eventCategory = Data); bytesTransferredOut is in additionaleventdata
SELECT useridentity.arn AS actor, sourceipaddress, useragent,
json_extract_scalar(requestparameters, '$.bucketName') AS bucket,
count(*) AS reads,
sum(cast(json_extract_scalar(additionaleventdata, '$.bytesTransferredOut') AS bigint)) AS bytes_out,
min(eventtime) AS first_read, max(eventtime) AS last_read
FROM cloudtrail_logs
WHERE eventsource = 's3.amazonaws.com'
AND eventname IN ('GetObject','HeadObject','ListObjects','ListObjectsV2','CopyObject','SelectObjectContent','GetObjectTorrent')
AND eventtime > '<window-start ISO8601>'
AND (useridentity.accesskeyid = '<AKIA... or ASIA...>' OR sourceipaddress IN ('<attacker-ip>'))
GROUP BY 1, 2, 3, 4
ORDER BY bytes_out DESC;
Stage 8 · Data accessBytes out of a compromised instance — VPC flow logs
Why: for a workload foothold the control plane does not show the exfiltration or the C2; the ENI's flow logs do, as 5-tuples with byte counts.
They are off by default and the default format has no direction flag, so read from the instance's point of view.
GuardDuty reads flow logs independently of whether you enabled them, so its Backdoor:EC2/C&CActivity.B, CryptoCurrency:EC2/BitcoinTool.B and Trojan:EC2/DNSDataExfiltration findings exist even when your own flow logs don't.
-- table from the VPC flow logs Athena DDL (default v2 fields); "end" is reserved, hence the quotes
SELECT dstaddr, dstport, protocol, sum(bytes) AS bytes, count(*) AS flows,
from_unixtime(min(start)) AS first_seen, from_unixtime(max("end")) AS last_seen
FROM vpc_flow_logs
WHERE interface_id = '<eni-of-compromised-instance>' AND action = 'ACCEPT'
AND start > to_unixtime(timestamp '<window-start, e.g. 2026-09-20 00:00:00>')
GROUP BY 1, 2, 3
ORDER BY bytes DESC
LIMIT 50;
Stage 9 · ImpactDeletion, re-encryption, lifecycle wipes, backup and key destruction
SSEApplied = SSE_C in the data event), a KMS key scheduled for deletion (7–30 day window, nothing readable after), snapshots and recovery points deleted, and CloseAccount or LeaveOrganization to take the account out of your reach.A DeleteObject whose requestparameters carries a versionId is a permanent delete of that version, not a delete marker. DeleteObjects and SSE-C writes are data events.
SELECT eventtime, awsregion, eventsource, eventname, useridentity.arn AS actor, sourceipaddress,
json_extract_scalar(additionaleventdata, '$.SSEApplied') AS sse, requestparameters
FROM cloudtrail_logs
WHERE eventtime > '<window-start ISO8601>' AND errorcode IS NULL AND (
(eventsource = 's3.amazonaws.com' AND eventname IN ('DeleteBucket','PutBucketLifecycle','PutBucketReplication','DeleteBucketReplication',
'PutBucketVersioning','PutBucketEncryption','PutObjectLockConfiguration'))
OR (eventsource = 's3.amazonaws.com' AND eventname IN ('DeleteObjects','DeleteObject','PutObject','CopyObject') -- data events
AND (json_extract_scalar(additionaleventdata, '$.SSEApplied') = 'SSE_C' OR eventname LIKE 'Delete%'))
OR (eventsource = 'ec2.amazonaws.com' AND eventname IN ('DeleteSnapshot','DeregisterImage','DeleteVolume','TerminateInstances'))
OR (eventsource = 'rds.amazonaws.com' AND eventname IN ('DeleteDBSnapshot','DeleteDBClusterSnapshot','DeleteDBInstance','DeleteDBCluster'))
OR (eventsource = 'backup.amazonaws.com' AND eventname IN ('DeleteBackupVault','DeleteRecoveryPoint','DeleteBackupPlan','DeleteBackupSelection',
'PutBackupVaultAccessPolicy','DeleteBackupVaultLockConfiguration'))
OR (eventsource = 'kms.amazonaws.com' AND eventname IN ('ScheduleKeyDeletion','DisableKey','DeleteAlias','DeleteImportedKeyMaterial','PutKeyPolicy'))
OR (eventsource = 'dynamodb.amazonaws.com' AND eventname IN ('DeleteTable','DeleteBackup','UpdateContinuousBackups'))
OR (eventsource = 'glacier.amazonaws.com' AND eventname IN ('DeleteVault','DeleteArchive'))
OR (eventsource = 'organizations.amazonaws.com' AND eventname IN ('CloseAccount','LeaveOrganization','RemoveAccountFromOrganization')))
ORDER BY eventtime;
aws support describe-cases --include-resolved-cases, Business plan or higher), or through the provider's abuse channel; the ransom-note object itself is a stage 9 PutObject that needs S3 data events to be visible. The mail tenant and leak-site monitoring are the sources.Act
Preserve first, then cut broad, then surgical once the scope is known. Containment rewrites IAM, so the T0 dump comes before any of it; memory comes before isolation; compromised workloads are rebuilt from infrastructure-as-code, not cleaned.
T+0–15 · PreserveValidate the trail and dump the T0 state before you change anything
Why: every revoke and quarantine below rewrites IAM; the authorization-details dump and the credential report are the only record of what the attacker's principals held, and which keys existed, before you cut them.
validate-logs needs log-file validation enabled on the trail and proves the files in S3 are the ones CloudTrail delivered.
The hash list is the chain of custody.
aws cloudtrail validate-logs --region <trail-home-region> --trail-arn <org-trail-arn> --start-time <window-start>
aws iam get-account-authorization-details > t0-iam-$(date -u +%Y%m%dT%H%MZ).json
aws iam generate-credential-report && sleep 5 && \
aws iam get-credential-report --query Content --output text | base64 -d > t0-credential-report.csv
aws organizations list-accounts > t0-accounts.json # from the management or a delegated-admin account
aws sso-admin list-instances > t0-identity-center.json # then list-permission-sets / list-account-assignments per instance
sha256sum t0-* | tee t0-SHA256SUMS
aws s3 cp . s3://<evidence-bucket>/<case>/t0/ --recursive --exclude '*' --include 't0-*' # bucket with Object Lock, in the evidence account
T+15–30 · QuarantineMove the account into the quarantine OU
Why: the OU's deny-all SCP stops every credential in the account at once, including the ones you haven't found; only your IR role is exempt.
Three gaps to know — SCPs never apply to the management account or to service-linked roles, and they don't stop principals in other organisations using access a resource policy already granted them (that needs a resource control policy or the resource's own policy).
The OU and SCP must exist before the incident; building them now takes longer than the attacker needs.
aws organizations move-account --account-id <id> \
--source-parent-id <current-ou> --destination-parent-id <quarantine-ou>
aws organizations list-policies-for-target --target-id <id> --filter SERVICE_CONTROL_POLICY --output table # confirm it applied
T+15–30 · RevokeDeactivate the user's key, deny the user, and find the roles it assumed
Why: an inactive key stops new calls, and AWSDenyAll on the user denies the user's own sessions (console, GetSessionToken) on each call.
It does not touch sessions of roles the user assumed — those carry the role's permissions, not the user's, and keep working until the role itself is revoked (next step).
Deleting the login profile removes the console password.
aws iam update-access-key --user-name <user> --access-key-id <AKIA...> --status Inactive
aws iam attach-user-policy --user-name <user> --policy-arn arn:aws:iam::aws:policy/AWSDenyAll
aws iam delete-login-profile --user-name <user> # NoSuchEntity means there was no console password
# every role this user assumed in the window needs its own session revoke (next step)
aws cloudtrail lookup-events --region us-east-1 --start-time <window-start> \
--lookup-attributes AttributeKey=Username,AttributeValue=<user> --output json \
| jq -r '.Events[] | select(.EventName == "AssumeRole") | .CloudTrailEvent | fromjson | .requestParameters.roleArn' | sort -u
T+15–30 · RevokeRevoke the role's active sessions — and expect shared workloads to break
Why: the inline deny on aws:TokenIssueTime rejects every session minted before the timestamp, including the attacker's, on each call.
It also rejects every legitimate session of that role: instance profiles, Lambda, EKS pods and pipelines using it fail until they fetch a new session, which SDKs only do at expiry — restart them.
New sessions still work until the trust policy is fixed.
The console's “Revoke active sessions” writes exactly this policy.
NOW=$(date -u +%Y-%m-%dT%H:%M:%SZ)
cat > revoke-older-sessions.json <<JSON
{"Version":"2012-10-17","Statement":[{"Effect":"Deny","Action":"*","Resource":"*",
"Condition":{"DateLessThan":{"aws:TokenIssueTime":"$NOW"}}}]}
JSON
aws iam put-role-policy --role-name <role> --policy-name AWSRevokeOlderSessions --policy-document file://revoke-older-sessions.json
aws iam get-role --role-name <role> --query Role.AssumeRolePolicyDocument # then narrow the trust before anyone re-assumes it
T+15–30 · RevokeRoot compromised: there is no session revoke — replace the credential
Why: nothing revokes a root console session; it lives until it expires.
The containment is a new password and a new MFA device, then watching for userIdentity.type = 'Root' after that time.
With centralised root access enabled on the organisation, the management account can do it without the root password by assuming root with the IAMDeleteRootUserCredentials task policy (15-minute session) and deleting the login profile and MFA.
Otherwise it is the root email's recovery flow — and if the attacker changed the email, AWS Support with proof of ownership.
# from the management account, if enable-organizations-root-sessions was done in peacetime
aws sts assume-root --target-principal <member-account-id> --duration-seconds 900 \
--task-policy-arn arn=arn:aws:iam::aws:policy/root-task/IAMDeleteRootUserCredentials
# with those credentials:
aws iam delete-login-profile
aws iam list-mfa-devices && aws iam deactivate-mfa-device --serial-number <arn>
aws iam list-access-keys && aws iam delete-access-key --access-key-id <AKIA...>
# then, as root of the member account, set a new password and MFA; verify who is root now:
aws account get-contact-information && aws account get-alternate-contact --alternate-contact-type SECURITY
T+30–60 · IsolateMemory, then snapshot, then isolate — in that order, and don't terminate
Why: the isolation security group cuts SSM too unless the VPC has interface endpoints for ssm, ssmmessages and ec2messages and the isolation group allows 443 to them, so take memory while the agent can still talk.
AVML writes LiME format; stage it in the evidence bucket (write-only for the instance role) rather than fetching it from the internet.
Snapshots encrypted with the default aws/ebs key cannot be shared to another account: copy to a customer-managed key whose policy includes the evidence account, then share.
Security-group changes don't cut connections already tracked — add a subnet NACL deny if the cut must be immediate.
Keep the instance profile attached until the memory upload has finished; credentials already fetched from the metadata service stay valid until the role's sessions are revoked.
# 1. memory, while the network and the instance role are still there (Windows: AWS-RunPowerShellScript with a memory tool staged the same way)
aws ssm send-command --region <region> --instance-ids <i-id> --document-name AWS-RunShellScript --comment "IR <case> memory" \
--parameters 'commands=["aws s3 cp s3://<evidence-bucket>/tools/avml /tmp/avml","chmod +x /tmp/avml","/tmp/avml --compress /tmp/<i-id>.lime.compressed","aws s3 cp /tmp/<i-id>.lime.compressed s3://<evidence-bucket>/<case>/","sha256sum /tmp/<i-id>.lime.compressed"]'
# 2. disk
aws ec2 create-snapshot --volume-id <vol-id> --description "IR <case> <i-id>" \
--tag-specifications 'ResourceType=snapshot,Tags=[{Key=ir-case,Value=<case>}]'
aws ec2 copy-snapshot --source-region <region> --source-snapshot-id <snap-id> --encrypted --kms-key-id <evidence-cmk-arn> # aws/ebs-encrypted snapshots cannot be shared
aws ec2 modify-snapshot-attribute --snapshot-id <copied-snap-id> --attribute createVolumePermission --operation-type add --user-ids <evidence-account>
# 3. isolate
aws ec2 modify-instance-attribute --instance-id <i-id> --groups <isolation-sg>
aws ec2 modify-instance-attribute --instance-id <i-id> --disable-api-termination
aws ec2 disassociate-iam-instance-profile --association-id <iip-assoc-id>
KQL in Sentinel or Log Analytics.
- Control plane:
AzureActivity(write operations only — ARM reads are never logged; 90 days in the portal, longer only through a diagnostic setting into a workspace). - Entra ID:
SigninLogs,AADNonInteractiveUserSignInLogs,AADServicePrincipalSignInLogs,AADManagedIdentitySignInLogs,AuditLogs— 30 days on P1/P2, 7 days on the free tier, longer only in a workspace.
MicrosoftGraphActivityLogs, AZKVAuditLogs, StorageBlobLogs and NTANetAnalytics each need their own diagnostic setting and are off by default.
SecurityAlert carries Defender for Cloud.
Current state comes from Resource Graph (az graph query, extension resource-graph) and az.
Hunt
Stage 1 · Initial accessUsers, service principals and managed identities signing in from new places
let window = 7d; let baseline = 30d;
let signins = union isfuzzy=true
(SigninLogs | extend Principal = UserPrincipalName, Kind = "user"),
(AADNonInteractiveUserSignInLogs | extend Principal = UserPrincipalName, Kind = "user-noninteractive"),
(AADServicePrincipalSignInLogs | extend Principal = ServicePrincipalName, Kind = "service-principal"),
(AADManagedIdentitySignInLogs | extend Principal = ServicePrincipalName, Kind = "managed-identity")
| where TimeGenerated > ago(baseline)
| project TimeGenerated, Principal, Kind, IPAddress, ResultType, ResourceDisplayName;
let known = signins | where TimeGenerated between (ago(baseline) .. ago(window)) | distinct Principal, IPAddress;
signins
| where TimeGenerated > ago(window)
| join kind=leftanti known on Principal, IPAddress
| summarize FirstSeen = min(TimeGenerated), Calls = count(), Failures = countif(ResultType != "0"),
Resources = make_set(ResourceDisplayName, 5) by Principal, Kind, IPAddress
| order by FirstSeen desc
Stage 2 · DiscoveryGraph and ARM enumeration — AzureHound, ROADtools and friends
MicrosoftGraphActivityLogs (needs the Entra diagnostic setting) and in the sign-in tables as a token for Graph or ARM with a scripting user agent. Two hundred GETs against users, groups, service principals and role assignments in an hour from one token is a collector.Graph activity logs off? Fall back to union SigninLogs, AADNonInteractiveUserSignInLogs | where ResourceDisplayName in ("Microsoft Graph", "Windows Azure Service Management API") | where UserAgent has_any ("python", "azurehound", "roadtx", "go-http") — you see the token being obtained, not what it fetched.
MicrosoftGraphActivityLogs
| where TimeGenerated > ago(14d)
| where RequestMethod == "GET"
| where UserAgent has_any ("python-requests", "azurehound", "roadtx", "roadrecon", "AADInternals", "GraphRunner", "go-http-client")
or RequestUri has_any ("/directoryRoles", "/roleManagement/directory/roleAssignments", "/servicePrincipals",
"/applications", "/users?$top=999", "/groups?$top=999", "/devices?$top=999")
| summarize Calls = count(), DistinctUris = dcount(RequestUri), Status = make_set(ResponseStatusCode, 5),
First = min(TimeGenerated), Last = max(TimeGenerated)
by Who = coalesce(UserId, ServicePrincipalId), AppId, IPAddress, UserAgent
| where Calls > 200 or UserAgent has_any ("azurehound", "roadtx", "roadrecon", "AADInternals", "GraphRunner")
| order by Calls desc
Stage 3 · Privilege escalationElevate access, role assignments, custom roles and PIM requests
AuditLogs under the service “Azure RBAC (Elevated Access)”, not in AzureActivity. Owner or User Access Administrator written to a new principal, a custom role definition with *, or a PIM eligibility request are the Azure-side versions.Current state, independent of log retention: az graph query -q "authorizationresources | where type == 'microsoft.authorization/roleassignments' | extend p = properties | where todatetime(p.createdOn) > ago(30d) | project p.createdOn, p.principalId, p.principalType, p.roleDefinitionId, p.scope". Owner is 8e3af657-a8ff-443c-a75c-2fe8c4bcb635, User Access Administrator 18d7d88d-d35e-4fb5-a5c3-7773c20a72d9.
let elevate = AuditLogs
| where TimeGenerated > ago(30d)
| where LoggedByService == "Azure RBAC (Elevated Access)" or OperationName has "elevated their access"
| project TimeGenerated, Source = "Entra", Operation = OperationName,
Who = tostring(InitiatedBy.user.userPrincipalName), IP = tostring(InitiatedBy.user.ipAddress), Detail = Result;
let rbac = AzureActivity
| where TimeGenerated > ago(30d)
| where ActivityStatusValue =~ "Success"
| where OperationNameValue in~ ("MICROSOFT.AUTHORIZATION/ROLEASSIGNMENTS/WRITE", "MICROSOFT.AUTHORIZATION/ROLEDEFINITIONS/WRITE",
"MICROSOFT.AUTHORIZATION/ROLEELIGIBILITYSCHEDULEREQUESTS/WRITE",
"MICROSOFT.AUTHORIZATION/ROLEASSIGNMENTSCHEDULEREQUESTS/WRITE")
| extend body = parse_json(tostring(parse_json(Properties).requestbody))
| extend Detail = strcat(tostring(body.Properties.PrincipalType), " ", tostring(body.Properties.PrincipalId),
" -> ", tostring(body.Properties.RoleDefinitionId), " @ ", tostring(body.Properties.Scope))
| project TimeGenerated, Source = "ARM", Operation = OperationNameValue, Who = Caller, IP = CallerIpAddress, Detail;
elevate | union rbac | order by TimeGenerated desc
Stage 4 · PersistenceCredentials, federated credentials, owners, roles, CA policy; extensions, run commands, runbooks, functions, subscriptions
FederatedIdentityCredentials in the modified properties, which is why naive filters miss it. On the Azure side the same goal is a CustomScript extension or run command on every VM, an Automation runbook on a schedule, a Logic App or function app, a federated credential on a user-assigned managed identity, or a new subscription created on your billing account.MICROSOFT.SUBSCRIPTION/ALIASES/WRITE is logged at tenant scope and only lands in a workspace if the tenant-level Activity Log is exported; the list of subscriptions in Resource Graph (resourcecontainers | where type == 'microsoft.resources/subscriptions') is the reliable check.
let entra = AuditLogs
| where TimeGenerated > ago(30d)
| where OperationName in ("Add service principal credentials", "Update application – Certificates and secrets management",
"Add owner to application", "Add owner to service principal", "Add app role assignment to service principal",
"Consent to application", "Add member to role", "Add eligible member to role",
"Update conditional access policy", "Delete conditional access policy", "Add user",
"User registered security info", "Add partner to company")
or (OperationName == "Update application" and tostring(TargetResources[0].modifiedProperties) has "FederatedIdentityCredentials")
| project TimeGenerated, Source = "Entra", Operation = OperationName,
Who = coalesce(tostring(InitiatedBy.user.userPrincipalName), tostring(InitiatedBy.app.displayName)),
IP = tostring(InitiatedBy.user.ipAddress), Target = tostring(TargetResources[0].displayName), Detail = Result;
let arm = AzureActivity
| where TimeGenerated > ago(30d)
| where ActivityStatusValue =~ "Success"
| where OperationNameValue in~ ("MICROSOFT.COMPUTE/VIRTUALMACHINES/EXTENSIONS/WRITE", "MICROSOFT.COMPUTE/VIRTUALMACHINES/RUNCOMMAND/ACTION",
"MICROSOFT.COMPUTE/VIRTUALMACHINES/RUNCOMMANDS/WRITE", "MICROSOFT.COMPUTE/VIRTUALMACHINESCALESETS/EXTENSIONS/WRITE",
"MICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/RUNBOOKS/WRITE", "MICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/RUNBOOKS/DRAFT/CONTENT/WRITE",
"MICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/JOBSCHEDULES/WRITE", "MICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/WEBHOOKS/WRITE",
"MICROSOFT.LOGIC/WORKFLOWS/WRITE", "MICROSOFT.WEB/SITES/WRITE", "MICROSOFT.WEB/SITES/CONFIG/WRITE",
"MICROSOFT.MANAGEDIDENTITY/USERASSIGNEDIDENTITIES/WRITE",
"MICROSOFT.MANAGEDIDENTITY/USERASSIGNEDIDENTITIES/FEDERATEDIDENTITYCREDENTIALS/WRITE",
"MICROSOFT.KEYVAULT/VAULTS/ACCESSPOLICIES/WRITE", "MICROSOFT.SUBSCRIPTION/ALIASES/WRITE")
| project TimeGenerated, Source = "ARM", Operation = OperationNameValue, Who = Caller, IP = CallerIpAddress, Target = ResourceId, Detail = ActivityStatusValue;
entra | union arm | order by TimeGenerated desc
Stage 5 · Defence evasionDiagnostic settings, workspaces, Defender plans, Sentinel connectors, policy and locks removed
az rest against /providers/Microsoft.Insights/eventtypes/management/values), not in AzureActivity.AzureActivity
| where TimeGenerated > ago(30d)
| where ActivityStatusValue =~ "Success"
| where OperationNameValue in~ ("MICROSOFT.INSIGHTS/DIAGNOSTICSETTINGS/DELETE", "MICROSOFT.INSIGHTS/DIAGNOSTICSETTINGS/WRITE",
"MICROSOFT.OPERATIONALINSIGHTS/WORKSPACES/DELETE", "MICROSOFT.OPERATIONALINSIGHTS/WORKSPACES/DATAEXPORTS/DELETE",
"MICROSOFT.OPERATIONALINSIGHTS/WORKSPACES/TABLES/WRITE",
"MICROSOFT.SECURITY/PRICINGS/WRITE", "MICROSOFT.SECURITY/SETTINGS/WRITE", "MICROSOFT.SECURITY/AUTOMATIONS/DELETE",
"MICROSOFT.SECURITYINSIGHTS/DATACONNECTORS/DELETE", "MICROSOFT.SECURITYINSIGHTS/ALERTRULES/DELETE",
"MICROSOFT.AUTHORIZATION/POLICYASSIGNMENTS/DELETE", "MICROSOFT.AUTHORIZATION/POLICYEXEMPTIONS/WRITE",
"MICROSOFT.AUTHORIZATION/LOCKS/DELETE", "MICROSOFT.NETWORK/NETWORKWATCHERS/FLOWLOGS/DELETE",
"MICROSOFT.EVENTHUB/NAMESPACES/EVENTHUBS/DELETE")
| project TimeGenerated, OperationNameValue, Caller, CallerIpAddress, ResourceId, SubscriptionId
| order by TimeGenerated desc
Stage 5 · Defence evasionDefender for Cloud alerts nobody read
SecurityAlert
| where TimeGenerated > ago(14d)
| where ProductName in ("Azure Security Center", "Microsoft Defender for Cloud", "Azure Active Directory Identity Protection", "Microsoft Defender for Identity")
| project TimeGenerated, AlertName, AlertSeverity, CompromisedEntity, Tactics, Description
| order by TimeGenerated desc
Stage 6 · Lateral movementA principal reaching subscriptions it never touched; Lighthouse, management groups, tenant moves
Why: Azure lateral movement is the same identity acting in a subscription it has no history in, a role assignment at management-group scope that cascades everywhere, a Lighthouse registration that delegates your subscription to another tenant, or a subscription accepted into or out of the tenant.
Elevate access (stage 3) is the tenant-wide version.
The Entra side is a B2B invite or Add partner to company (GDAP).
Delegations in force now: az managedservices assignment list --include-definition for Lighthouse, Get-MgTenantRelationshipDelegatedAdminRelationship for GDAP partners.
let baseline = AzureActivity | where TimeGenerated between (ago(30d) .. ago(7d)) | distinct Caller, SubscriptionId;
AzureActivity
| where TimeGenerated > ago(7d)
| join kind=leftanti baseline on Caller, SubscriptionId
| summarize First = min(TimeGenerated), Ops = count(), Operations = make_set(OperationNameValue, 8),
IPs = make_set(CallerIpAddress, 5) by Caller, SubscriptionId
| union (
AzureActivity
| where TimeGenerated > ago(7d)
| where OperationNameValue in~ ("MICROSOFT.MANAGEDSERVICES/REGISTRATIONASSIGNMENTS/WRITE",
"MICROSOFT.MANAGEDSERVICES/REGISTRATIONDEFINITIONS/WRITE",
"MICROSOFT.MANAGEMENT/MANAGEMENTGROUPS/SUBSCRIPTIONS/WRITE",
"MICROSOFT.SUBSCRIPTION/ACCEPTCHANGETENANT/ACTION")
| project First = TimeGenerated, Caller, SubscriptionId, Ops = 1,
Operations = pack_array(OperationNameValue), IPs = pack_array(CallerIpAddress))
| order by First desc
Stage 7 · Resource abuseVMs, scale sets, AKS, ML compute, Batch, Azure OpenAI deployments, quota tickets
AzureActivity
| where TimeGenerated > ago(14d)
| where ActivityStatusValue =~ "Success"
| where OperationNameValue in~ ("MICROSOFT.COMPUTE/VIRTUALMACHINES/WRITE", "MICROSOFT.COMPUTE/VIRTUALMACHINESCALESETS/WRITE",
"MICROSOFT.CONTAINERSERVICE/MANAGEDCLUSTERS/WRITE", "MICROSOFT.CONTAINERSERVICE/MANAGEDCLUSTERS/AGENTPOOLS/WRITE",
"MICROSOFT.MACHINELEARNINGSERVICES/WORKSPACES/COMPUTES/WRITE", "MICROSOFT.BATCH/BATCHACCOUNTS/POOLS/WRITE",
"MICROSOFT.CONTAINERINSTANCE/CONTAINERGROUPS/WRITE", "MICROSOFT.COGNITIVESERVICES/ACCOUNTS/WRITE",
"MICROSOFT.COGNITIVESERVICES/ACCOUNTS/DEPLOYMENTS/WRITE", "MICROSOFT.SUPPORT/SUPPORTTICKETS/WRITE",
"MICROSOFT.QUOTA/QUOTAS/WRITE", "MICROSOFT.SUBSCRIPTION/ALIASES/WRITE")
| summarize Writes = count(), First = min(TimeGenerated), Last = max(TimeGenerated),
Resources = make_set(ResourceId, 10) by Caller, CallerIpAddress, OperationNameValue, SubscriptionId
| order by Writes desc
Stage 7 · Resource abuseWhat is running now — GPU and HPC SKUs and anything created this week
properties.timeCreated dates the launch. Cost Management by resource location and meter for the same window is the figure for the billing case.az graph query --first 1000 -o table -q "
resources
| where type in~ ('microsoft.compute/virtualmachines', 'microsoft.compute/virtualmachinescalesets',
'microsoft.machinelearningservices/workspaces/computes', 'microsoft.cognitiveservices/accounts')
| extend size = tostring(coalesce(properties.hardwareProfile.vmSize, sku.name)),
created = todatetime(properties.timeCreated)
| where size matches regex '^Standard_(N|H)' or created > ago(7d)
| project created, name, type, location, size, subscriptionId, resourceGroup
| order by created desc"
Stage 8 · Data accessStorage keys and SAS listed, vault and firewall opened, disks exported, commands run
beginGetAccess on a disk or snapshot hands out a SAS to download the whole VHD; a storage-account or Key Vault write that sets defaultAction: Allow or allowBlobPublicAccess opens the door from the internet; config/list on a web app dumps its app settings; Run Command executes as SYSTEM or root without a login.AzureActivity
| where TimeGenerated > ago(14d)
| where ActivityStatusValue =~ "Success"
| where OperationNameValue in~ ("MICROSOFT.STORAGE/STORAGEACCOUNTS/LISTKEYS/ACTION", "MICROSOFT.STORAGE/STORAGEACCOUNTS/LISTACCOUNTSAS/ACTION",
"MICROSOFT.STORAGE/STORAGEACCOUNTS/LISTSERVICESAS/ACTION", "MICROSOFT.STORAGE/STORAGEACCOUNTS/REGENERATEKEY/ACTION",
"MICROSOFT.STORAGE/STORAGEACCOUNTS/WRITE", "MICROSOFT.STORAGE/STORAGEACCOUNTS/BLOBSERVICES/CONTAINERS/WRITE",
"MICROSOFT.COMPUTE/SNAPSHOTS/BEGINGETACCESS/ACTION", "MICROSOFT.COMPUTE/DISKS/BEGINGETACCESS/ACTION",
"MICROSOFT.COMPUTE/VIRTUALMACHINES/RUNCOMMAND/ACTION", "MICROSOFT.COMPUTE/VIRTUALMACHINES/RUNCOMMANDS/WRITE",
"MICROSOFT.DOCUMENTDB/DATABASEACCOUNTS/LISTKEYS/ACTION", "MICROSOFT.DOCUMENTDB/DATABASEACCOUNTS/LISTCONNECTIONSTRINGS/ACTION",
"MICROSOFT.SQL/SERVERS/FIREWALLRULES/WRITE", "MICROSOFT.SQL/SERVERS/DATABASES/EXPORT/ACTION",
"MICROSOFT.KEYVAULT/VAULTS/WRITE", "MICROSOFT.KEYVAULT/VAULTS/ACCESSPOLICIES/WRITE",
"MICROSOFT.WEB/SITES/CONFIG/LIST/ACTION", "MICROSOFT.WEB/SITES/PUBLISHXML/ACTION")
| project TimeGenerated, OperationNameValue, Caller, CallerIpAddress, ResourceId
| order by TimeGenerated desc
Stage 8 · Data accessWhat was read — Key Vault and blob data-plane logs
Why: AZKVAuditLogs shows every SecretGet, KeyDecrypt and SecretList with the caller's claims; StorageBlobLogs shows every GetBlob with the auth type — a SAS or account-key read carries no identity, only an AuthenticationHash and an IP.
Both need the resource diagnostic setting (AuditEvent; StorageRead) and are off by default; if they were off, the notification is scoped on what the principal could read, and the status line says so.
A Defender for Storage alert on mass extraction or SAS use from an unusual IP is often the first sign; record its user agent, auth type and source IP.
Re-read the alert at intervals and log the extracted size and file count each time, because they keep growing after the alert first fires.
let kv = AZKVAuditLogs
| where TimeGenerated > ago(14d)
| where OperationName in ("SecretGet", "SecretList", "SecretBackup", "KeyDecrypt", "KeyUnwrap", "KeySign", "KeyBackup", "CertificateGet",
"VaultAccessPolicyChangedEventGridNotification")
| extend Source = "KeyVault",
Who = tostring(coalesce(Identity.claim["http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn"], Identity.claim.appid, Identity.claim.oid))
| summarize Calls = count(), Ops = make_set(OperationName, 10), Objects = dcount(Id), First = min(TimeGenerated), Last = max(TimeGenerated)
by Source, Who, CallerIpAddress;
let blob = StorageBlobLogs
| where TimeGenerated > ago(14d)
| where OperationName in ("GetBlob", "ListBlobs", "CopyBlob", "GetBlobProperties", "ListContainers")
| extend Source = "Blob", Who = strcat(AuthenticationType, ":", coalesce(RequesterObjectId, RequesterAppId, AuthenticationHash))
| summarize Calls = count(), Ops = make_set(OperationName, 10), Objects = dcount(ObjectKey), First = min(TimeGenerated), Last = max(TimeGenerated)
by Source, Who, CallerIpAddress;
kv | union blob | order by Calls desc
Stage 8 · Data accessBytes out of a compromised VM — VNet flow logs
NTANetAnalytics
| where TimeGenerated > ago(7d)
| where SubType == "FlowLog"
| where SrcIp == "<vm-private-ip>"
| where FlowType in ("ExternalPublic", "MaliciousFlow", "AzurePublic")
| summarize Bytes = sum(BytesSrcToDest + BytesDestToSrc), Flows = count(), First = min(TimeGenerated), Last = max(TimeGenerated)
by DestIp, DestPublicIps, DestPort, L4Protocol, FlowType
| order by Bytes desc
| take 50
Stage 9 · ImpactBackups, snapshots, vaults, keys and locks deleted; soft delete switched off; mass blob deletes
backupconfig/write turning soft delete off, then protected items and recovery points deleted, a Resource Guard removed to defeat multi-user authorisation, snapshots and disks deleted, a vault purged past its soft-delete window, an encryption scope rewritten to an attacker-controlled key, and a resource lock deleted a minute before each of them. Blob deletes are data plane and only visible if StorageDelete logging was on.let arm = AzureActivity
| where TimeGenerated > ago(14d)
| where ActivityStatusValue =~ "Success"
| where OperationNameValue in~ ("MICROSOFT.RECOVERYSERVICES/VAULTS/DELETE", "MICROSOFT.RECOVERYSERVICES/VAULTS/BACKUPCONFIG/WRITE",
"MICROSOFT.RECOVERYSERVICES/VAULTS/BACKUPFABRICS/PROTECTIONCONTAINERS/PROTECTEDITEMS/DELETE",
"MICROSOFT.RECOVERYSERVICES/VAULTS/BACKUPFABRICS/PROTECTIONCONTAINERS/PROTECTEDITEMS/RECOVERYPOINTS/DELETE",
"MICROSOFT.DATAPROTECTION/BACKUPVAULTS/DELETE", "MICROSOFT.DATAPROTECTION/BACKUPVAULTS/BACKUPINSTANCES/DELETE",
"MICROSOFT.DATAPROTECTION/RESOURCEGUARDS/DELETE",
"MICROSOFT.COMPUTE/SNAPSHOTS/DELETE", "MICROSOFT.COMPUTE/DISKS/DELETE", "MICROSOFT.COMPUTE/VIRTUALMACHINES/DELETE",
"MICROSOFT.COMPUTE/RESTOREPOINTCOLLECTIONS/DELETE",
"MICROSOFT.STORAGE/STORAGEACCOUNTS/DELETE", "MICROSOFT.STORAGE/STORAGEACCOUNTS/BLOBSERVICES/CONTAINERS/DELETE",
"MICROSOFT.STORAGE/STORAGEACCOUNTS/BLOBSERVICES/WRITE", "MICROSOFT.STORAGE/STORAGEACCOUNTS/MANAGEMENTPOLICIES/WRITE",
"MICROSOFT.STORAGE/STORAGEACCOUNTS/ENCRYPTIONSCOPES/WRITE",
"MICROSOFT.KEYVAULT/VAULTS/DELETE", "MICROSOFT.KEYVAULT/MANAGEDHSMS/DELETE", "MICROSOFT.KEYVAULT/LOCATIONS/DELETEDVAULTS/PURGE/ACTION",
"MICROSOFT.SQL/SERVERS/DATABASES/DELETE", "MICROSOFT.SQL/SERVERS/DELETE", "MICROSOFT.DBFORPOSTGRESQL/FLEXIBLESERVERS/DELETE",
"MICROSOFT.AUTHORIZATION/LOCKS/DELETE", "MICROSOFT.RESOURCES/SUBSCRIPTIONS/RESOURCEGROUPS/DELETE",
"MICROSOFT.RESOURCES/DEPLOYMENTSTACKS/DELETE")
| project TimeGenerated, Source = "ARM", Op = OperationNameValue, Who = Caller, IP = CallerIpAddress, Target = ResourceId;
let blob = StorageBlobLogs
| where TimeGenerated > ago(14d)
| where OperationName in ("DeleteBlob", "DeleteContainer", "SetBlobTier", "PutBlob", "CopyBlob")
| summarize Count = count(), First = min(TimeGenerated)
by bin(TimeGenerated, 1h), OperationName, AuthenticationType, Who = coalesce(RequesterObjectId, RequesterAppId, AuthenticationHash),
IP = CallerIpAddress, AccountName
| where Count > 500
| project TimeGenerated = First, Source = "Blob", Op = strcat(OperationName, " x", Count), Who, IP, Target = AccountName;
arm | union blob | order by TimeGenerated desc
az support tickets list, extension support); the ransom-note blob itself is a stage 9 PutBlob that is only logged with StorageWrite diagnostics on. The mail tenant and leak-site monitoring are the sources.Act
Preserve first, then cut broad, then surgical once the scope is known.
Azure has no ad-hoc deny-all, so the wide cut is disabling the identities and stripping what they hold at subscription and management-group scope; a denyAction policy pins resources against deletion while you work.
Memory before isolation; compromised workloads are rebuilt from infrastructure-as-code, not cleaned.
T+0–15 · PreserveDump the T0 state and export the Entra logs before you change anything
Why: role assignments, app credentials and directory-role membership are what you are about to rewrite; Resource Graph and Graph give the authoritative “before” in minutes.
The sign-in and audit exports matter because the tables keep 30 days on P1/P2 and 7 on the free tier, and the Activity Log 90 — all shorter than the investigation.
The hash list is the chain of custody.
az graph query --first 1000 -o json -q "authorizationresources | where type == 'microsoft.authorization/roleassignments'
| extend p = properties | project createdOn = p.createdOn, principalId = p.principalId, principalType = p.principalType,
roleDefinitionId = p.roleDefinitionId, scope = p.scope" > t0-roleassignments.json
az ad app list --all -o json --query "[].{appId:appId,name:displayName,secrets:passwordCredentials[].{keyId:keyId,end:endDateTime},certs:keyCredentials[].{keyId:keyId,end:endDateTime}}" > t0-app-creds.json
az ad sp list --all -o json --query "[].{id:id,appId:appId,name:displayName,enabled:accountEnabled,type:servicePrincipalType}" > t0-service-principals.json
Get-MgDirectoryRole -All | ForEach-Object { $r = $_; Get-MgDirectoryRoleMember -DirectoryRoleId $_.Id -All |
Select-Object @{n='Role';e={$r.DisplayName}}, Id, @{n='Upn';e={$_.AdditionalProperties.userPrincipalName}} } | Export-Csv t0-directory-roles.csv
Get-MgAuditLogSignIn -All -Filter "createdDateTime ge <window-start>" | Export-Csv t0-signins.csv
Get-MgAuditLogDirectoryAudit -All -Filter "activityDateTime ge <window-start>" | Export-Csv t0-directory-audit.csv
shasum -a 256 t0-* | tee t0-SHA256SUMS # then copy the set into an immutable container in the evidence subscription
T+15–30 · QuarantineStrip the principal's role assignments at every scope, and pin resources against deletion
Why: removing the roles cuts what every credential of that principal can reach, including secrets you haven't found, and is evaluated on each call — the propagation takes minutes, which is faster than any token expiry.
Azure Policy deny only blocks creating or changing resources through ARM; denyAction blocks deletes; neither touches data-plane reads or Entra.
Deployment Stacks' denySettings protect only the stack's own resources and have excludedPrincipals for the IR identity.
Build the denyAction definition in peacetime.
az role assignment list --assignee <objectId> --all --include-inherited -o table
az role assignment delete --assignee <objectId> --scope <scope> # repeat per scope; management groups included
az policy assignment create -n ir-deny-delete --scope /subscriptions/<sub-id> --policy <denyAction-definition-id> # effect denyAction, actionNames [delete]
T+15–30 · RevokeDisable the service principal and delete every credential it has — on the SP and on the app
az ad app credential only removes credentials on the app registration; secrets added directly to the service principal object and federated credentials need their own commands, and an attacker who added a federated credential needs no secret at all.Update-MgServicePrincipal -ServicePrincipalId <sp-objectId> -AccountEnabled:$false
az ad sp credential list --id <sp-objectId> -o table
az ad sp credential delete --id <sp-objectId> --key-id <keyId> # add --cert for certificate credentials
az ad app credential list --id <appId> -o table
az ad app credential delete --id <appId> --key-id <keyId>
az ad app federated-credential list --id <appId> -o table
az ad app federated-credential delete --id <appId> --federated-credential-id <id>
T+15–30 · RevokeDisable the user and revoke their sessions — access tokens outlive both
Why: disabling stops new sign-ins; Revoke-MgUserSignInSession invalidates the refresh tokens and sessions already issued.
Access tokens already in hand stay valid until they expire, 60–90 minutes by default: Exchange, SharePoint, Teams and Graph enforce the revocation within minutes through continuous access evaluation, Azure Resource Manager does not, so the role strip is what actually ends the attacker's ARM access.
Reset the password as well; a password change alone revokes nothing.
Update-MgUser -UserId <upn> -AccountEnabled:$false
Revoke-MgUserSignInSession -UserId <upn>
az role assignment list --assignee <upn> --all --include-inherited -o table # then delete, as in Quarantine
T+15–30 · RevokeCompromised managed identity: nothing to rotate, so take its roles and its federated credentials
Why: a managed identity has no secret you can reset, its tokens are cached by the platform for around 24 hours, and there is no way to force a refresh.
Role assignments are evaluated on every call, so removing them is the revoke.
A system-assigned identity can simply be removed from the VM, which deletes it; a user-assigned one may carry federated credentials the attacker added.
az role assignment list --assignee <mi-principalId> --all --include-inherited -o table
az role assignment delete --assignee <mi-principalId> --scope <scope>
az vm identity remove -g <rg> -n <vm> # system-assigned: the identity is deleted outright
az identity federated-credential list -g <rg> --identity-name <uai> -o table
az identity federated-credential delete -g <rg> --identity-name <uai> -n <fic-name>
T+15–30 · RevokeInvalidate storage keys and user-delegation SAS
az storage account keys renew -g <rg> -n <account> --key primary
az storage account keys renew -g <rg> -n <account> --key secondary
az storage account revoke-delegation-keys -g <rg> -n <account>
T+30–60 · IsolateMemory, then snapshot, then cut the VM's network — don't delete it
Why: Run Command executes as root or SYSTEM over the VM agent, so take memory first with AVML staged in the evidence container and uploaded straight back to it with a write-only SAS.
The snapshot keeps the disk; grant-access gives a read SAS to copy it into the evidence subscription.
NSG rule changes apply to new connections only — existing flows are not re-evaluated — so a deny-all outbound rule does not cut a live C2 session; a route table with next hop None on 0.0.0.0/0 on the subnet does, for every VM in that subnet.
Never deallocate: it discards memory.
# 1. memory (Windows: --command-id RunPowerShellScript with a memory tool staged the same way)
az vm run-command invoke -g <rg> -n <vm> --command-id RunShellScript --scripts \
'curl -sSf -o /tmp/avml "<evidence-container-sas-url>/tools/avml" && chmod +x /tmp/avml && /tmp/avml --compress /tmp/mem.lime.compressed && sha256sum /tmp/mem.lime.compressed && curl -sSf -X PUT -H "x-ms-blob-type: BlockBlob" --upload-file /tmp/mem.lime.compressed "<evidence-container-sas-url>/<case>/<vm>.lime.compressed"'
# 2. disk
az snapshot create -g <rg> -n ir-<case>-<vm>-osdisk --source <os-disk-id>
az snapshot grant-access -g <rg> -n ir-<case>-<vm>-osdisk --duration-in-seconds 3600 --access-level Read --query accessSas -o tsv # then azcopy it into the evidence container
az resource lock create --lock-type CanNotDelete -n ir-<case> --resource <vm-id>
# 3. isolate
az network nsg rule create -g <rg> --nsg-name <nsg> -n ir-deny-out --priority 100 --direction Outbound --access Deny \
--protocol '*' --destination-address-prefixes '*' --destination-port-ranges '*'
az network route-table create -g <rg> -n ir-blackhole && \
az network route-table route create -g <rg> --route-table-name ir-blackhole -n all --address-prefix 0.0.0.0/0 --next-hop-type None && \
az network vnet subnet update -g <rg> --vnet-name <vnet> -n <subnet> --route-table ir-blackhole # cuts existing flows; whole subnet
gcloud logging read against Cloud Audit Logs, plus Cloud Asset Inventory for current state and IAM history.
Logs live with the resource that produced them: run each query per affected project (gcloud projects list and loop), or at --organization/--folder/--billing-account for the entries those resources produce themselves.
Admin Activity logs are always on and kept 400 days in the _Required bucket; Data Access logs (object reads, secret access, GenerateAccessToken, every Get/List) are off unless you enabled them per service — BigQuery is the one exception — and they land in _Default, which keeps 30 days unless you changed it.
VPC Flow Logs are per subnet and off by default.
Hunt
Stage 1 · Initial accessEverything one principal did, from where, with which client and which key
serviceAccountKeyName separate the attacker's use of a key from the workload's own: a workload calls from its own egress with an SDK string; the attacker calls from elsewhere with gcloud or python-requests, and the key name says which of the account's keys leaked.Keys and their state: gcloud iam service-accounts keys list --iam-account=<sa-email> --format='table(name.basename(),validAfterTime,disabled,keyType)'. If the organisation policy iam.serviceAccountKeyExposureResponse is DISABLE_KEY (the default since June 2024), Google disables a key it finds in a public repository and writes an audit-log event and an email to the project owners and security contacts; under WAIT_FOR_ABUSE it only notifies.
gcloud logging read \
'logName:"cloudaudit.googleapis.com" AND protoPayload.authenticationInfo.principalEmail="<principal-email>"' \
--project=<project-id> --freshness=14d --order=asc \
--format='table(timestamp,protoPayload.serviceName,protoPayload.methodName,protoPayload.requestMetadata.callerIp,protoPayload.requestMetadata.callerSuppliedUserAgent,protoPayload.authenticationInfo.serviceAccountKeyName,protoPayload.status.code)'
Stage 2 · DiscoveryEnumeration and permission probing — only visible with Data Access logs on
Get, List, TestIamPermissions and asset search is an ADMIN_READ or DATA_READ entry, which Google does not log unless Data Access audit logs were enabled for the service or for all services. If they were off, discovery is invisible and the status line says so; Security Command Center's Event Threat Detection still raises Discovery: Service Account Self-Investigation from its own feed.SCC findings for the window: gcloud scc findings list <org-id> --source=- --filter='category="Discovery: Service Account Self-Investigation" AND state="ACTIVE"' (Premium or Enterprise tier).
gcloud logging read \
'logName:"cloudaudit.googleapis.com%2Fdata_access" AND
protoPayload.methodName:("GetIamPolicy" OR "TestIamPermissions" OR "ListServiceAccounts" OR "ListServiceAccountKeys" OR
"SearchAllIamPolicies" OR "SearchAllResources" OR "ListProjects" OR "storage.buckets.list" OR
"ListSecrets" OR "ListCryptoKeys" OR "compute.instances.list" OR "compute.projects.get")' \
--project=<project-id> --freshness=7d \
--format='table(timestamp,protoPayload.authenticationInfo.principalEmail,protoPayload.methodName,protoPayload.requestMetadata.callerIp,protoPayload.requestMetadata.callerSuppliedUserAgent)'
Stage 3 · Privilege escalationIAM bindings added, with the member and the role
SetIamPolicy, and the entry carries the diff: bindingDeltas with ADD, the role and the member. Owner, Editor, any *Admin, iam.serviceAccountTokenCreator or iam.serviceAccountUser on a privileged account is the escalation; a new key on someone else's service account (stage 4) is the same thing by another route.Organisation- and folder-level bindings are logged at that resource: rerun with --organization=<org-id> or --folder=<id>. The before/after policy, independent of log retention: gcloud asset get-history --project=<project-id> --asset-names=//cloudresourcemanager.googleapis.com/projects/<project-id> --content-type=iam-policy --start-time=<window-start>.
gcloud logging read \
'logName:"cloudaudit.googleapis.com%2Factivity" AND protoPayload.methodName:"SetIamPolicy" AND
protoPayload.serviceData.policyDelta.bindingDeltas.action="ADD"' \
--project=<project-id> --freshness=14d \
--format='table(timestamp,protoPayload.authenticationInfo.principalEmail,protoPayload.resourceName,protoPayload.serviceData.policyDelta.bindingDeltas[].role,protoPayload.serviceData.policyDelta.bindingDeltas[].member)'
Stage 4 · PersistenceService accounts and keys, SSH keys in metadata, OS Login off, scheduled code, identity pools, new projects
ssh-keys entry in instance or project metadata is a login that survives every IAM change (and enable-oslogin=false makes room for it); a startup script, Cloud Scheduler job, Cloud Function or Cloud Run service is code on a timer; a workload identity pool provider trusts an issuer the attacker controls; a new project on your billing account is where the next stage hides.Metadata hits: read the full entry (--format=json) for the keys ssh-keys, enable-oslogin and startup-script. Identity pools in force: gcloud iam workload-identity-pools providers list --workload-identity-pool=<pool> --location=global --format='table(name,issuerUri,attributeCondition)' — an empty attribute condition trusts every identity the issuer vouches for.
gcloud logging read \
'logName:"cloudaudit.googleapis.com%2Factivity" AND
protoPayload.methodName:("CreateServiceAccount" OR "CreateServiceAccountKey" OR "UploadServiceAccountKey" OR "EnableServiceAccount" OR
"setMetadata" OR "setCommonInstanceMetadata" OR "ImportSshPublicKey" OR
"CreateWorkloadIdentityPool" OR "CreateWorkloadIdentityPoolProvider" OR "UpdateWorkloadIdentityPoolProvider" OR
"CloudScheduler.CreateJob" OR "CloudScheduler.UpdateJob" OR "FunctionService.CreateFunction" OR "FunctionService.UpdateFunction" OR
"CloudFunctionsService.CreateFunction" OR "CloudFunctionsService.UpdateFunction" OR
"Services.CreateService" OR "Services.ReplaceService" OR "Eventarc.CreateTrigger" OR
"CreateProject" OR "UpdateProjectBillingInfo")' \
--project=<project-id> --freshness=30d \
--format='table(timestamp,protoPayload.authenticationInfo.principalEmail,protoPayload.methodName,protoPayload.resourceName,protoPayload.requestMetadata.callerIp)'
# projects attached to the billing account, including ones in no folder you watch
gcloud logging read 'protoPayload.methodName:"UpdateProjectBillingInfo"' --billing-account=<billing-account-id> --freshness=30d \
--format='table(timestamp,protoPayload.authenticationInfo.principalEmail,protoPayload.resourceName)'
Stage 5 · Defence evasionSinks, exclusions, log buckets, Data Access config, organisation policy, SCC mutes, flow logs
DeleteLog removes what is there; a SetIamPolicy carrying auditConfigDeltas turns Data Access logging off; an organisation-policy change removes the guardrail; an SCC mute rule hides the finding; a subnet patch turns off its flow logs.Aggregated sinks and organisation policies are logged at the organisation: rerun with --organization=<org-id>. What is in force now: gcloud logging sinks list --organization=<org-id>, gcloud logging sinks list --project=<project-id> and gcloud logging buckets describe _Default --location=global --project=<project-id> for retention.
gcloud logging read \
'logName:"cloudaudit.googleapis.com%2Factivity" AND (
(protoPayload.serviceName="logging.googleapis.com" AND
protoPayload.methodName:("DeleteSink" OR "UpdateSink" OR "CreateExclusion" OR "UpdateExclusion" OR "DeleteBucket" OR "UpdateBucket" OR "UpdateCmekSettings" OR "DeleteLog"))
OR protoPayload.serviceData.policyDelta.auditConfigDeltas:*
OR protoPayload.serviceName="orgpolicy.googleapis.com"
OR (protoPayload.serviceName="securitycenter.googleapis.com" AND protoPayload.methodName:("MuteConfig" OR "DeleteNotificationConfig" OR "BulkMuteFindings"))
OR protoPayload.methodName:"compute.subnetworks.patch")' \
--project=<project-id> --freshness=30d \
--format='table(timestamp,protoPayload.authenticationInfo.principalEmail,protoPayload.serviceName,protoPayload.methodName,protoPayload.resourceName)'
Stage 6 · Lateral movementImpersonation and delegation chains — who minted tokens for which service account
GenerateAccessToken, GenerateIdToken, SignBlob or SignJwt on a service account the attacker can impersonate, hopping from account to account and project to project; serviceAccountDelegationInfo lists the chain. These are Data Access entries on iamcredentials.googleapis.com and exist only if Data Access audit logs for the IAM API were on in the project that owns the target account.Where the compromised principal holds anything, across the organisation and independent of logs: gcloud asset search-all-iam-policies --scope=organizations/<org-id> --query='policy:<principal-email>'. Then rerun the stage 1 query in every project it names.
gcloud logging read \
'logName:"cloudaudit.googleapis.com%2Fdata_access" AND protoPayload.serviceName="iamcredentials.googleapis.com" AND
protoPayload.methodName:("GenerateAccessToken" OR "GenerateIdToken" OR "SignBlob" OR "SignJwt")' \
--project=<project-id> --freshness=14d \
--format='table(timestamp,protoPayload.authenticationInfo.principalEmail,protoPayload.methodName,protoPayload.request.name,protoPayload.requestMetadata.callerIp,protoPayload.authenticationInfo.serviceAccountDelegationInfo[].firstPartyPrincipal.principalEmail)'
Stage 7 · Resource abuseCompute, GKE, Dataproc, Vertex AI, Cloud Run jobs, Batch and quota requests in every project
What is running now, with accelerators, per project: gcloud compute instances list --project=<project-id> --filter='guestAccelerators:* OR creationTimestamp>-P7D' --format='table(name,zone.basename(),machineType.basename(),guestAccelerators[].acceleratorType.basename(),creationTimestamp)'. The billing export to BigQuery, by SKU and region for the window, is the figure for the billing case.
for p in $(gcloud projects list --format='value(projectId)'); do
gcloud logging read \
'logName:"cloudaudit.googleapis.com%2Factivity" AND
protoPayload.methodName:("compute.instances.insert" OR "compute.instances.bulkInsert" OR "instanceGroupManagers.insert" OR "instanceGroupManagers.resize" OR
"ClusterManager.CreateCluster" OR "ClusterManager.CreateNodePool" OR "ClusterController.CreateCluster" OR
"JobService.CreateCustomJob" OR "Jobs.CreateJob" OR "Jobs.RunJob" OR "BatchService.CreateJob" OR "CreateQuotaPreference")' \
--project="$p" --freshness=7d \
--format='table(timestamp,protoPayload.authenticationInfo.principalEmail,protoPayload.methodName,protoPayload.resourceName,protoPayload.request.machineType)' \
| sed "s/^/$p /"
done
Stage 8 · Data accessBuckets made public, snapshots and images shared, exports, secret and SQL exposure
allUsers or allAuthenticatedUsers on a bucket opens it to the internet without a single object read; an IAM change on a snapshot or image shares the whole disk to another project; a Cloud SQL export or a backup run copies the database out; a secret's IAM policy or a new version is the attacker reaching the secret store. All of these are Admin Activity entries, so they exist even when Data Access logging was off.gcloud logging read \
'logName:"cloudaudit.googleapis.com%2Factivity" AND (
(protoPayload.methodName="storage.setIamPermissions" AND
protoPayload.serviceData.policyDelta.bindingDeltas.member:("allUsers" OR "allAuthenticatedUsers"))
OR protoPayload.methodName:("compute.snapshots.setIamPolicy" OR "compute.images.setIamPolicy" OR "compute.disks.createSnapshot" OR
"compute.snapshots.insert" OR "compute.images.insert" OR "storage.buckets.update" OR
"instances.export" OR "backupRuns.insert" OR "SecretManagerService.SetIamPolicy" OR "SecretManagerService.AddSecretVersion"))' \
--project=<project-id> --freshness=14d \
--format='table(timestamp,protoPayload.authenticationInfo.principalEmail,protoPayload.methodName,protoPayload.resourceName,protoPayload.requestMetadata.callerIp)'
Stage 8 · Data accessWhat was read — Cloud Storage, Secret Manager and BigQuery data access
Why: object reads (storage.objects.get), AccessSecretVersion and KMS Decrypt are Data Access entries and exist only if logging was enabled for those services before the incident — and then only in _Default for 30 days by default.
BigQuery is the exception: its Data Access logs are on by default, so table reads are always answerable.
If the storage logs were off, say so and scope the notification on what the principal could read.
gcloud logging read \
'logName:"cloudaudit.googleapis.com%2Fdata_access" AND protoPayload.authenticationInfo.principalEmail="<principal-email>" AND
(protoPayload.methodName:("storage.objects.get" OR "storage.objects.list" OR "AccessSecretVersion" OR "Decrypt"))' \
--project=<project-id> --freshness=14d \
--format='table(timestamp,protoPayload.serviceName,protoPayload.methodName,protoPayload.resourceName,protoPayload.requestMetadata.callerIp,protoPayload.requestMetadata.callerSuppliedUserAgent)'
# BigQuery reads are logged by default
gcloud logging read \
'protoPayload.serviceName="bigquery.googleapis.com" AND protoPayload.metadata.tableDataRead:* AND protoPayload.authenticationInfo.principalEmail="<principal-email>"' \
--project=<project-id> --freshness=14d \
--format='table(timestamp,protoPayload.resourceName,protoPayload.metadata.tableDataRead.reason,protoPayload.requestMetadata.callerIp)'
Stage 8 · Data accessBytes out of a compromised VM — VPC Flow Logs
gcloud logging read \
'logName:"logs/compute.googleapis.com%2Fvpc_flows" AND jsonPayload.connection.src_ip="<vm-internal-ip>"' \
--project=<project-id> --freshness=7d --limit=5000 --format=json \
| jq -r '.[] | .jsonPayload | [.connection.dest_ip, .connection.dest_port, .connection.protocol,
(.bytes_sent | tonumber), (.dest_location.country // ""), (.dest_instance.vm_name // "")] | @tsv' \
| awk -F'\t' '{ b[$1"\t"$2"\t"$3"\t"$5"\t"$6] += $4 } END { for (k in b) print b[k] "\t" k }' \
| sort -rn | head -50
Stage 9 · ImpactBuckets, snapshots, backups, databases, keys, secrets and projects deleted; billing detached; soft delete removed
DeleteProject starts a 30-day shutdown and UpdateProjectBillingInfo detaching the billing account stops every service in it; a storage.buckets.update that sets the soft-delete retention to zero or removes the retention policy is the attacker making the deletes stick. Object deletes themselves are Data Access entries.Soft-deleted objects (7-day default since 2024, 0–90 configurable): gcloud storage ls --soft-deleted gs://<bucket>/** and gcloud storage restore gs://<bucket>/<object>#<generation>. Key versions in DESTROY_SCHEDULED: gcloud kms keys versions restore before the window ends.
gcloud logging read \
'logName:"cloudaudit.googleapis.com%2Factivity" AND
protoPayload.methodName:("storage.buckets.delete" OR "storage.buckets.update" OR "compute.snapshots.delete" OR "compute.disks.delete" OR
"compute.instances.delete" OR "compute.images.delete" OR "instances.delete" OR "backupRuns.delete" OR
"DeleteBackupVault" OR "DeleteBackup" OR "DeleteBackupPlan" OR
"DestroyCryptoKeyVersion" OR "DisableCryptoKeyVersion" OR "UpdateCryptoKeyPrimaryVersion" OR
"DeleteSecret" OR "DestroySecretVersion" OR "DeleteProject" OR "UpdateProjectBillingInfo" OR
"DeleteDataset" OR "ClusterManager.DeleteCluster" OR "OrgPolicy.DeletePolicy")' \
--project=<project-id> --freshness=14d \
--format='table(timestamp,protoPayload.authenticationInfo.principalEmail,protoPayload.methodName,protoPayload.resourceName,protoPayload.requestMetadata.callerIp)'
# object deletes and overwrites per hour and principal (Data Access, DATA_WRITE; off by default)
gcloud logging read 'logName:"cloudaudit.googleapis.com%2Fdata_access" AND protoPayload.methodName:("storage.objects.delete" OR "storage.objects.create")' \
--project=<project-id> --freshness=14d --format='value(timestamp,protoPayload.authenticationInfo.principalEmail)' \
| awk -F'\t' '{ print substr($1, 1, 13), $2 }' | sort | uniq -c | sort -rn | head
gcloud essential-contacts list --organization=<org-id>), the billing-account administrators or a support case; the ransom-note object itself is a stage 9 storage.objects.create that is only logged with Data Access (DATA_WRITE) on. The mail tenant and leak-site monitoring are the sources.Act
Preserve first, then cut broad, then surgical once the scope is known.
The wide cut is a deny policy or an IR-only project IAM policy; disabling a service account rejects its existing tokens, which is the one revoke in the three clouds that is immediate.
Memory before isolation; compromised workloads are rebuilt from infrastructure-as-code, not cleaned.
T+0–15 · PreserveDump the T0 IAM state and copy the log buckets before you change anything
Why: search-all-iam-policies is every binding in the organisation in one call and is what you are about to rewrite; get-history gives the policy as it was at any time in the last 35 days without depending on logs.
gcloud logging copy moves a whole log bucket to Cloud Storage, which matters because _Default — Data Access and flow logs — keeps 30 days.
The hash list is the chain of custody.
gcloud asset search-all-iam-policies --scope=organizations/<org-id> --format=json > t0-iam-bindings.json
gcloud asset export --organization=<org-id> --content-type=resource --output-path=gs://<evidence-bucket>/<case>/t0-resources.json
gcloud asset get-history --project=<project-id> --asset-names=//cloudresourcemanager.googleapis.com/projects/<project-id> \
--content-type=iam-policy --start-time=<window-start> --format=json > t0-iam-history-<project-id>.json
for sa in $(gcloud iam service-accounts list --project=<project-id> --format='value(email)'); do
gcloud iam service-accounts keys list --iam-account="$sa" --format='csv(name,validAfterTime,disabled,keyType)' | sed "s/^/$sa,/"
done > t0-sa-keys-<project-id>.csv
gcloud logging copy _Default gs://<evidence-bucket>/<case>/logs-<project-id> --location=global --project=<project-id>
sha256sum t0-* | tee t0-SHA256SUMS && gcloud storage cp t0-* gs://<evidence-bucket>/<case>/t0/ # bucket with retention lock, in the evidence project
T+15–30 · QuarantineDeny policy on the project — or replace the project's IAM policy with IR-only
Why: a deny policy is evaluated before any allow, applies to the principals it names (principalSet://goog/public:all for everyone) with exceptionPrincipals for the IR identities, and takes effect for tokens already minted.
Two limits: it only covers the permissions on Google's supported list, so a service outside it is not cut, and it does nothing at the organisation or folder level above its attachment point.
When the deny list is in doubt, the blunt cut is set-iam-policy with a policy that grants only the IR group — the T0 dump is the file you restore from.
gcloud iam policies create ir-quarantine --kind=denypolicies \
--attachment-point=cloudresourcemanager.googleapis.com%2Fprojects%2F<project-id> \
--policy-file=deny-all-except-ir.json
# or: IR-only allow policy (gcloud projects get-iam-policy <project-id> > t0-project-policy.json first)
gcloud projects set-iam-policy <project-id> ir-only-policy.json
T+15–30 · RevokeDisable the service account — its existing tokens are rejected — then its keys
Why: Google's IAM API reference states that disabling a service account rejects its existing access tokens and refuses new ones; nothing says that for a disabled key alone, so treat tokens minted from a key as live for up to an hour until the account is disabled.
Disabling the account also stops the keys you missed and any impersonation chain that ends in it.
Compute instances using the account lose access the same moment — have the list.
gcloud iam service-accounts disable <sa-email> --project=<project-id>
gcloud iam service-accounts keys list --iam-account=<sa-email> --managed-by=user
gcloud iam service-accounts keys disable <key-id> --iam-account=<sa-email> # keep, don't delete: the key id is evidence
gcloud compute instances list --project=<project-id> --filter="serviceAccounts.email=<sa-email>" --format='table(name,zone.basename())'
T+15–30 · RevokeCompromised user: sign out, reset, and strip roles/owner
gcloud: the Admin console's Sign out (Directory API users.signOut) invalidates them, a password reset alone does not. Owner at project level survives nothing you do in the console, so remove it explicitly, and check the organisation for roles the user granted themselves.# Admin console: Users > <user> > Security > Sign out; or Directory API users.signOut, then reset the password and re-enrol 2SV
gcloud asset search-all-iam-policies --scope=organizations/<org-id> --query='policy:user:<user-email>' --format='table(resource,policy.bindings.role)'
gcloud projects remove-iam-policy-binding <project-id> --member=user:<user-email> --role=roles/owner
T+30–60 · IsolateMemory, then snapshot, then tag the VM into the deny-all egress rule — don't delete it
Why: the SSH session through IAP works while the VM still has a network path, so take memory first with AVML staged in the evidence bucket and copied straight back.
The snapshot keeps the disk; deletion protection stops anyone destroying the instance.
The priority-0 egress deny cuts command-and-control and exfiltration for every tagged instance, so more hosts join the cut with one tag.
The VM's service account can't be detached while it runs — disable the account instead (previous step).
# 1. memory (the VM's service account needs objectCreator on the evidence bucket; write-only, retention-locked)
gcloud compute ssh <instance> --zone=<zone> --project=<project-id> --tunnel-through-iap --command \
'sudo sh -c "gcloud storage cp gs://<evidence-bucket>/tools/avml /tmp/avml && chmod +x /tmp/avml && /tmp/avml --compress /tmp/mem.lime.compressed && sha256sum /tmp/mem.lime.compressed && gcloud storage cp /tmp/mem.lime.compressed gs://<evidence-bucket>/<case>/$(hostname).lime.compressed"'
# 2. disk
gcloud compute snapshots create ir-<case>-<instance> --source-disk=<disk> --source-disk-zone=<zone> --project=<project-id>
gcloud compute instances update <instance> --zone=<zone> --deletion-protection
# 3. isolate
gcloud compute firewall-rules create ir-quarantine-egress --network=<vpc> --direction=EGRESS \
--action=DENY --rules=all --destination-ranges=0.0.0.0/0 --target-tags=ir-quarantine --priority=0
gcloud compute instances add-tags <instance> --zone=<zone> --tags=ir-quarantine
By Scenario: What Changes
1. Leaked Access Key or Service Credential
Variation: Assume it was used — public keys are harvested by scanners within minutes; aws iam get-access-key-last-used says when and from which service even before the trail is queried. For keys exposed on GitHub, AWS typically attaches its AWSCompromisedKeyQuarantineV3 managed policy within seconds, opens a support case and raises an AWS Health event of type AWS_RISK_CREDENTIALS_EXPOSED. Leave that policy in place — AWS says not to remove it — but it only denies a set of high-risk actions: it does not revoke the key, so deactivate it and deny the user anyway. GCP disables a leaked service-account key itself when the organisation policy iam.serviceAccountKeyExposureResponse is DISABLE_KEY (the default since June 2024) and writes an audit-log event plus an email to the project owners and security contacts; under WAIT_FOR_ABUSE it only notifies — check which one you run. Then run the full persistence hunt for everything the key could have created, and remove the key from the repository's history — deleting the file leaves it in every clone.
2. Cryptomining / Resource Abuse
Variation: It looks like a billing problem and it is an access problem: the miner is only the visible part. Check every region and every shape — fleets, SageMaker, ECS/EKS, scale sets, AKS, ML compute, GKE, Dataproc — and since 2024 the model-access variant: Bedrock entitlements, Azure OpenAI deployments and Vertex AI jobs resold as LLM access. Look where the bill hides: an attacker with billing rights creates their own account, subscription (Microsoft.Subscription/aliases/write) or project (UpdateProjectBillingInfo seen with --billing-account) and runs the miners there, outside every folder you watch. Snapshot one instance for evidence, then stop the rest — cost is the harm here. The credential that launched them still gets the full hunt; miners are often not the only thing a stolen key was used for. Open the billing case with the provider early.
3. Data Exposed or Taken from Storage
allUsers, a removed public-access block), a snapshot or image shared with an unknown account, storage keys listed, a disk SAS from beginGetAccess, an extortion email with sample files.Variation: Close the exposure first, then work out what could be read. Azure SAS tokens signed with an account key can't be revoked one by one — rotating both account keys is the only way to kill them; user-delegation SAS die when the delegation keys are revoked. Remove external snapshot permissions and look at whether the other account already copied them. If object-level logging was off (S3 data events, StorageBlobLogs, GCP Data Access), notification is scoped on what was reachable, not what was proven read — and the status line says which.
aws ec2 modify-snapshot-attribute --snapshot-id <snap-id> \
--attribute createVolumePermission --operation-type remove --user-ids <external-account>
gcloud storage buckets remove-iam-policy-binding gs://<bucket> --member=allUsers --role=roles/storage.objectViewer
4. Storage Held Hostage or Destroyed
Variation: With SSE-C the provider never holds the key — there is nothing to recover it from, so recovery comes from object versions, Object Lock, or backups the attacker couldn't reach. The quiet versions are deletion by rule and deletion by key: a lifecycle rule that expires everything (remove the rule — objects already expired are only recoverable if versioning kept noncurrent versions), a replication rule to the attacker's bucket, and a KMS key scheduled for deletion, which makes every object it wrapped unreadable when the 7–30 day window ends — aws kms cancel-key-deletion while it is pending. Azure: Recovery Services soft delete keeps deleted backup data 14 days and a Resource Guard makes turning it off a multi-user operation; Key Vault purge protection stops a purge inside the 7–90 day soft-delete window. GCP: soft delete keeps deleted objects 7 days by default (gcloud storage ls --soft-deleted, gcloud storage restore) and a Cloud KMS version in DESTROY_SCHEDULED can be restored for at least 24 hours. Stop the principal before protecting the backups; then follow Ransomware Response, Scenario 6 for the extortion and board decisions.
aws s3api list-object-versions --bucket <bucket> --prefix <path> --max-items 20
aws s3api get-bucket-lifecycle-configuration --bucket <bucket>; aws s3api delete-bucket-lifecycle --bucket <bucket>
aws kms cancel-key-deletion --key-id <key-id>
gcloud kms keys versions restore <version> --key=<key> --keyring=<ring> --location=<loc>
5. Compromised Workload
InstanceCredentialExfiltration findings (the instance role's credentials used from outside AWS).Variation: Two containments, not one. The host: memory first, through SSM, az vm run-command or gcloud compute ssh while the agent channel still works, then snapshot, then isolate — and keep it running. The identity: the role's credentials may already be in use from elsewhere, so revoke the role's sessions and hunt what it did, even after the host is isolated. Flow logs are the only record of what left the host. Enforce IMDSv2 before anything is redeployed.
6. Compromised CI/CD or Deployment Identity
pull_request_target that runs fork code with secrets.Variation: Where humans are read-only, the pipeline identity is the most privileged principal you have — treat it like a domain admin. Pause the pipelines, revoke the role's sessions, and narrow the trust before re-enabling: on AWS the OIDC condition on token.actions.githubusercontent.com:sub must name the exact repository and branch or environment and :aud must be sts.amazonaws.com; on GCP the workload identity pool provider needs an attribute condition such as attribute.repository == "org/repo"; on Entra a federated credential's subject and issuer are the trust, and its addition is logged only as Update application with FederatedIdentityCredentials in the modified properties. Treat the Terraform or Pulumi state as a secret store that was read: it holds database passwords, keys and tokens in plain text, so rotate everything in it, not only the pipeline's own credential. Then diff deployed state against the code: anything the attacker deployed through the pipeline looks like a legitimate change.
7. Root, Management Account or Tenant Admin Compromised
AssumeRoot, root email, billing contact, alternate contact or payment method changed, MFA devices changed, accounts closed or leaving the organisation, SCPs detached, Global Administrator activity nobody owns, an elevateAccess entry, a partner or delegation nobody set up.Variation: This is the one incident where your own guardrails don't help: SCPs never apply to the management account, and a Global Admin can remove every other control. AWS: there is no session revoke for root — replace the password and MFA, then watch for userIdentity.type = 'Root' after that time. With centralised root access enabled, the management account recovers a member account's root without its password (aws sts assume-root with the IAMDeleteRootUserCredentials task policy). Hunt the account itself: StartPrimaryEmailUpdate / AcceptPrimaryEmailUpdate, PutContactInformation and PutAlternateContact on account.amazonaws.com, SetAccountPreferences and SetAdditionalContacts on billingconsole.amazonaws.com, payment instruments on payments.amazonaws.com, and in the organisation CloseAccount, LeaveOrganization, RemoveAccountFromOrganization, DetachPolicy and DisablePolicyType. The attacker's other route is a support case: a convincing “lost MFA” account-recovery request to the provider — keep the alternate security contact current and tell Support the account is under incident response so recovery requests are challenged. Azure: an elevateAccess entry (Entra AuditLogs, service “Azure RBAC (Elevated Access)”) makes a Global Admin owner of every subscription; check it, then the delegations that reach your tenant from outside — GDAP and CSP relationships (Get-MgTenantRelationshipDelegatedAdminRelationship) and Lighthouse (az managedservices assignment list). Use the break-glass accounts, which are excluded from Conditional Access and monitored, not the compromised admin. For Entra ID admin compromise, follow Identity Breach Response in parallel. GCP: the organisation administrator and billing-account roles are the equivalent; search-all-iam-policies at organisation scope for roles/resourcemanager.organizationAdmin and roles/billing.admin says who holds them now.