// #region PREPARSE /************************************************************ ****** Parse timestamp and log headers ****** Extract message field for parsing ****** Parse structured data ************************************************************/ parseJson(prefix="Vendor.", excludeEmpty="true", removePrefixes=["detail."], handleNull="discard") | case{ Vendor.eventTime=* | parseTimestamp(field=Vendor.eventTime); Vendor.digestStartTime=* | parseTimestamp(field="Vendor.digestStartTime"); Vendor.time=* | parseTimestamp(field="Vendor.time"); *; } // #endregion // #region METADATA /************************************************************ ****** Static Metadata Definitions ************************************************************/ | Parser.version := "4.0.0" | Vendor := "aws" | event.module := "cloudtrail" | event.dataset := "cloudtrail.log" | event.kind := "event" | ecs.version := "9.0.0" | Cps.version := "1.1.0" // #endregion // #region NORMALIZATION /************************************************************ ****** Parse unstructured data (i.e. message field) ****** Normalize fields to data model ************************************************************/ // user agent core fields | user_agent.original := Vendor.userAgent // source core fields | source.address := lower(Vendor.sourceIPAddress) | case { cidr("source.address", subnet=["0.0.0.0/0", "::/0"]) | source.ip := source.address; *; } // cloud extended fields | cloud.region := Vendor.awsRegion | cloud.provider := "aws" | case { Vendor.awsAccountId = * | cloud.account.id := Vendor.awsAccountId; Vendor.userIdentity.accountId = * | cloud.account.id := Vendor.userIdentity.accountId; Vendor.recipientAccountId = * | cloud.account.id := Vendor.recipientAccountId; * } | cloud.instance.id := Vendor.requestParameters.instanceId // request related fields | tls := splitString(Vendor.tlsDetails.tlsVersion, by="v") // nolint: invalid-field-check: this is renamed to ecs fields below | tls.version_protocol := tls[0] | tls.version := tls[1] | tls.version_protocol := lower(tls.version_protocol) | tls.cipher := Vendor.tlsDetails.cipherSuite | drop([tls]) | array:drop("tls[]") // event core fields | case { (Vendor.errorCode = * or Vendor.errorMessage = *) | event.outcome := "failure" | event.reason:=Vendor.errorMessage; event.action="ConsoleLogin" | event.outcome:=lower(Vendor.responseElements.ConsoleLogin); // nolint: invalid-enum-check, ignoring as it is correct * | event.outcome := "success"; } | event.id := Vendor.eventID // event extended fields | event.provider := Vendor.eventSource | event.action := Vendor.eventName // error core fields | error.code := Vendor.errorCode | error.message := Vendor.errorMessage // file fields | file.path := Vendor.digestS3Object | case { file.path=/.*\/(?.*)/; *; } | case { Vendor.previousDigestHashAlgorithm = "SHA-256" | file.hash.sha256 := lower(Vendor.previousDigestHashValue); * } // host fields | host.name:=lower(requestParameters.Host) // user fields | user.id := Vendor.userIdentity.principalId | Vendor.userIdentity.type match { "IAMUser" => user.name := lower(Vendor.userIdentity.userName); "AssumedRole" => user.name := lower(Vendor.userIdentity.sessionContext.sessionIssuer.userName) | array:append("user.roles[]", values=[Vendor.userIdentity.sessionContext.sessionIssuer.arn]); in(values=["SAMLUser", "Role"]) => user.name := lower(Vendor.userIdentity.userName) | array:append("user.roles[]", values=[Vendor.requestParameters.roleArn]); "AWSAccount" => user.name:=lower(Vendor.requestParameters.roleSessionName) | array:append("user.roles[]", values=[Vendor.requestParameters.roleArn]); "AWSService" => user.name:=lower(Vendor.userIdentity.invokedBy) | array:append("user.roles[]", values=[Vendor.requestParameters.roleArn]); "IdentityCenterUser" => user.id := Vendor.userIdentity.onBehalfOf.userId | array:append("user.roles[]", values=[Vendor.userIdentity.onBehalfOf.identityStoreArn]); * => *; } // In case of unknown or identity center user userIdentity type - one of the two fields will exist, overwrite with userIdentity if both exist | case { user.name !=* | user.name := Vendor.additionalEventData.UserName | user.name := Vendor.userIdentity.userName; *; } | case{ Vendor.requestParameters.bucketName = * | cloud.Storage.bucket_name := Vendor.requestParameters.bucketName; // nolint: invalid-field-check, ignoring - valid custom field Vendor.digestS3Bucket = * | cloud.Storage.bucket_name := Vendor.digestS3Bucket; // nolint: invalid-field-check, ignoring - valid custom field *; } // Based on the action we can apply event categorization and typing | event.action match { // AWS Identity and Access Management actions in(values=["AttachGroupPolicy", "DetachGroupPolicy", "PutGroupPolicy", "RemoveUserFromGroup", "UpdateGroup"]) => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["group", "change"]) | event.dataset:="cloudtrail.iam" | observer.type := "identity"; in(values=["DeleteGroupPolicy", "DeleteGroup"]) => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["group", "change", "deletion"]) | event.dataset:="cloudtrail.iam" | observer.type := "identity"; "AssumeRole" => array:append("event.category[]", values=["iam"]) | array:append(array="event.type[]", values=["user", "change"]) | event.dataset:="cloudtrail.iam" | observer.type := "identity"; "ConsoleLogin" => array:append("event.category[]", values=["authentication"]) | array:append("event.type[]", values=["start"]) | event.dataset:="cloudtrail.auth"; in(values=["AttachUserPolicy", "ChangePassword", "DeactivateMFADevice", "DetachUserPolicy", "EnableMFADevice", "PutUserPermissionsBoundary", "PutUserPolicy", "TagUser", "UntagUser", "UpdateAccessKey", "UpdateAccountPasswordPolicy", "UpdateLoginProfile"]) => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["user", "change"]) | event.dataset:="cloudtrail.iam" | observer.type := "identity"; in(values=["CreateAccessKey", "CreateVirtualMFADevice"]) => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["user", "change", "creation"]) | event.dataset:="cloudtrail.iam" | observer.type := "identity"; in(values=["DeleteAccessKey", "DeleteSSHPublicKey", "DeleteUserPermissionsBoundary", "DeleteUserPolicy", "DeleteVirtualMFADevice", "DeleteUser"]) => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["user", "change", "deletion"]) | event.dataset:="cloudtrail.iam" | observer.type := "identity"; in(values=["GetGroup", "GetGroupPolicy", "ListAttachedGroupPolicies", "ListGroupPolicies", "ListGroups"]) => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["group", "info"]) | event.dataset:="cloudtrail.iam" | observer.type := "identity"; in(values=["GetUser", "GetUserPolicy", "ListAttachedUserPolicies", "ListUserPolicies", "ListUsers", "ListUserTags"]) => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["user"]) | event.dataset:="cloudtrail.iam" | observer.type := "identity"; "ListGroupsForUser" => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["user", "group"]) | event.dataset:="cloudtrail.iam" | observer.type := "identity"; in(values=["AddUserToGroup","RemoveUserFromGroup"]) => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["user", "group", "change"]) | event.dataset:="cloudtrail.iam" | observer.type := "identity"; in(values=["SetDefaultPolicyVersion", "SetSecurityTokenServicePreferences"]) => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["admin", "change"]) | event.dataset:="cloudtrail.iam" | observer.type := "identity"; // AWS S3 actions in(values=["DescribeMultiRegionAccessPointOperation","GetStorageLensConfiguration","GetStorageLensConfigurationTagging","GetStorageLensGroup","ListStorageLensConfigurations","ListStorageLensGroups","ListTagsForResource"]) => array:append("event.category[]", values=["configuration"]) | array:append("event.type[]", values=["access"]) | event.dataset:="cloudtrail.s3"; in(values=["PutStorageLensConfiguration","PutStorageLensConfigurationTagging","TagResource","UntagResource","UpdateStorageLensGroup"]) => array:append("event.category[]", values=["configuration"]) | array:append("event.type[]", values=["change"]) | event.dataset:="cloudtrail.s3"; in(values=["DeleteStorageLensConfiguration","DeleteStorageLensConfigurationTagging","DeleteStorageLensGroup"]) => array:append("event.category[]", values=["configuration"]) | array:append("event.type[]", values=["deletion"]) | event.dataset:="cloudtrail.s3"; in(values=["GetBucketAccelerateConfiguration","GetBucketAcl","GetBucketAnalyticsConfiguration","GetBucketCors","GetBucketEncryption","GetBucketIntelligentTieringConfiguration","GetBucketInventoryConfiguration","GetBucketLifecycle","GetBucketLifecycleConfiguration","GetBucketLifecycleConfiguration","GetBucketLocation","GetBucketLogging","GetBucketMetricsConfiguration","GetBucketNotification","GetBucketNotificationConfiguration","GetBucketOwnershipControls","GetBucketPolicy","GetBucketPolicyStatus","GetBucketReplication","GetBucketRequestPayment","GetBucketTagging","GetBucketVersioning","GetBucketWebsite","GetObjectAcl","GetObjectAttributes","GetObjectLegalHold","GetObjectLockConfiguration","GetObjectRetention","GetObjectTagging","GetObjectTorrent","GetPublicAccessBlock","ListBucketAnalyticsConfigurations","ListBucketIntelligentTieringConfigurations","ListBucketInventoryConfigurations","ListBucketMetricsConfigurations","ListObjectVersions"]) => array:append("event.category[]", values=["configuration", "file"]) | array:append("event.type[]", values=["access"]) | event.dataset:="cloudtrail.s3"; in(values=["PutBucketAccelerateConfiguration","PutBucketAcl","PutBucketAnalyticsConfiguration","PutBucketCors","PutBucketEncryption","PutBucketIntelligentTieringConfiguration","PutBucketInventoryConfiguration","PutBucketLifecycle","PutBucketLifecycleConfiguration","PutBucketLogging","PutBucketMetricsConfiguration","PutBucketNotification","PutBucketNotificationConfiguration","PutBucketOwnershipControls","PutBucketPolicy","PutBucketPolicy","PutBucketReplication","PutBucketReplication","PutBucketRequestPayment","PutBucketTagging","PutBucketTagging","PutBucketVersioning","PutBucketVersioning","PutBucketWebsite","PutObjectAcl","PutObjectLegalHold","PutObjectLockConfiguration","PutObjectRetention","PutObjectTagging","PutPublicAccessBlock"]) => array:append("event.category[]", values=["configuration", "file"]) | array:append("event.type[]", values=["change"]) | event.dataset:="cloudtrail.s3"; in(values=["DeleteBucketAnalyticsConfiguration","DeleteBucketCors","DeleteBucketEncryption","DeleteBucketIntelligentTieringConfiguration","DeleteBucketInventoryConfiguration","DeleteBucketLifecycle","DeleteBucketLifecycleConfiguration","DeleteBucketMetricsConfiguration","DeleteBucketOwnershipControls","DeleteBucketPolicy","DeleteBucketReplication","DeleteBucketTagging","DeleteBucketWebsite","DeleteObjectTagging","DeletePublicAccessBlock"]) => array:append("event.category[]", values=["configuration", "file"]) | array:append("event.type[]", values=["deletion"]) | event.dataset:="cloudtrail.s3"; in(values=["GetAccessGrant","GetAccessGrantsInstance","GetAccessGrantsInstanceForPrefix","GetAccessGrantsInstanceResourcePolicy","GetAccessGrantsLocation","GetAccessPoint","GetAccessPointConfigurationForObjectLambda","GetAccessPointForObjectLambda","GetAccessPointPolicy","GetAccessPointPolicyForObjectLambda","GetAccessPointPolicyStatus","GetAccessPointPolicyStatusForObjectLambda","GetMultiRegionAccessPoint","GetMultiRegionAccessPointPolicy","GetMultiRegionAccessPointPolicyStatus","GetMultiRegionAccessPointRoutes","ListAccessGrants","ListAccessGrantsInstances","ListAccessGrantsLocations","ListAccessPoints","ListAccessPointsForObjectLambda","ListMultiRegionAccessPoints"]) => array:append("event.category[]", values=["configuration", "iam"]) | array:append("event.type[]", values=["access"]) | event.dataset:="cloudtrail.s3"; in(values=["DissociateAccessGrantsIdentityCenter","PutAccessGrantsInstanceResourcePolicy","PutAccessPointConfigurationForObjectLambda","PutAccessPointPolicy","PutAccessPointPolicyForObjectLambda","PutMultiRegionAccessPointPolicy","SubmitMultiRegionAccessPointRoutes","UpdateAccessGrantsLocation"]) => array:append("event.category[]", values=["configuration", "iam"]) | array:append("event.type[]", values=["change"]) | event.dataset:="cloudtrail.s3"; "GetDataAccess" => array:append("event.category[]", values=["configuration", "iam"]) | array:append("event.type[]", values=["creation"]) | event.dataset:="cloudtrail.s3"; in(values=["DescribeJob","GetJobTagging"]) => array:append("event.category[]", values=["configuration", "process"]) | array:append("event.type[]", values=["access"]) | event.dataset:="cloudtrail.s3"; in(values=["PutJobTagging","UpdateJobPriority","UpdateJobStatus"]) => array:append("event.category[]", values=["configuration", "process"]) | array:append("event.type[]", values=["change"]) | event.dataset:="cloudtrail.s3"; "DeleteJobTagging" => array:append("event.category[]", values=["configuration", "process"]) | array:append("event.type[]", values=["deletion"]) | event.dataset:="cloudtrail.s3"; in(values=["GetBucket","GetObject","HeadBucket","HeadObject","ListBuckets","ListDirectoryBuckets","ListObjects","ListObjectsV2","ListRegionalBuckets","SelectObjectContent","WriteGetObjectResponse"]) => array:append("event.category[]", values=["file"]) | array:append("event.type[]", values=["access"]) | event.dataset:="cloudtrail.s3"; "CopyObject" => array:append("event.category[]", values=["file"]) | array:append("event.type[]", values=["access", "creation"]) | event.dataset:="cloudtrail.s3"; in(values=["PutObject","RestoreObject"]) => array:append("event.category[]", values=["file"]) | array:append("event.type[]", values=["change"]) | event.dataset:="cloudtrail.s3"; in(values=["CreateBucket","UploadPart","UploadPartCopy"]) => array:append("event.category[]", values=["file"]) | array:append("event.type[]", values=["creation"]) | event.dataset:="cloudtrail.s3"; in(values=["DeleteBucket","DeleteObject","DeleteObjects"]) => array:append("event.category[]", values=["file"]) | array:append("event.type[]", values=["deletion"]) | event.dataset:="cloudtrail.s3"; "CompleteMultipartUpload" => array:append("event.category[]", values=["file", "network"]) | array:append("event.type[]", values=["creation", "end"]) | event.dataset:="cloudtrail.s3"; "AbortMultipartUpload" => array:append("event.category[]", values=["file", "network"]) | array:append("event.type[]", values=["creation", "end"]) | event.dataset:="cloudtrail.s3"; in(values=["CreateAccessGrant","CreateAccessGrantsInstance","CreateAccessGrantsLocation","CreateAccessPoint","CreateAccessPointForObjectLambda","CreateMultiRegionAccessPoint","CreateStorageLensGroup"]) => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["creation"]) | event.dataset:="cloudtrail.s3"; in(values=["DeleteAccessGrant","DeleteAccessGrantsInstance","DeleteAccessGrantsInstanceResourcePolicy","DeleteAccessGrantsLocation","DeleteAccessPoint","DeleteAccessPointForObjectLambda","DeleteAccessPointPolicy","DeleteAccessPointPolicyForObjectLambda","DeleteMultiRegionAccessPoint"]) => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["deletion"]) | event.dataset:="cloudtrail.s3"; "CreateMultipartUpload" => array:append("event.category[]", values=["file", "network"]) | array:append("event.type[]", values=["creation", "start"]) | event.dataset:="cloudtrail.s3"; "CreateSession" => array:append("event.category[]", values=["session"]) | array:append("event.type[]", values=["start"]) | event.dataset:="cloudtrail.s3"; in(values=["ListJobs","ListMultipartUploads","ListParts"]) => array:append("event.category[]", values=["process"]) | array:append("event.type[]", values=["info"]) | event.dataset:="cloudtrail.s3"; "CreateJob" => array:append("event.category[]", values=["process"]) | array:append("event.type[]", values=["start"]) | event.dataset:="cloudtrail.s3"; in(values=["StartInstances", "RunInstances"]) => array:append("event.category[]", values=["host"]) | array:append("event.type[]", values=["start"]) | event.dataset:="cloudtrail.s3"; "CreateKeyPair" => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["admin", "creation"]) | event.dataset:="cloudtrail.s3"; "AssumeRoleWithSAML" => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["user", "change"]) | event.dataset:="cloudtrail.iam"; "DescribeInstances" => array:append("event.category[]", values=["configuration"]) | array:append("event.type[]", values=["info"]); "UpdateTrail" => array:append("event.category[]", values=["configuration"]) | array:append("event.type[]", values=["change"]); "CreateGroup" => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["group", "creation"]) | event.dataset:="cloudtrail.iam" | observer.type := "identity"; "CreateKeyPair" => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["admin", "creation"]) | event.dataset:="cloudtrail.iam" | observer.type := "identity"; "CreateUser" => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["user", "creation"]) | event.dataset:="cloudtrail.iam" | observer.type := "identity"; "UpdateRole" => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["admin", "change"]) | event.dataset:="cloudtrail.iam" | observer.type := "identity"; in(values=["UpdateSSHPublicKey", "UpdateUser"]) => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["user", "change"]) | event.dataset:="cloudtrail.iam" | observer.type := "identity"; "UserAuthentication" => array:append(array="event.category[]", values=["authentication"]) | array:append(array="event.type[]", values=["start"]) | event.outcome := "success" | event.dataset:="cloudtrail.auth"; "*User*" => array:append("event.category[]", values=["iam"]) | array:append("event.type[]", values=["user"]) | event.dataset:="cloudtrail.iam" | observer.type := "identity"; "*Delete*" => array:append("event.category[]", values=["configuration"]) | array:append("event.type[]", values=["deletion"]); "*Create*" => array:append("event.category[]", values=["configuration"]) | array:append("event.type[]", values=["creation"]); in(values=["Update*", "Modify*"])=> array:append("event.category[]", values=["configuration"]) | array:append("event.type[]", values=["change"]); in(values=["Get*", "List*", "Describe*"]) => array:append("event.category[]", values=["configuration"]) | array:append("event.type[]", values=["info"]); "GenerateDataKey" => array:append(array="event.category[]", values=["configuration"]) | array:append(array="event.type[]", values=["creation"]) | event.outcome := "success"; * => *; } // #endregion // #region POST-NORMALIZATION /************************************************************ ****** Post Normalization ****** Custom parser logic needed after normalization ************************************************************/ // #endregion