+
+ | Rule Parameter |
+ Expected Value Type |
+ Description |
+
+
+ | dsUsernameKey |
+ string |
+ SSM Parameter Store key to retrive username of the Deep Security account to use for querying anti-malware status |
+
+
+ | dsPasswordKey |
+ string |
+ SSM Parameter Store key to retrive password for the Deep Security account to use for querying anti-malware status. |
+
+
+ | dsPasswordEncryptionContext |
+ string or URI |
+ The encryption context used to encrypt the dsPassword. If this parameter is given, the rule will include the encryption context information when decrypting the dsPassword value. Requires dsPasswordKey to be useful. See [Protecting Your Deep Security Manager API Password](#protecting-your-deep-security-manager-api-password) below for more details.
+ |
+
+ | dsTenant |
+ string |
+ Optional as long as dsHostname is specified. Indicates which tenant to sign in to within Deep Security |
+
+
+ | dsHostname |
+ string |
+ Optional as long as dsTenant is specified. Defaults to Deep Security as a Service. Indicates which Deep Security manager the rule should sign in to |
+
+
+ | dsPort |
+ int |
+ Optional. Defaults to 443. Indicates the port to connect to the Deep Security manager on |
+
+
+ | dsIgnoreSslValidation |
+ boolean (true or false) |
+ Optional. Use only when connecting to a Deep Security manager that is using a self-signed SSL certificate |
+
+
+
+During execution, this rule sign in to the Deep Security API. You should setup a dedicated API access account to do this. Deep Security contains a robust role-based access control (RBAC) framework which you can use to ensure that this set of credentials has the least amount of privileges to success.
+
+This rule requires view access to one or more computers within Deep Security.
+
+#### IsInstanceProtectedBy
+
+Checks to see if the current instance is protected by any of Deep Security's controls. Controls must be "on" and set to their strongest setting (a/k/a "real-time" or "prevention") in order for the rule to be considered compliant.
+
+This is the generic version of *IsInstanceProtectedByAntiMalware*.
+
+Lambda handler: **dsIsInstanceProtectedBy.aws_config_rule_handler**
+
+##### Rule Parameters:
+
+
+
+ | Rule Parameter |
+ Expected Value Type |
+ Description |
+
+
+ | dsUsernameKey |
+ string |
+ SSM Parameter Store key to retrive username of the Deep Security account to use for querying anti-malware status |
+
+
+ | dsPasswordKey |
+ string |
+ SSM Parameter Store key to retrive password for the Deep Security account to use for querying anti-malware status. |
+
+
+ | dsPasswordEncryptionContext |
+ string or URI |
+ The encryption context used to encrypt the dsPassword. If this parameter is given, the rule will include the encryption context information when decrypting the dsPassword value. Requires dsPasswordKey to be useful. See [Protecting Your Deep Security Manager API Password](#protecting-your-deep-security-manager-api-password) below for more details.
+ |
+
+ | dsTenant |
+ string |
+ Optional as long as dsHostname is specified. Indicates which tenant to sign in to within Deep Security |
+
+
+ | dsHostname |
+ string |
+ Optional as long as dsTenant is specified. Defaults to Deep Security as a Service. Indicates which Deep Security manager the rule should sign in to |
+
+
+ | dsPort |
+ int |
+ Optional. Defaults to 443. Indicates the port to connect to the Deep Security manager on |
+
+
+ | dsIgnoreSslValidation |
+ boolean (true or false) |
+ Optional. Use only when connecting to a Deep Security manager that is using a self-signed SSL certificate |
+
+
+ | dsControl |
+ string |
+ The name of the control to verify. Must be one of [ anti_malware, web_reputation, firewall, intrusion_prevention, integrity_monitoring, log_inspection ] |
+
+
+
+During execution, this rule signs in to the Deep Security API. You should setup a dedicated API access account to do this. Deep Security contains a robust role-based access control (RBAC) framework which you can use to ensure that this set of credentials has the least amount of privileges to success.
+
+This rule requires view access to one or more computers within Deep Security.
+
+#### DoesInstanceHavePolicy
+
+Checks to see if the current instance is protected by a specific Deep Security policy.
+
+Lambda handler: **dsDoesInstanceHavePolicy.aws_config_rule_handler**
+
+##### Rule Parameters:
+
+
+
+ | Rule Parameter |
+ Expected Value Type |
+ Description |
+
+
+ | dsUsernameKey |
+ string |
+ SSM Parameter Store key to retrive username of the Deep Security account to use for querying anti-malware status |
+
+
+ | dsPasswordKey |
+ string |
+ SSM Parameter Store key to retrive password for the Deep Security account to use for querying anti-malware status. |
+
+
+ | dsPasswordEncryptionContext |
+ string or URI |
+ The encryption context used to encrypt the dsPassword. If this parameter is given, the rule will include the encryption context information when decrypting the dsPassword value. Requires dsPasswordKey to be useful. See [Protecting Your Deep Security Manager API Password](#protecting-your-deep-security-manager-api-password) below for more details.
+ |
+
+ | dsTenant |
+ string |
+ Optional as long as dsHostname is specified. Indicates which tenant to sign in to within Deep Security |
+
+
+ | dsHostname |
+ string |
+ Optional as long as dsTenant is specified. Defaults to Deep Security as a Service. Indicates which Deep Security manager the rule should sign in to |
+
+
+ | dsPort |
+ int |
+ Optional. Defaults to 443. Indicates the port to connect to the Deep Security manager on |
+
+
+ | dsIgnoreSslValidation |
+ boolean (true or false) |
+ Optional. Use only when connecting to a Deep Security manager that is using a self-signed SSL certificate |
+
+
+ | dsPolicy |
+ string |
+ The name of the policy to verify |
+
+
+
+During execution, this rule signs in to the Deep Security API. You should setup a dedicated API access account to do this. Deep Security contains a robust role-based access control (RBAC) framework which you can use to ensure that this set of credentials has the least amount of privileges to success.
+
+This rule requires view access to one or more computers within Deep Security.
+
+#### IsInstanceClear
+
+Checks to see if the current instance is has any warnings, alerts, or errors in Deep Security. An instance is compliant if it does **not** have any warnings, alerts, or errors (a/k/a compliant, which means everything is working as expected with no active security alerts).
+
+Lambda handler: **dsIsInstanceClear.aws_config_rule_handler**
+
+##### Rule Parameters:
+
+
+
+ | Rule Parameter |
+ Expected Value Type |
+ Description |
+
+
+ | dsUsernameKey |
+ string |
+ SSM Parameter Store key to retrive username of the Deep Security account to use for querying anti-malware status |
+
+
+ | dsPasswordKey |
+ string |
+ SSM Parameter Store key to retrive password for the Deep Security account to use for querying anti-malware status. |
+
+
+ | dsPasswordEncryptionContext |
+ string or URI |
+ The encryption context used to encrypt the dsPassword. If this parameter is given, the rule will include the encryption context information when decrypting the dsPassword value. Requires dsPasswordKey to be useful. See [Protecting Your Deep Security Manager API Password](#protecting-your-deep-security-manager-api-password) below for more details.
+ |
+
+ | dsTenant |
+ string |
+ Optional as long as dsHostname is specified. Indicates which tenant to sign in to within Deep Security |
+
+
+ | dsHostname |
+ string |
+ Optional as long as dsTenant is specified. Defaults to Deep Security as a Service. Indicates which Deep Security manager the rule should sign in to |
+
+
+ | dsPort |
+ int |
+ Optional. Defaults to 443. Indicates the port to connect to the Deep Security manager on |
+
+
+ | dsIgnoreSslValidation |
+ boolean (true or false) |
+ Optional. Use only when connecting to a Deep Security manager that is using a self-signed SSL certificate |
+
+
+
+During execution, this rule signs in to the Deep Security API. You should setup a dedicated API access account to do this. Deep Security contains a robust role-based access control (RBAC) framework which you can use to ensure that this set of credentials has the least amount of privileges to success.
+
+This rule requires view access to one or more computers within Deep Security.
+
+## Support
+
+This is an Open Source community project. Project contributors may be able to help,
+depending on their time and availability. Please be specific about what you're
+trying to do, your system, and steps to reproduce the problem.
+
+For bug reports or feature requests, please
+[open an issue](../issues).
+You are welcome to [contribute](#contribute).
+
+Official support from Trend Micro is not available. Individual contributors may be
+Trend Micro employees, but are not official support.
+
+## Contribute
+
+We accept contributions from the community. To submit changes:
+
+1. Fork this repository.
+1. Create a new feature branch.
+1. Make your changes.
+1. Submit a pull request with an explanation of your changes or additions.
+
+We will review and work with you to release the code.
diff --git a/build.sh b/build.sh
new file mode 100755
index 0000000..236742e
--- /dev/null
+++ b/build.sh
@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+
+set -e
+
+echo ""
+echo "Running unit tests..."
+
+pytest -s -vv
+
+echo ""
+echo "Copying dependencies..."
+
+for i in rules/*; do
+ cp -r src requirements.txt $i
+done
+
+echo ""
+echo "Building from source..."
+
+if [[ "`uname`" == "Linux" ]]; then
+ sam build -t deep-security.yml
+else
+ sam build -t deep-security.yml -u
+fi
+
+echo ""
diff --git a/clean.sh b/clean.sh
new file mode 100755
index 0000000..61accdd
--- /dev/null
+++ b/clean.sh
@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+
+set -e
+
+echo ""
+echo "Cleaning up..."
+
+for i in rules/*; do
+ rm -rf $i/src $i/tests $i/requirements.txt
+done
+
+rm -rf .aws-sam
+
+echo ""
diff --git a/deep-security.yml b/deep-security.yml
new file mode 100644
index 0000000..93c1676
--- /dev/null
+++ b/deep-security.yml
@@ -0,0 +1,278 @@
+Transform: AWS::Serverless-2016-10-31
+Description: AWS CloudFormation template to create custom AWS Config rules that interact
+ with the Trend Micro Deep Security Manager. You will be billed for the AWS resources
+ used if you create a stack from this template.
+Parameters:
+ ConfigBucket:
+ Description: Name of the S3 bucket for AWS Config to store history and files
+ Type: String
+ MinLength: 1
+ MaxLength: 255
+ ConfigPrefix:
+ Description: Object prefix inside config bucket
+ Type: String
+ MinLength: 1
+ MaxLength: 255
+ DSUsernameKey:
+ Description: Parameter Store key name for Deep Security Manager username
+ Type: String
+ DSPasswordKey:
+ Description: Parameter Store key name for Deep Security Manager password
+ Type: String
+ DSHostname:
+ Description: Deep Security Manager hostname
+ Type: String
+ DSPort:
+ Description: Deep Security Manager port
+ Type: Number
+ Default: 443
+ DSTenant:
+ Description: Deep Security tenant name
+ Type: String
+ Default: ''
+ DSIgnoreSslValidation:
+ Description: Whether to ignore SSL validation on connection
+ Type: String
+ Default: false
+ DSPolicy:
+ Description: Deep Security policy to check against
+ Type: String
+ DSControl:
+ Description: Deep Security protection name to check against
+ Type: String
+ AllowedValues: [ anti_malware, web_reputation, firewall, intrusion_prevention, integrity_monitoring, log_inspection ]
+Metadata:
+ AWS::ServerlessRepo::Application:
+ Name: Deep-Security-Config-Rules
+ Description: A set of AWS Config Rules to help ensure that your AWS deployments are leveraging the protection of Deep Security.
+ These rules help centralize your compliance information in one place, AWS Config.
+ Author: Trend Micro
+ SpdxLicenseId: Apache-2.0
+ LicenseUrl: LICENSE
+ ReadmeUrl: README.md
+ Labels: ['trendmicro', 'deepsecurity', 'security', 'config']
+ HomePageUrl: https://github.com/deep-security/aws-config
+ SemanticVersion: 0.0.1
+ SourceCodeUrl: https://github.com/deep-security/aws-config
+Resources:
+ dsDoesInstanceHavePolicyLambda:
+ Type: AWS::Serverless::Function
+ Properties:
+ CodeUri: rules/ds-DoesInstanceHavePolicy
+ Description: Custom AWS Config rule that checks with the Trend Micro Deep Security
+ Manager to see if the named policy is in effect. See https://github.com/deep-security/aws-config
+ for more details.
+ Handler: dsDoesInstanceHavePolicy.aws_config_rule_handler
+ Role: !GetAtt dsConfigRuleRole.Arn
+ Runtime: python3.7
+ Timeout: 60
+ dsDoesInstanceHavePolicyLambdaPermission:
+ Type: AWS::Lambda::Permission
+ Properties:
+ FunctionName: !GetAtt dsDoesInstanceHavePolicyLambda.Arn
+ Action: lambda:InvokeFunction
+ Principal: config.amazonaws.com
+ dsIsInstanceClearLambda:
+ Type: AWS::Serverless::Function
+ Properties:
+ CodeUri: rules/ds-IsInstanceClear
+ Description: Custom AWS Config rule that checks with the Trend Micro Deep Security
+ Manager to see if the instance is clear of any alerts, warnings, or errors.
+ See https://github.com/deep-security/aws-config for more details.
+ Handler: dsIsInstanceClear.aws_config_rule_handler
+ Role: !GetAtt dsConfigRuleRole.Arn
+ Runtime: python3.7
+ Timeout: 60
+ dsIsInstanceClearLambdaPermission:
+ Type: AWS::Lambda::Permission
+ Properties:
+ FunctionName: !GetAtt dsIsInstanceClearLambda.Arn
+ Action: lambda:InvokeFunction
+ Principal: config.amazonaws.com
+ dsIsInstanceProtectedByLambda:
+ Type: AWS::Serverless::Function
+ Properties:
+ CodeUri: rules/ds-IsInstanceProtectedBy
+ Description: Custom AWS Config rule that checks with the Trend Micro Deep Security
+ Manager to see if the instance is protected using the specified security control.
+ See https://github.com/deep-security/aws-config for more details.
+ Handler: dsIsInstanceProtectedBy.aws_config_rule_handler
+ Role: !GetAtt dsConfigRuleRole.Arn
+ Runtime: python3.7
+ Timeout: 60
+ dsIsInstanceProtectedByLambdaPermission:
+ Type: AWS::Lambda::Permission
+ Properties:
+ FunctionName: !GetAtt dsIsInstanceProtectedByLambda.Arn
+ Action: lambda:InvokeFunction
+ Principal: config.amazonaws.com
+ dsIsInstanceProtectedByAntiMalwareLambda:
+ Type: AWS::Serverless::Function
+ Properties:
+ CodeUri: rules/ds-IsInstanceProtectedByAntiMalware
+ Description: Custom AWS Config rule that checks with the Trend Micro Deep Security
+ Manager to see if the instance is protected using the anti-malware security
+ control. See https://github.com/deep-security/aws-config for more details.
+ Handler: dsIsInstanceProtectedByAntiMalware.aws_config_rule_handler
+ Role: !GetAtt dsConfigRuleRole.Arn
+ Runtime: python3.7
+ Timeout: 60
+ dsIsInstanceProtectedByAntiMalwareLambdaPermission:
+ Type: AWS::Lambda::Permission
+ Properties:
+ FunctionName: !GetAtt dsIsInstanceProtectedByAntiMalwareLambda.Arn
+ Action: lambda:InvokeFunction
+ Principal: config.amazonaws.com
+ dsConfigRuleRole:
+ Type: AWS::IAM::Role
+ Properties:
+ AssumeRolePolicyDocument:
+ Statement:
+ - Effect: Allow
+ Principal:
+ Service: lambda.amazonaws.com
+ Action: sts:AssumeRole
+ Policies:
+ - PolicyName: dsConfigRulePolicy
+ PolicyDocument:
+ Statement:
+ - Action:
+ - ssm:GetParameter
+ Effect: Allow
+ Resource:
+ - !Sub arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/*
+ - Action:
+ - kms:Decrypt
+ Effect: Allow
+ Resource: '*'
+ - Action:
+ - s3:GetObject
+ Effect: Allow
+ Resource: !Sub arn:aws:s3:::${ConfigBucket}/${ConfigPrefix}/*
+ - Action:
+ - logs:CreateLogGroup
+ - logs:CreateLogStream
+ - logs:PutLogEvents
+ - logs:DescribeLogStreams
+ Effect: Allow
+ Resource: '*'
+ - Action:
+ - config:PutEvaluations
+ Effect: Allow
+ Resource: '*'
+ dsDoesInstanceHavePolicyRule:
+ Type: AWS::Config::ConfigRule
+ Properties:
+ Description: This rule checks with the Trend Micro Deep Security
+ Manager to see if the named policy is in effect.
+ Scope:
+ ComplianceResourceTypes:
+ - AWS::EC2::Instance
+ Source:
+ Owner: CUSTOM_LAMBDA
+ SourceIdentifier: !GetAtt dsDoesInstanceHavePolicyLambda.Arn
+ SourceDetails:
+ - EventSource: aws.config
+ MessageType: ConfigurationItemChangeNotification
+ - EventSource: aws.config
+ MessageType: OversizedConfigurationItemChangeNotification
+ InputParameters:
+ dsUsernameKey: !Ref DSUsernameKey
+ dsPasswordKey: !Ref DSPasswordKey
+ dsHostname: !Ref DSHostname
+ dsPort: !Ref DSPort
+ dsTenant: !Ref DSTenant
+ dsIgnoreSslValidation: !Ref DSIgnoreSslValidation
+ dsPolicy: !Ref DSPolicy
+ DependsOn:
+ - dsDoesInstanceHavePolicyLambdaPermission
+ dsIsInstanceClearRule:
+ Type: AWS::Config::ConfigRule
+ Properties:
+ Description: This rule checks with the Trend Micro Deep Security Manager to see
+ if the instance is clear of any alerts, warnings, or errors.
+ Scope:
+ ComplianceResourceTypes:
+ - AWS::EC2::Instance
+ Source:
+ Owner: CUSTOM_LAMBDA
+ SourceIdentifier: !GetAtt dsIsInstanceClearLambda.Arn
+ SourceDetails:
+ - EventSource: aws.config
+ MessageType: ConfigurationItemChangeNotification
+ - EventSource: aws.config
+ MessageType: OversizedConfigurationItemChangeNotification
+ InputParameters:
+ dsUsernameKey: !Ref DSUsernameKey
+ dsPasswordKey: !Ref DSPasswordKey
+ dsHostname: !Ref DSHostname
+ dsPort: !Ref DSPort
+ dsTenant: !Ref DSTenant
+ dsIgnoreSslValidation: !Ref DSIgnoreSslValidation
+ DependsOn:
+ - dsIsInstanceClearLambdaPermission
+ dsIsInstanceProtectedByRule:
+ Type: AWS::Config::ConfigRule
+ Properties:
+ Description: This rule checks with the Trend Micro Deep Security
+ Manager to see if the instance is protected using the specified security control.
+ Scope:
+ ComplianceResourceTypes:
+ - AWS::EC2::Instance
+ Source:
+ Owner: CUSTOM_LAMBDA
+ SourceIdentifier: !GetAtt dsIsInstanceProtectedByLambda.Arn
+ SourceDetails:
+ - EventSource: aws.config
+ MessageType: ConfigurationItemChangeNotification
+ - EventSource: aws.config
+ MessageType: OversizedConfigurationItemChangeNotification
+ InputParameters:
+ dsUsernameKey: !Ref DSUsernameKey
+ dsPasswordKey: !Ref DSPasswordKey
+ dsHostname: !Ref DSHostname
+ dsPort: !Ref DSPort
+ dsTenant: !Ref DSTenant
+ dsIgnoreSslValidation: !Ref DSIgnoreSslValidation
+ dsControl: !Ref DSControl
+ DependsOn:
+ - dsIsInstanceProtectedByLambdaPermission
+ dsIsInstanceProtectedByAntiMalwareRule:
+ Type: AWS::Config::ConfigRule
+ Properties:
+ Description: This rule checks with the Trend Micro Deep Security
+ Manager to see if the instance is protected using the anti-malware security
+ control.
+ Scope:
+ ComplianceResourceTypes:
+ - AWS::EC2::Instance
+ Source:
+ Owner: CUSTOM_LAMBDA
+ SourceIdentifier: !GetAtt dsIsInstanceProtectedByAntiMalwareLambda.Arn
+ SourceDetails:
+ - EventSource: aws.config
+ MessageType: ConfigurationItemChangeNotification
+ - EventSource: aws.config
+ MessageType: OversizedConfigurationItemChangeNotification
+ InputParameters:
+ dsUsernameKey: !Ref DSUsernameKey
+ dsPasswordKey: !Ref DSPasswordKey
+ dsHostname: !Ref DSHostname
+ dsPort: !Ref DSPort
+ dsTenant: !Ref DSTenant
+ dsIgnoreSslValidation: !Ref DSIgnoreSslValidation
+ DependsOn:
+ - dsIsInstanceProtectedByAntiMalwareLambdaPermission
+Outputs:
+ dsDoesInstanceHavePolicyLambda:
+ Description: ARN for the dsDoesInstanceHavePolicy lambda
+ Value: !GetAtt dsDoesInstanceHavePolicyLambda.Arn
+ dsIsInstanceClearLambda:
+ Description: ARN for the dsIsInstanceClearLambda lambda
+ Value: !GetAtt dsIsInstanceClearLambda.Arn
+ dsIsInstanceProtectedByLambda:
+ Description: ARN for the dsIsInstanceProtectedByLambda lambda
+ Value: !GetAtt dsIsInstanceProtectedByLambda.Arn
+ dsIsInstanceProtectedByAntiMalwareLambda:
+ Description: ARN for the dsIsInstanceProtectedByAntiMalwareLambda lambda
+ Value: !GetAtt dsIsInstanceProtectedByAntiMalwareLambda.Arn
diff --git a/deploy.config b/deploy.config
new file mode 100644
index 0000000..71afb00
--- /dev/null
+++ b/deploy.config
@@ -0,0 +1,35 @@
+
+# CFN stack name targeted for deployment
+STACK_NAME=my-stack
+
+# S3 Bucket and object prefix to upload Lambda source code to
+LAMBDA_BUCKET=my-lambda-bucket
+LAMBDA_PREFIX=my-lambda-object-prefix
+
+# S3 Bucket and object prefix where AWS Config stores artifacts
+CONFIG_BUCKET=my-config-bucket
+CONFIG_PREFIX=my-config-object-prefix
+
+# Deep Security connection properties
+# Note: username and password will be store in AWS Secret Manager
+DS_HOSTNAME=0.0.0.0
+DS_PORT=443
+DS_TENANT=
+DS_IGNORE_SSL_VALIDATION=true
+
+# Deep Security Config rules use SSM Parameter Store and KMS to
+# retrieve credentials securely
+# Below are Parameter Store keys for username and password
+DS_USERNAME_PARAM_STORE_KEY=my-ds-username-key
+DS_PASSWORD_PARAM_STORE_KEY=my-ds-password-key
+
+
+# Other Deep Security parameters
+
+# Policy to check for DoesInstanceHavePolicy rule
+DS_POLICY='my webserver policy'
+
+# Control to check for IsInstanceProtectedBy rule
+# Valid values are [ anti_malware, web_reputation, firewall,
+# intrusion_prevention, integrity_monitoring, log_inspection ]
+DS_CONTROL=firewall
diff --git a/deploy.sh b/deploy.sh
new file mode 100755
index 0000000..67bb3b0
--- /dev/null
+++ b/deploy.sh
@@ -0,0 +1,53 @@
+#!/usr/bin/env bash
+
+set -e
+
+source deploy.config
+
+if [[ -z "$STACK_NAME" || \
+ -z "$CONFIG_BUCKET" || \
+ -z "$CONFIG_PREFIX" || \
+ -z "$DS_USERNAME_PARAM_STORE_KEY" || \
+ -z "$DS_PASSWORD_PARAM_STORE_KEY" || \
+ -z "$DS_HOSTNAME" || \
+ -z "$DS_POLICY" || \
+ -z "$DS_CONTROL" ]]
+then
+ echo "" >&2
+ echo "Required parameters missing in 'deploy.config':" >&2
+ echo " - STACK_NAME" >&2
+ echo " - CONFIG_BUCKET" >&2
+ echo " - CONFIG_PREFIX" >&2
+ echo " - DS_USERNAME_PARAM_STORE_KEY" >&2
+ echo " - DS_PASSWORD_PARAM_STORE_KEY" >&2
+ echo " - DS_HOSTNAME" >&2
+ echo " - DS_POLICY" >&2
+ echo " - DS_CONTROL" >&2
+ echo "" >&2
+ exit 1
+fi
+
+./package.sh
+
+echo ""
+echo "Deploying Lambda functions and Config rules..."
+
+sam deploy \
+ --stack-name "$STACK_NAME" \
+ --template-file packaged.yml \
+ --capabilities CAPABILITY_IAM \
+ --parameter-overrides \
+ ConfigBucket="$CONFIG_BUCKET" \
+ ConfigPrefix="$CONFIG_PREFIX" \
+ DSUsernameKey="$DS_USERNAME_PARAM_STORE_KEY" \
+ DSPasswordKey="$DS_PASSWORD_PARAM_STORE_KEY" \
+ DSHostname="$DS_HOSTNAME" \
+ DSPort="$DS_PORT" \
+ DSTenant="$DS_TENANT" \
+ DSIgnoreSslValidation="$DS_IGNORE_SSL_VALIDATION" \
+ DSPolicy="$DS_POLICY" \
+ DSControl="$DS_CONTROL"
+
+./clean.sh
+
+echo ""
diff --git a/package.sh b/package.sh
new file mode 100755
index 0000000..653d8a9
--- /dev/null
+++ b/package.sh
@@ -0,0 +1,29 @@
+#!/usr/bin/env bash
+
+set -e
+
+source deploy.config
+
+if [[ -z "$LAMBDA_BUCKET" || \
+ -z "$LAMBDA_PREFIX" ]]
+then
+ echo "" >&2
+ echo "Required parameters missing in 'deploy.config':" >&2
+ echo " - LAMBDA_BUCKET" >&2
+ echo " - LAMBDA_PREFIX" >&2
+ echo "" >&2
+ exit 1
+fi
+
+./build.sh
+
+echo ""
+echo "Packaging Lambda functions..."
+
+sam package \
+ --template-file .aws-sam/build/template.yaml \
+ --s3-bucket "$LAMBDA_BUCKET" \
+ --s3-prefix "$LAMBDA_PREFIX" \
+ --output-template-file packaged.yml
+
+echo ""
diff --git a/publish.sh b/publish.sh
new file mode 100755
index 0000000..26209cf
--- /dev/null
+++ b/publish.sh
@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+
+set -e
+
+./package.sh
+
+echo ""
+echo "Publishing to AWS Serverless Application Repository..."
+
+sam publish --template packaged.yml
+
+./clean.sh
+
+echo ""
diff --git a/requirements-sam-cli.txt b/requirements-sam-cli.txt
new file mode 100644
index 0000000..46a2b29
--- /dev/null
+++ b/requirements-sam-cli.txt
@@ -0,0 +1,2 @@
+awscli==1.16.209
+aws-sam-cli==0.19.0
diff --git a/requirements-test.txt b/requirements-test.txt
new file mode 100644
index 0000000..ac5baf0
--- /dev/null
+++ b/requirements-test.txt
@@ -0,0 +1,5 @@
+pytest==5.0.1
+pytest-cov==2.7.1
+boto3==1.9.199
+xmltodict==0.10.1
+jsonpickle==1.2
diff --git a/requirements.txt b/requirements.txt
new file mode 100644
index 0000000..2d65c8c
--- /dev/null
+++ b/requirements.txt
@@ -0,0 +1 @@
+xmltodict==0.10.1
diff --git a/rules/ds-DoesInstanceHavePolicy/README.md b/rules/ds-DoesInstanceHavePolicy/README.md
new file mode 100755
index 0000000..cfe05a8
--- /dev/null
+++ b/rules/ds-DoesInstanceHavePolicy/README.md
@@ -0,0 +1,9 @@
+# ds-DoesInstanceHavePolicy
+
+## Dependencies
+
+This is dependent on the `deepsecurity` code stored in the `src` directory.
+
+## Deployment
+
+Run the `deploy.sh` script in this project's root directory.
diff --git a/rules/ds-DoesInstanceHavePolicy/dsDoesInstanceHavePolicy.py b/rules/ds-DoesInstanceHavePolicy/dsDoesInstanceHavePolicy.py
new file mode 100755
index 0000000..55aa5b4
--- /dev/null
+++ b/rules/ds-DoesInstanceHavePolicy/dsDoesInstanceHavePolicy.py
@@ -0,0 +1,28 @@
+# *********************************************************************
+# Deep Security - Does Instance Have Policy ______?
+# *********************************************************************
+
+# Standard library
+import logging
+logging.getLogger().setLevel(logging.INFO)
+
+from src.rules.rule_does_instance_have_policy import RuleDoesInstanceHavePolicy
+
+
+def aws_config_rule_handler(event, context):
+ try:
+ rule = RuleDoesInstanceHavePolicy(event)
+ response = rule.execute()
+ result = {
+ 'annotation': rule.compliance_msg,
+ 'response': response,
+ 'result': 'success'
+ }
+ except Exception as ex:
+ logging.error('Exception thrown: {}'.format(ex))
+ result = {
+ 'response': ex,
+ 'result': 'failure'
+ }
+
+ return result
diff --git a/rules/ds-IsInstanceClear/README.md b/rules/ds-IsInstanceClear/README.md
new file mode 100755
index 0000000..7f6032d
--- /dev/null
+++ b/rules/ds-IsInstanceClear/README.md
@@ -0,0 +1,9 @@
+# ds-IsInstanceClear
+
+## Dependencies
+
+This is dependent on the `deepsecurity` code stored in the `src` directory.
+
+## Deployment
+
+Run the `deploy.sh` script in this project's root directory.
diff --git a/rules/ds-IsInstanceClear/dsIsInstanceClear.py b/rules/ds-IsInstanceClear/dsIsInstanceClear.py
new file mode 100755
index 0000000..26689be
--- /dev/null
+++ b/rules/ds-IsInstanceClear/dsIsInstanceClear.py
@@ -0,0 +1,28 @@
+# *********************************************************************
+# Deep Security - Is Instance Clear?
+# *********************************************************************
+
+# Standard library
+import logging
+logging.getLogger().setLevel(logging.INFO)
+
+from src.rules.rule_is_instance_clear import RuleIsInstanceClear
+
+
+def aws_config_rule_handler(event, context):
+ try:
+ rule = RuleIsInstanceClear(event)
+ response = rule.execute()
+ result = {
+ 'annotation': rule.compliance_msg,
+ 'response': response,
+ 'result': 'success'
+ }
+ except Exception as ex:
+ logging.error('Exception thrown: {}'.format(ex))
+ result = {
+ 'response': ex,
+ 'result': 'failure'
+ }
+
+ return result
diff --git a/rules/ds-IsInstanceProtectedBy/README.md b/rules/ds-IsInstanceProtectedBy/README.md
new file mode 100755
index 0000000..eebfd63
--- /dev/null
+++ b/rules/ds-IsInstanceProtectedBy/README.md
@@ -0,0 +1,9 @@
+# ds-IsInstanceProtectedBy
+
+## Dependencies
+
+This is dependent on the `deepsecurity` code stored in the `src` directory.
+
+## Deployment
+
+Run the `deploy.sh` script in this project's root directory.
diff --git a/rules/ds-IsInstanceProtectedBy/dsIsInstanceProtectedBy.py b/rules/ds-IsInstanceProtectedBy/dsIsInstanceProtectedBy.py
new file mode 100755
index 0000000..570b44d
--- /dev/null
+++ b/rules/ds-IsInstanceProtectedBy/dsIsInstanceProtectedBy.py
@@ -0,0 +1,28 @@
+# *********************************************************************
+# Deep Security - Is Instance Protected By _______?
+# *********************************************************************
+
+# Standard library
+import logging
+logging.getLogger().setLevel(logging.INFO)
+
+from src.rules.rule_is_instance_protected_by import RuleIsInstanceProtectedBy
+
+
+def aws_config_rule_handler(event, context):
+ try:
+ rule = RuleIsInstanceProtectedBy(event)
+ response = rule.execute()
+ result = {
+ 'annotation': rule.compliance_msg,
+ 'response': response,
+ 'result': 'success'
+ }
+ except Exception as ex:
+ logging.error('Exception thrown: {}'.format(ex))
+ result = {
+ 'response': ex,
+ 'result': 'failure'
+ }
+
+ return result
diff --git a/rules/ds-IsInstanceProtectedByAntiMalware/README.md b/rules/ds-IsInstanceProtectedByAntiMalware/README.md
new file mode 100755
index 0000000..ad29dec
--- /dev/null
+++ b/rules/ds-IsInstanceProtectedByAntiMalware/README.md
@@ -0,0 +1,9 @@
+# ds-IsInstanceProtectedByAntiMalware
+
+## Dependencies
+
+This is dependent on the `deepsecurity` code stored in the `src` directory.
+
+## Deployment
+
+Run the `deploy.sh` script in this project's root directory.
diff --git a/rules/ds-IsInstanceProtectedByAntiMalware/dsIsInstanceProtectedByAntiMalware.py b/rules/ds-IsInstanceProtectedByAntiMalware/dsIsInstanceProtectedByAntiMalware.py
new file mode 100755
index 0000000..584e7d5
--- /dev/null
+++ b/rules/ds-IsInstanceProtectedByAntiMalware/dsIsInstanceProtectedByAntiMalware.py
@@ -0,0 +1,28 @@
+# *********************************************************************
+# Deep Security - Is Instance Protected By AntiMalware?
+# *********************************************************************
+
+# Standard library
+import logging
+logging.getLogger().setLevel(logging.INFO)
+
+from src.rules.rule_is_instance_protected_by_anti_malware import RuleIsInstanceProtectedByAntiMalware
+
+
+def aws_config_rule_handler(event, context):
+ try:
+ rule = RuleIsInstanceProtectedByAntiMalware(event)
+ response = rule.execute()
+ result = {
+ 'annotation': rule.compliance_msg,
+ 'response': response,
+ 'result': 'success'
+ }
+ except Exception as ex:
+ logging.error('Exception thrown: {}'.format(ex))
+ result = {
+ 'response': ex,
+ 'result': 'failure'
+ }
+
+ return result
diff --git a/src/README.md b/src/README.md
new file mode 100755
index 0000000..705f8db
--- /dev/null
+++ b/src/README.md
@@ -0,0 +1,5 @@
+# Source Folder
+
+This folder contains the python modules required by various rules to execute. For an AWS Lambda deployment, it's far easier to embed requirements within the function vs. installing during execution. This stabilizes the code base and ensures that execution is consistent.
+
+Each function requires its own copy of its pre-requisites.
diff --git a/src/__init__.py b/src/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/src/deepsecurity/__init__.py b/src/deepsecurity/__init__.py
new file mode 100755
index 0000000..d198ca6
--- /dev/null
+++ b/src/deepsecurity/__init__.py
@@ -0,0 +1,11 @@
+# make sure we add the current path structure to sys.path
+# this is required to import local dependencies
+import sys
+import os
+current_path = os.path.dirname(os.path.realpath(__file__))
+sys.path.append(current_path)
+
+# import project files as required
+import dsm
+import translation
+translation.Terms.read_terms_file()
\ No newline at end of file
diff --git a/src/deepsecurity/computers.py b/src/deepsecurity/computers.py
new file mode 100755
index 0000000..043c2be
--- /dev/null
+++ b/src/deepsecurity/computers.py
@@ -0,0 +1,303 @@
+# standard library
+import datetime
+
+# 3rd party libraries
+
+# project libraries
+import core
+
+class Computers(core.CoreDict):
+ def __init__(self, manager=None):
+ core.CoreDict.__init__(self)
+ self.manager = manager
+ self.log = self.manager.log if self.manager else None
+
+ def get(self, detail_level='HIGH', computer_id=None, computer_group_id=None, policy_id=None, computer_name=None, external_id=None, external_group_id=None):
+ """
+ Get all or a filtered set of computers from Deep Security
+
+ Can filter by:
+ computer_id
+ computer_group_id
+ policy_id
+ computer_name (specific or starts with using name*)
+ external_id
+ external_group_id
+
+ If multiple filters are requested, only one is applied. The priority is;
+ external_id
+ external_group_id
+ computer_name
+ computer_id
+ computer_group_id
+ policy_id
+
+ detail_level can be set to one of ['HIGH', 'MEDIUM', 'LOW']
+ """
+ # make sure we have a valid detail level
+ detail_level = detail_level.upper()
+ if not detail_level in ['HIGH', 'MEDIUM', 'LOW']: detail_level = 'HIGH'
+
+ call = None
+ if external_id and external_group_id:
+ call = self.manager._get_request_format(call='hostDetailRetrieveByExternal')
+ if external_id:
+ call['data'] = {
+ 'externalFilter': {
+ 'hostExternalID': external_id,
+ 'hostGroupExternalID': None,
+ },
+ 'hostDetailLevel': detail_level
+ }
+ elif external_group_id:
+ call['data'] = {
+ 'externalFilter': {
+ 'hostExternalID': None,
+ 'hostGroupExternalID': external_group_id,
+ },
+ 'hostDetailLevel': detail_level
+ }
+ elif computer_name:
+ if computer_name.endswith('*'):
+ call = self.manager._get_request_format(call='hostDetailRetrieveByNameStartsWith')
+ call['data'] = {
+ 'hostname': computer_name,
+ 'hostDetailLevel': detail_level
+ }
+ else:
+ call = self.manager._get_request_format(call='hostDetailRetrieveByName')
+ call['data'] = {
+ 'startsWithHostname': computer_name.rstrip('*'),
+ 'hostDetailLevel': detail_level
+ }
+ else:
+ # get with no arguments = hostRetrieve() with ALL_HOSTS
+ call = self.manager._get_request_format(call='hostDetailRetrieve')
+ if computer_id:
+ call['data'] = {
+ 'hostFilter': {
+ 'hostGroupID': None,
+ 'hostID': computer_id,
+ 'securityProfileID': None,
+ 'type': 'ALL_HOSTS',
+ },
+ 'hostDetailLevel': detail_level
+ }
+ elif computer_group_id:
+ call['data'] = {
+ 'hostFilter': {
+ 'hostGroupID': computer_group_id,
+ 'hostID': None,
+ 'securityProfileID': None,
+ 'type': 'ALL_HOSTS',
+ },
+ 'hostDetailLevel': detail_level
+ }
+ elif policy_id:
+ call['data'] = {
+ 'hostFilter': {
+ 'hostGroupID': None,
+ 'hostID': None,
+ 'securityProfileID': policy_id,
+ 'type': 'ALL_HOSTS',
+ },
+ 'hostDetailLevel': detail_level
+ }
+ else:
+ call['data'] = {
+ 'hostFilter': {
+ 'hostGroupID': None,
+ 'hostID': None,
+ 'securityProfileID': None,
+ 'type': 'ALL_HOSTS',
+ },
+ 'hostDetailLevel': detail_level
+ }
+
+ response = self.manager._request(call)
+
+ if response and response['status'] == 200:
+ if not type(response['data']) == type([]): response['data'] = [response['data']]
+ for computer in response['data']:
+ computer_obj = None
+ try:
+ computer_obj = Computer(self.manager, computer, self.log)
+ except Exception as err:
+ self.log("Could not create Computer from API response")
+ if computer_obj:
+ self[computer_obj.id] = computer_obj
+ self.log("Added Computer {}".format(computer_obj.id), level='debug')
+
+ try:
+ # add this computer to any appropriate groups on the Manager()
+ if 'computer_group_id' in dir(computer_obj) and computer_obj.computer_group_id:
+ if self.manager.computer_groups and computer_obj.computer_group_id in self.manager.computer_groups:
+ self.manager.computer_groups[computer_obj.computer_group_id].computers[computer_obj.id] = computer_obj
+ self.log("Added Computer {} to ComputerGroup {}".format(computer_obj.id, computer_obj.computer_group_id), level='debug')
+ except Exception as hostGroupid_err:
+ self.log("Could not add Computer {} to ComputerGroup".format(computer_obj.id), err=hostGroupid_err)
+
+ try:
+ # add this computer to any appropriate policies on the Manager()
+ if 'policy_id' in dir(computer_obj) and computer_obj.policy_id:
+ if self.manager.policies and computer_obj.policy_id in self.manager.policies:
+ self.manager.policies[computer_obj.policy_id].computers[computer_obj.id] = computer_obj
+ self.log("Added Computer {} to Policy {}".format(computer_obj.id, computer_obj.policy_id), level='debug')
+ except Exception as securityProfileid_err:
+ self.log("Could not add Computer {} to Policy".format(computer_obj.id), err=securityProfileid_err)
+
+ return len(self)
+
+class ComputerGroups(core.CoreDict):
+ def __init__(self, manager=None):
+ core.CoreDict.__init__(self)
+ self.manager = manager
+ self._exempt_from_find.append('computers')
+ self.computers = core.CoreDict()
+ self.log = self.manager.log if self.manager else None
+
+ def get(self, name=None, group_id=None):
+ """
+ Get all or a filtered set of computer groups from Deep Security
+
+ If a name or group_id is specified, will only retrieve the
+ computer groups matching the name or group_id.
+
+ If both are specified, name takes priority
+ """
+ call = None
+ if name or group_id:
+ # filtered call
+ if name:
+ call = self.manager._get_request_format(call='hostGroupRetrieveByName')
+ call['data'] = {
+ 'Name': name
+ }
+ elif group_id:
+ call = self.manager._get_request_format(call='hostGroupRetrieve')
+ call['data'] = {
+ 'id': '{}'.format(group_id)
+ }
+ else:
+ call = self.manager._get_request_format(call='hostGroupRetrieveAll')
+
+ response = self.manager._request(call)
+ if response and response['status'] == 200:
+ self.clear() # empty the current groups
+ if not type(response['data']) == type([]): response['data'] = [response['data']]
+ for group in response['data']:
+ computer_group_obj = ComputerGroup(self.manager, group, self.log)
+ if computer_group_obj:
+ self[computer_group_obj.id] = computer_group_obj
+ self.log("Added ComputerGroup {}".format(computer_group_obj.id), level='debug')
+
+ return len(self)
+
+class Computer(core.CoreObject):
+ def __init__(self, manager=None, api_response=None, log_func=None):
+ self.manager = manager
+ self.recommended_rules = None
+ if api_response: self._set_properties(api_response, log_func)
+
+ if not ('id') in dir(self): raise Exception("Could not create Computer from response")
+
+ def send_events(self):
+ """
+ Send the latest set of events to this computer's Manager
+ """
+ return self.manager.request_events_from_computer(self.id)
+
+ def clear_alerts_and_warnings(self):
+ """
+ Clear any alerts or warnings for the computer
+ """
+ return self.manager.clear_alerts_and_warnings_from_computers(self.id)
+
+ def scan_for_malware(self):
+ """
+ Request a malware scan be run on the computer
+ """
+ return self.manager.scan_computers_for_malware(self.id)
+
+ def scan_for_integrity(self):
+ """
+ Request an integrity scan be run on the computer
+ """
+ return self.manager.scan_computers_for_integrity(self.id)
+
+ def scan_for_recommendations(self):
+ """
+ Request a recommendation scan be run on the computer
+ """
+ return self.manager.scan_computers_for_recommendations(self.id)
+
+ def assign_policy(self, policy_id):
+ """
+ Assign the specified policy to the computer
+ """
+ return self.manager.assign_policy_to_computers(policy_id, self.id)
+
+ def get_recommended_rules(self):
+ """
+ Recommend a set of rules to apply to the computer
+ """
+ self.recommended_rules = self.manager.get_rule_recommendations_for_computer(self.id)
+ return self.recommended_rules['total_recommedations']
+
+class ComputerGroup(core.CoreObject):
+ def __init__(self, manager=None, api_response=None, log_func=None):
+ self.manager = manager
+ if api_response: self._set_properties(api_response, log_func)
+ self.computers = core.CoreDict()
+
+ def send_events(self):
+ """
+ Send the latest set of events for all computers in this group
+ """
+ results = {}
+ for computer_id, computer in self.computers.items():
+ if 'send_events' in dir(computer):
+ results[computer_id] = computer.send_events()
+
+ return results
+
+ def clear_alerts_and_warnings(self):
+ """
+ Clear any alerts or warnings for all computers in this group
+ """
+ return self.manager.clear_alerts_and_warnings_from_computers(self.computers.keys())
+
+ def scan_for_malware(self):
+ """
+ Request a malware scan be run on all computers in this group
+ """
+ return self.manager.scan_computers_for_malware(self.computers.keys())
+
+ def scan_for_integrity(self):
+ """
+ Request an integrity scan be run on all computers in this group
+ """
+ return self.manager.scan_computers_for_integrity(self.computers.keys())
+
+ def scan_for_recommendations(self):
+ """
+ Request a recommendation scan be run on all computers in this group
+ """
+ return self.manager.scan_computers_for_recommendations(self.computers.keys())
+
+ def assign_policy(self, policy_id):
+ """
+ Assign the specified policy to all computers in this group
+ """
+ return self.manager.assign_policy_to_computers(policy_id, self.computers.keys())
+
+ def get_recommended_rules(self):
+ """
+ Recommend a set of rules to apply for each computer in this group
+ """
+ results = {}
+
+ for computer_id in self.computers.keys():
+ results[computer_id] = self.computers[computer_id].get_recommended_rules()
+
+ return results
\ No newline at end of file
diff --git a/src/deepsecurity/core.py b/src/deepsecurity/core.py
new file mode 100755
index 0000000..4090df8
--- /dev/null
+++ b/src/deepsecurity/core.py
@@ -0,0 +1,558 @@
+# standard library
+import collections
+import json
+import logging
+import re
+import ssl
+from urllib.request import build_opener, HTTPSHandler, Request
+from urllib.parse import urlencode
+
+# 3rd party libraries
+import xmltodict
+
+# project libraries
+import translation
+
+class CoreApi(object):
+ def __init__(self):
+ self.API_TYPE_REST = 'REST'
+ self.API_TYPE_SOAP = 'SOAP'
+ self._rest_api_endpoint = ''
+ self._soap_api_endpoint = ''
+ self._sessions = { self.API_TYPE_REST: None, self.API_TYPE_SOAP: None }
+ self.ignore_ssl_validation = False
+ self._log_at_level = logging.WARNING
+ self.logger = self._set_logging()
+
+ # *******************************************************************
+ # properties
+ # *******************************************************************
+ @property
+ def log_at_level(self): return self._log_at_level
+
+ @log_at_level.setter
+ def log_at_level(self, value):
+ """
+ Make sure logging is always set at a valid level
+ """
+ if value in [
+ logging.CRITICAL,
+ logging.DEBUG,
+ logging.ERROR,
+ logging.FATAL,
+ logging.INFO,
+ logging.WARNING,
+ ]:
+ self._log_at_level = value
+ self._set_logging()
+ else:
+ if not self._log_at_level:
+ self._log_at_level = logging.WARNING
+ self._set_logging()
+
+ # *******************************************************************
+ # methods
+ # *******************************************************************
+ def _set_logging(self):
+ """
+ Setup the overall logging environment
+ """
+ # Based on tips from http://www.blog.pythonlibrary.org/2012/08/02/python-101-an-intro-to-logging/
+ logging.basicConfig(level=self.log_at_level)
+
+ # setup module logging
+ logger = logging.getLogger("DeepSecurity.API")
+ logger.setLevel(self.log_at_level)
+
+ # reset any existing handlers
+ logging.root.handlers = [] # @TODO evaluate impact to other modules
+ logger.handlers = []
+
+ # add the desired handler
+ formatter = logging.Formatter('[%(asctime)s]\t%(message)s', '%Y-%m-%d %H:%M:%S')
+ stream_handler = logging.StreamHandler()
+ stream_handler.setFormatter(formatter)
+ logger.addHandler(stream_handler)
+
+ return logger
+
+ def _get_request_format(self, api=None, call=None, use_cookie_auth=False):
+ if not api: api = self.API_TYPE_SOAP
+ return {
+ 'api': api,
+ 'call': call,
+ 'use_cookie_auth': use_cookie_auth,
+ 'query': None,
+ 'data': None,
+ }
+
+ def _request(self, request, auth_required=True):
+ """
+ Make an HTTP(S) request to an API endpoint based on what's specified in the
+ request object passed
+
+ ## Input
+
+ Required request keys:
+ api
+ Either REST or SOAP
+
+ call
+ Name of the SOAP method or relative path of the REST URL
+
+ Optional keys:
+ query
+ Contents of the query string passed as a dict
+
+ data
+ Data to post. For SOAP API calls this will be the SOAP envelope. For
+ REST API calls this will be a dict converted to JSON automatically
+ by this method
+
+ use_cookie_auth
+ Whether or not to use an HTTP Cookie in lieu of a querystring for authorization
+
+ ## Output
+
+ Returns a dict:
+ status
+ Number HTTP status code returned by the response, if any
+
+ raw
+ The raw contents of the response, if any
+
+ data
+ A python dict representing the data contained in the response, if any
+ """
+ for required_key in [
+ 'api',
+ 'call'
+ ]:
+ if not (required_key in request and request[required_key]):
+ self.log("All requests are required to have a key [{}] with a value".format(required_key), level='critical')
+ return None
+
+ url = None
+ if request['api'] == self.API_TYPE_REST:
+ url = "{}/{}".format(self._rest_api_endpoint, request['call'].lstrip('/'))
+ else:
+ url = self._soap_api_endpoint
+
+ self.log("Making a request to {}".format(url), level='debug')
+
+ # add the authentication parameters
+ if auth_required:
+ if request['api'] == self.API_TYPE_REST:
+ if not request['use_cookie_auth']: # sID is a query string
+ if not request['query']: request['query'] = {}
+ request['query']['sID'] = self._sessions[self.API_TYPE_REST]
+ elif request['api'] == self.API_TYPE_SOAP:
+ # sID is part of the data
+ if not request['data']: request['data'] = {}
+ request['data']['sID'] = self._sessions[self.API_TYPE_SOAP]
+
+ # remove any blank request keys
+ for k, v in request.items():
+ if not v: request[k] = None
+
+ # prep the query string
+ if 'query' in request and request['query']:
+ # get with query string
+ qs = {}
+ for k, v in request['query'].items(): # strip out null entries
+ if v: qs[k] = v
+
+ url += '?%s' % urlencode(qs)
+ self.log("Added query string. Full URL is now {}".format(url), level='debug')
+
+ self.log("URL to request is: {}".format(url))
+
+ # Prep the SSL context
+ ssl_context = ssl.create_default_context()
+ if self.ignore_ssl_validation:
+ ssl_context.check_hostname = False
+ ssl_context.verify_mode = ssl.CERT_NONE
+ self.log("SSL certificate validation has been disabled for this call", level='warning')
+
+ # Prep the URL opener
+ url_opener = build_opener(HTTPSHandler(context=ssl_context))
+
+ # Prep the request
+ request_type = 'GET'
+ headers = {
+ 'Accept': 'application/json,text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*',
+ 'Content-Type': 'application/json',
+ }
+
+ # authentication calls don't accept the Accept header
+ if request['call'].startswith('authentication'): del(headers['Accept'])
+
+ # some rest calls use a cookie to pass the sID
+ if request['api'] == self.API_TYPE_REST and request['use_cookie_auth']:
+ headers['Cookie'] = 'sID="{}"'.format(self._sessions[self.API_TYPE_REST])
+
+ if request['api'] == self.API_TYPE_REST and request['call'] in [
+ 'apiVersion',
+ 'status/manager/ping'
+ ]:
+ headers = {
+ 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*',
+ 'Content-Type': 'text/plain',
+ }
+
+ if request['api'] == self.API_TYPE_SOAP:
+ # always a POST
+ headers = {
+ 'SOAPAction': '',
+ 'content-type': 'application/soap+xml'
+ }
+ data = self._prep_data_for_soap(request['call'], request['data'])
+ url_request = Request(url, data=data, headers=headers)
+ request_type = 'POST'
+ self.log("Making a SOAP request with headers {}".format(headers), level='debug')
+ self.log(" and data {}".format(data), level='debug')
+ elif request['call'] == 'authentication/logout':
+ url_request = Request(url, headers=headers)
+ setattr(url_request, 'get_method', lambda: 'DELETE') # make this request use the DELETE HTTP verb
+ request_type = 'DELETE'
+ self.log("Making a REST DELETE request with headers {}".format(headers), level='debug')
+ elif 'data' in request and request['data']:
+ # POST
+ url_request = Request(url, data=json.dumps(request['data']).encode('utf-8'), headers=headers)
+ request_type = 'POST'
+ self.log("Making a REST POST request with headers {}".format(headers), level='debug')
+ self.log(" and data {}".format(request['data']), level='debug')
+ else:
+ # GET
+ url_request = Request(url, headers=headers)
+ self.log("Making a REST GET request with headers {}".format(headers), level='debug')
+
+ # Make the request
+ response = None
+ try:
+ response = url_opener.open(url_request)
+ except Exception as url_err:
+ self.log("Failed to make {} {} call [{}]".format(request['api'].upper(), request_type, request['call'].lstrip('/')), err=url_err)
+
+ # Convert the request from JSON
+ result = {
+ 'status': response.getcode() if response else None,
+ 'raw': response.read() if response else None,
+ 'headers': dict(response.headers) if response else dict(),
+ 'data': None
+ }
+ bytes_of_data = len(result['raw']) if result['raw'] else 0
+ self.log("Call returned HTTP status {} and {} bytes of data".format(result['status'], bytes_of_data), level='debug')
+
+ if response:
+ if request['api'] == self.API_TYPE_SOAP:
+ # XML response
+ try:
+ if result['raw']:
+ full_data = xmltodict.parse(result['raw'])
+ if 'soapenv:Envelope' in full_data and 'soapenv:Body' in full_data['soapenv:Envelope']:
+ result['data'] = full_data['soapenv:Envelope']['soapenv:Body']
+ if '{}Response'.format(request['call']) in result['data']:
+ if '{}Return'.format(request['call']) in result['data']['{}Response'.format(request['call'])]:
+ result['data'] = result['data']['{}Response'.format(request['call'])]['{}Return'.format(request['call'])]
+ else:
+ result['data'] = result['data']['{}Response'.format(request['call'])]
+ else:
+ result['data'] = full_data
+ except Exception as xmltodict_err:
+ self.log("Could not convert response from call {}".format(request['call']), err=xmltodict_err)
+ else:
+ # JSON response
+ try:
+ if result['raw'] and result['status'] != 204:
+ result['type'] = result['headers']['content-type']
+ result['data'] = json.loads(result['raw']) if 'json' in result['type'] else None
+ except Exception as json_err:
+ # report the exception as 'info' because it's not fatal and the data is
+ # still captured in result['raw']
+ self.log("Could not convert response from call {} to JSON. Threw exception:\n\t{}".format(request['call'], json_err), level='info')
+
+ return result
+
+ def _prefix_keys(self, prefix, d):
+ """
+ Add the specified XML namespace prefix to all keys in the
+ passed dict
+ """
+ if not type(d) == type({}): return d
+ new_d = d.copy()
+ for k,v in d.items():
+ new_key = u"{}:{}".format(prefix, k)
+ new_v = v
+ if type(v) == type({}): new_v = self._prefix_keys(prefix, v)
+ new_d[new_key] = new_v
+ del(new_d[k])
+
+ return new_d
+
+ def _prep_data_for_soap(self, call, details):
+ """
+ Prepare the complete XML SOAP envelope
+ """
+ data = xmltodict.unparse(self._prefix_keys('ns1', { call: details }), pretty=False, full_document=False)
+ soap_xml = """
+
+