Cloud Scanner

Ensuring Cloud Resource Compliance

The Cloud Scanner automatically assesses your cloud resources for compliance with predefined policies, helping you maintain security and best practices across your cloud environments.

It integrates seamlessly with your Kubernetes cluster, leveraging powerful tools like AWS Cloud Control API and Kyverno JSON policies to simplify compliance management.

Purpose

The Cloud Scanner proactively identifies and reports on policy violations within your cloud infrastructure.

This helps you:

Enforce security best practices: Ensure your cloud resources adhere to organizational security standards.
Maintain compliance: Meet regulatory requirements and industry best practices within your cloud environment.
Prevent misconfigurations: Detect potentially harmful cloud configurations before they impact your applications.
Gain visibility: Obtain a comprehensive overview of your cloud resource compliance status.

Why You Need Cloud Scanner

Manual cloud compliance checks are time-consuming, error-prone, and difficult to scale.

The Cloud Scanner automates this process, providing:

Proactive monitoring: Continuous scanning ensures your cloud environment stays compliant.
Automated reporting: Detailed, actionable reports help you quickly address any compliance violations.
Reduced risk: Early detection of misconfigurations minimizes security vulnerabilities in your cloud environment.
Improved efficiency: Automation frees up valuable time for other critical tasks.

How it Works

The Cloud Scanner operates within your Kubernetes cluster and leverages Kyverno JSON policies to evaluate your cloud resources.

For AWS, it uses the AWS Cloud Control API to interact with your AWS infrastructure. Here’s a simplified workflow:

Configuration: Define the scope of the AWS scan by specifying your account details, regions, and services to target using the AWSAccountConfig custom resource. This includes providing necessary credentials securely, such as assuming an IAM role.
Resource Retrieval: The scanner uses the AWS Cloud Control API to fetch the resources defined in your AWSAccountConfig.
Policy Evaluation: The fetched AWS resources are evaluated against predefined Kyverno JSON policies, which define the compliance rules.
Reporting: The scanner generates ClusterEphemeralReports intermediary resources for further processing by the Reports Controller. These reports summarize the compliance status of your AWS resources. They highlight any violations, providing information on the affected resources and the specific policies that were violated. You can easily access these reports within your Kubernetes cluster.

NOTE: Currently, only AWS Cloud is supported. Support for other cloud providers is planned for future releases.

AWS Cloud

Leveraging the AWS Cloud Control API

The Cloud Scanner utilizes the AWS Cloud Control API to fetch resource information so you can use the aws cloudcontrol command to explore resource structures, aiding in crafting precise compliance policies.

Fetching Resource Data with the AWS CLI

To retrieve resource data, you can use the AWS Cloud Control CLI. The aws cloudcontrol list-resources command provides a list of resources for a given service.

For example, to list ECS clusters:

aws cloudcontrol list-resources --type-name AWS::ECS::Cluster

This returns a JSON payload with summary information for each cluster:

{
    "ResourceDescriptions": [
        {
            "Identifier": "bad-cluster",
            "Properties": "{\"ClusterSettings\":[],\"DefaultCapacityProviderStrategy\":[],\"CapacityProviders\":[],\"ClusterName\":\"bad-cluster\",\"Arn\":\"arn:aws:ecs:us-east-1:844333597536:cluster/bad-cluster\",\"Tags\":[]}"
        },
        {
            "Identifier": "another-cluster",
            "Properties": "{\"ClusterSettings\":[],\"DefaultCapacityProviderStrategy\":[],\"CapacityProviders\":[],\"ClusterName\":\"another-cluster\",\"Arn\":\"arn:aws:ecs:us-east-1:844333597536:cluster/another-cluster\",\"Tags\":[]}"
        }
    ],
    "TypeName": "AWS::ECS::Cluster"
}

To retrieve the full details for a specific resource, use the aws cloudcontrol get-resource command.

For example, to get the detailed configuration of the “bad-cluster”:

aws cloudcontrol get-resource --type-name AWS::ECS::Cluster --identifier bad-cluster

This returns a more comprehensive JSON payload for the specified resource:

{
    "TypeName": "AWS::ECS::Cluster",
    "ResourceDescription": {
        "Identifier": "bad-cluster",
        "Properties": "{\"ClusterSettings\":[{\"Value\":\"disabled\",\"Name\":\"containerInsights\"}],\"DefaultCapacityProviderStrategy\":[],\"CapacityProviders\":[],\"ClusterName\":\"bad-cluster\",\"Arn\":\"arn:aws:ecs:us-east-1:844333597536:cluster/bad-cluster\",\"Tags\":[]}"
    }
}

Preprocessing the Payload

The raw payload retrieved from the AWS Cloud Control API is processed using Preprocessors to create a structured format suitable for policy evaluation.

Preprocessors extract relevant metadata like service, region, resource type, and identifier, combining it with the resource properties to create a final payload.

For example, an ECS Cluster Preprocessor might look like this:

apiVersion: nirmata.io/v1alpha1
kind: Preprocessor
metadata:
  name: ecs-cluster-preprocessor
spec:
  match:
    provider: AWS
    service: ECS
    resource: Cluster
  scan:
    arnPath: Arn
    payloadPath: "@"

This preprocessor extracts the ARN and the entire resource properties, transforming the raw data into a structured payload like this:

metadata:
  provider: AWS
  service: ecs
  region: us-east-1
  identifier: bad-cluster
payload:
  clusterSettings:
    - name: containerInsights
      value: disabled
  defaultCapacityProviderStrategy: []
  capacityProviders: []
  clusterName: bad-cluster
  arn: arn:aws:ecs:us-east-1:844333597536:cluster/bad-cluster
  tags: []

Writing ValidatingPolicies

Using the structured payload generated by the Preprocessor, you can write precise ValidatingPolicies. These policies can target specific resources based on their metadata and enforce compliance rules based on the payload.

For example, a policy ensuring all ECS clusters have a group tag might look like:

apiVersion: nirmata.io/v1alpha1
kind: ValidatingPolicy
metadata:
  name: check-ecs-cluster-tags
spec:
  scan: true
  rules:
    - name: check-tags
      identifier: payload.clusterName
      match:
        all:
        - (metadata.provider): "AWS"
        - (metadata.region): us-east-1
        - (metadata.service): "ecs"
        - (metadata.resource): "Cluster"
      assert:
        all:
        - message: A 'group' tag is required
          check:
            payload:
              (tags[?key=='group'] || `[]`):
                (length(@) > `0`): true

This policy checks that all ECS clusters in the us-east-1 region have a group tag. If a cluster is missing this tag, the policy will report a violation.

In order to apply this policy when scanning AWS resources, you need to set the field spec.scan to true in the policy definition. Otherwise, the policy will not be evaluated during the scan.

AWS Scan Configuration

To configure the Cloud Scanner to scan AWS, you need to define an AWSAccountConfig custom resource. This resource specifies the AWS account details, regions, and services to target.

For example, to scan all ECS clusters in the us-east-1 region, you would create an AWSAccountConfig like this:

apiVersion: nirmata.io/v1alpha1
kind: AWSAccountConfig
metadata:
  name: aws-account-config
spec:
  # The `scanInterval` field controls how often the scanner runs to scan the services configured in this resource.
  scanInterval: 2h
  # The `accountID` field specifies the AWS account ID to scan.
  accountID: "YOUR_AWS_ACCOUNT_ID"
  # The `accountName` field specifies the AWS account name.
  accountName: "YOUR_AWS_ACCOUNT_NAME"
  # The `regions` field specifies the AWS regions to scan.
  regions:
    - us-east-1
  # The `services` field specifies the AWS services to scan.
  services:
    - ECS

The spec.scanInterval field in AWSAccountConfig is optional. If omitted, it defaults to the value specified by the --scanInterval flag which is 1 hour by default.

In the provided example configuration, the Cloud Scanner operates on a two-hour cycle. Every two hours, the following steps occur:

Resource Discovery: The scanner queries AWS for all ECS clusters within the us-east-1 region.
Policy Filtering: It identifies all ValidatingPolicies where the spec.scan field is set to true. These are the policies that will be applied to the discovered ECS clusters.
Policy Evaluation: Each discovered ECS cluster is evaluated against the filtered policies. This determines whether the cluster complies with each policy’s rules.
Reporting: The scanner generates ClusterEphemeralReports resources. These intermediate reports contain the results of the policy evaluations for each ECS cluster in the us-east-1 region. The Reports Controller will then process, and consolidates them into ClusterPolicyReports. These final reports provide a comprehensive overview of the compliance status of your ECS clusters, making it easier to identify and address any policy violations. These reports are accessible within your Kubernetes cluster.

AWS Cross-Account Scanning

The Cloud Scanner supports scanning AWS resources across multiple accounts. This is useful for organizations with multiple AWS accounts that need to enforce compliance policies across all accounts.

To enable cross-account scanning, you need to assume an IAM role in each account that grants the necessary permissions to the Cloud Scanner.

Prerequisites:

An EKS cluster with the Cloud Controller Helm chart deployed.
An IAM role named cloud-scanner-source in your source AWS account (where the Cloud Controller runs). This role needs permissions to assume roles in your target accounts (the accounts you want to scan).

High-Level Overview:

The process involves setting up IAM roles and trust relationships between your source and target accounts, then configuring the Cloud Scanner to use those roles for access.

Here is a breakdown of the steps involved:

Configure your target accounts:

For each AWS account you want to scan (your target accounts), you will need to create an IAM role. We will refer to this role as cloud-scanner-target-role in these instructions, but you can choose a different name.
- Create the cloud-scanner-target-role: In your target account’s IAM console, create a new role. Choose “Another AWS account” as the trusted entity. In the “Account ID” field, enter your source account ID (the account where your EKS cluster and Cloud Controller are running).
- Define the Trust Policy: The trust policy defines which accounts can assume this role. Use the following policy document, replacing YOUR_SOURCE_ACCOUNT_ID with your actual source account ID:
```
{
      "Version": "2012-10-17",
      "Statement": [
          {
              "Sid": "Statement1",
              "Effect": "Allow",
              "Principal": {
                  "AWS": "arn:aws:iam::YOUR_SOURCE_ACCOUNT_ID:role/cloud-scanner-source"
              },
              "Action": [
                  "sts:AssumeRole",
                  "sts:TagSession"
              ]
          }
      ]
}
```
- Grant Permissions: Attach managed policies or create a custom policy to grant the cloud-scanner-target-role the necessary permissions to access the resources you want to scan.
  
  Follow the principle of least privilege: grant only the minimum required permissions. For example, To scan ECS services, attach the AmazonECS_FullAccess policy.
  
  NOTE: Every target account must have this role configured, but the permissions attached will vary based on the services being scanned.
  
  You must attach the AWSCloudFormationFullAccess policy as it is required for the scanner to function correctly.
Configure your source account:
- Grant sts:AssumeRole Permission: In your source account, modify the IAM policy attached to the cloud-scanner-source role. Add a statement granting permission to assume the cloud-scanner-target-role in your target accounts. Replace YOUR_TARGET_ACCOUNT_ID with the actual account ID of the target account:
```
{
    "Sid": "Statement2",
    "Effect": "Allow",
    "Action": "sts:AssumeRole",
    "Resource": "arn:aws:iam::YOUR_TARGET_ACCOUNT_ID:role/cloud-scanner-target-role"
}
```
  Repeat this step for each target account, adding a separate statement for each cloud-scanner-target-role ARN.
Define AWSAccountConfig:

For each target account, create an AWSAccountConfig that specifies the roleARN field. This field should contain the ARN of the IAM role you created in the target AWS account.

For example, to scan ECS clusters in the us-east-1 region across multiple AWS accounts, you would create an AWSAccountConfig like this:
```
apiVersion: nirmata.io/v1alpha1
kind: AWSAccountConfig
metadata:
  name: aws-account-config
spec:
  scanInterval: 2h
  accountID: "AWS_TARGET_ACCOUNT_ID"
  accountName: "AWS_TARGET_ACCOUNT_NAME"
  regions:
    - us-east-1
  services:
    - ECS
  roleARN: "arn:aws:iam::AWS_TARGET_ACCOUNT_ID:role/cloud-scanner-target-role"
```
In this configuration, the Cloud Scanner will assume the IAM role cloud-scanner-target-role in the target AWS account to scan ECS clusters in the us-east-1 region.