Export findings to S3

Export scan data generated by Endor Labs to an AWS S3 storage bucket. This enables long-term data retention for compliance requirements, integration with security information and event management (SIEM) systems, and custom analytics workflows. The export framework supports exporting findings in JSON or SARIF format, allowing flexible integration with your existing toolchain.

Amazon S3 is an object storage service provided by Amazon Web Services (AWS). It offers high durability, availability, and scalability for storing and retrieving any amount of data. S3 integrates with other AWS services and third-party tools, making it ideal for data archival, backup, and analytics workflows.

Prerequisites

Ensure that you meet the following prerequisites before exporting data to S3:

• An AWS account with permissions to create IAM roles, identity providers, and S3 buckets.
• An S3 bucket to store the exported data. See Create an S3 bucket.
• An OIDC identity provider configured to allow Endor Labs access. See Add an OIDC identity provider.
• An IAM role with permissions for Endor Labs to write to your S3 bucket. See Create an IAM role.
• Download and install endorctl. See Install endorctl.

Set up AWS for S3 export

Endor Labs uses OIDC federation to assume an IAM role in your AWS account to access the S3 bucket. Complete the following steps in the AWS Management Console to configure access.

1. Create an S3 bucket

An S3 bucket is a container for storing objects in Amazon S3. Each bucket has a globally unique name and is created in a specific AWS region.

You can create a new general purpose S3 bucket or reuse an existing one to store the exported data. Disable ACLs on the bucket to ensure all access is managed through IAM policies and bucket policies, preventing unintended public access. Refer to Creating a bucket in the Amazon S3 documentation for instructions.

Configure bucket lifecycle

You can configure S3 lifecycle rules to automatically delete exported data after a specified retention period. Exported objects do not expire unless you configure lifecycle rules.

1. In the AWS management console, navigate to Amazon S3 > Buckets.
2. Select your bucket.
3. Select Management and click Create lifecycle rule.
4. Enter a Lifecycle rule name, for example, endor-exports-expiry.
5. Under Filter type, select Limit the scope of this rule using one or more filters and enter endor/ as the prefix to apply the rule only to exported data.
6. Under Lifecycle rule actions, select Expire current versions of objects.
7. Under Expire current versions of objects, enter the number of days after which objects should be deleted.
8. Review the rule and click Create rule.

2. Add an OIDC identity provider

OpenID Connect (OIDC) federation allows Endor Labs to access AWS resources without requiring long-lived credentials, reducing the risk of credential exposure and simplifying secret rotation.

1. In the AWS management console, navigate to IAM > Access Management > Identity providers.
2. Click Add provider.
3. Under Provider details, select OpenID Connect.
4. For Provider URL, enter https://api.endorlabs.com.
5. For Audience, specify a unique identifier to validate incoming OIDC tokens from Endor Labs.
6. Optionally, add tags to help identify the provider.
7. Click Add provider.

3. Create an IAM role

Create an IAM role that Endor Labs can assume to write to your S3 bucket. This involves:

1. Create a permissions policy: Define the S3 write permissions.
2. Create an IAM role: Create a role with OIDC trust and attach the policy.

Create a permissions policy

1. In the AWS management console, navigate to IAM > Access Management > Policies.

2. Click Create policy.

3. Under Specify permissions, toggle the Policy editor to JSON.

4. Enter the following policy:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "s3:PutObject",
         ],
         "Resource": "arn:aws:s3:::<your-bucket-name>/*"
       }
     ]
   }
   

   Replace <your-bucket-name> with the name of your S3 bucket.

5. Click Next.

6. Under Review and create, enter a Policy name. For example, EndorLabsS3ExportPolicy.

7. Review the Permissions defined in this policy section to confirm that the expected Amazon S3 write actions are included.

8. Optionally, add a description and tags to your policy.

9. Click Create policy.

Create an IAM role

1. In the AWS management console, navigate to IAM > Access Management > Roles.
2. Click Create role.
3. Under Select trusted entity, select Custom trust policy.
4. Enter the following trust policy:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Sid": "EndorWebIdentity",
         "Effect": "Allow",
         "Principal": {
           "Federated": "arn:aws:iam::<aws-account-id>:oidc-provider/api.endorlabs.com"
         },
         "Action": "sts:AssumeRoleWithWebIdentity",
         "Condition": {
           "StringEquals": {
             "api.endorlabs.com:aud": "<oidc-audience>"
           },
           "StringLike": {
             "api.endorlabs.com:sub": [ "<your-namespace>/*", "<your-namespace>.*" ]
           }
         }
       }
     ]
   }
   

   Replace the placeholders with your values:
   • <aws-account-id>: Your AWS account ID
   • <oidc-audience>: The audience value you configured in the OIDC provider
   • <your-namespace>: Your Endor Labs namespace

5. Click Next.
6. Under Add permissions, search for and select the IAM policy you created.

7. Click Next.
8. Under Name, review, and create, enter a Role name for the S3 exporter role. For example, EndorLabsS3ExporterRole.

9. Optionally, add tags to help identify the role.
10. Click Create role.

Create an S3 exporter

Create an S3 exporter using the Endor Labs API to configure the export destination and data types.

The following table lists the configuration options required to create the exporter.

Run the following command to create an S3 exporter.

endorctl api create \
  --namespace=<namespace> \
  --resource=Exporter \
  --data '{
    "meta": {
      "name": "<exporter-name>"
    },
    "spec": {
      "exporter_type": "EXPORTER_TYPE_S3",
      "s3_config": {
        "bucket_name": "<your-bucket-name>",
        "region": "<aws-region>",
        "assume_role_arn": "<iam-role-arn>",
        "allowed_audience": "<oidc-audience>"
      },
      "message_type_configs": [
        {
          "message_type": "MESSAGE_TYPE_FINDING",
          "message_export_format": "MESSAGE_EXPORT_FORMAT_JSON"
        }
      ]
    }
  }'

For example, to create an S3 exporter named s3-findings-exporter in the namespace doe.deer that exports findings in JSON format, run the following command.

endorctl api create \
  --namespace=doe.deer \
  --resource=Exporter \
  --data '{
    "meta": {
      "name": "s3-findings-exporter"
    },
    "spec": {
      "exporter_type": "EXPORTER_TYPE_S3",
      "s3_config": {
        "bucket_name": "my-endorlabs-exports",
        "region": "us-west-2",
        "assume_role_arn": "arn:aws:iam::123456789012:role/EndorLabsS3ExportRole",
        "allowed_audience": "s3-exporter"
      },
      "message_type_configs": [
        {
          "message_type": "MESSAGE_TYPE_FINDING",
          "message_export_format": "MESSAGE_EXPORT_FORMAT_JSON"
        }
      ]
    }
  }'

Configure scan profile to use the S3 exporter

After creating the exporter, associate it with your scan profile. You can also set the scan profile as the default for your namespace so all projects use it automatically. See Scan profiles for more information.

Configure the scan profile

1. Select Settings from the left sidebar.
2. Select Scan Profiles.
3. Select the scan profile you want to configure and click Edit Scan Profile.
4. Select your exporter under Exporters and click Save Scan Profile.

Configure the project to use the scan profile

Associate your project with a scan profile to enable automatic export of scan data.

1. Select Projects from the left sidebar and select the project you want to configure.
2. Select Settings and select the scan profile you want to use under Scan Profile.

Scan projects to export data

After configuration, subsequent scans automatically export data to your S3 bucket. You can trigger a scan immediately using the rescan feature. See Rescan projects for more information.

Exported file structure

Endor Labs exports data to S3 using a hierarchical folder structure:

endor/
└── <exporter-uuid>-<exporter-name>/
    └── <namespace>/
        └── <project-uuid>-<project-name>/
            └── <scan-type>/
                └── <ref-or-pr>/
                    └── <timestamp>_<scan-uuid>.zip

Each path segment is defined as follows:

Example file path

my-bucket/endor/abc123-prod-exporter/acme-corp/6efgh-pythonrepo/schedule/main/20251215T143025Z_xyz789.zip

Manage the S3 exporter

You can list, update, and delete S3 exporters using the Endor Labs API.

endorctl api list --namespace=<namespace> --resource=Exporter

endorctl api update \
  --namespace=<namespace> \
  --resource=Exporter \
  --name=<exporter-name> \
  --field-mask "spec.s3_config.region" \
  --data '{
    "spec": {
      "s3_config": {
        "region": "us-west-2"
      }
    }
  }'

endorctl api delete --namespace=<namespace> --resource=Exporter --name=<exporter-name>