Data model

• 1: Common fields
• 2: Resource kinds

1 - Common fields

All objects adhere to the same high-level structure as outlined below. Object specific fields are defined in Spec. For more information, see Resource kinds.

UUID

All objects have a unique UUID. You can use UUID to retrieve objects individually through the API.

Meta

All objects include a common nested object called Meta. This mandatory object contains common fields for each object, including:

TenantMeta

Most objects include a common nested object called TenantMeta. TenantMeta contains the following field:

OSS tenant

There is a common tenant for all OSS projects called oss, to which customers have read access.

Spec

All objects include a common nested object called Spec. This mandatory object contains the specification of the object, representing its current state. For more information, see Resource kinds.

Context

Most objects include a common nested object called Context. Contexts keep objects from different scans separated. The context object has the following fields:

Context types

Each context has a type and an id. For example, objects created during a scan of the default branch belong to the main context, while objects for non-default branches have the context type ref.

Processing status

Project and PackageVersion objects include a common nested object called ProcessingStatus, which contains fields about the processing status of the object (when it was/will be scanned). The processing status object has the following fields:

Scan state

The following scan states are supported:

Example

The following is an example of a Project object:

{
  "meta": {
    "create_time": "2023-12-05T00:04:21.853Z",
    "kind": "Project",
    "name": "https://github.com/my_organization/my_repository.git",
    "update_time": "2024-05-01T16:50:03.830911988Z",
    "version": "v1"
  },
  "processing_status": {
    "analytic_time": "2024-05-01T16:45:06.483972413Z",
    "disable_automated_scan": true,
    "scan_state": "SCAN_STATE_IDLE",
    "scan_time": "2024-03-18T14:38:31.899249002Z"
  },
  "spec": {
    "git": {
      "full_name": "endorlabs/monorepo",
      "git_clone_url": "git@github.com:my_organization/my_repository.git",
      "http_clone_url": "https://github.com/my_organization/my_repository.git",
      "organization": "endorlabs",
      "path": "monorepo",
      "web_url": "https://api.github.com/my_organization/my_repository"
    },
    "internal_reference_key": "https://github.com/my_organization/my_repository.git",
    "platform_source": "PLATFORM_SOURCE_GITHUB"
  },
  "tenant_meta": {
    "namespace": "my_namespace"
  },
  "uuid": "656e69058032bf0abaaeb681"
}

2 - Resource kinds

Here is an overview diagram of the Endor Labs data model for the most commonly used resource kinds. Lighter shading identifies the objects that are re-computed on every scan.

This section describes the most commonly used resource kinds. For a complete list of supported resource kinds, see the Endor Labs OpenAPI documentation.

All objects contain a reference to the project UUID, either as the parent object (meta.parent_uuid) or a specific field in the object-specific data if the project is not the direct parent (spec.project_uuid if not specified otherwise).

Use the following command to get a list of all objects of a given resource kind in your tenant.

Here are a few useful options:

• --count
• --page-size=1
• --list-all
• --filter="meta.parent_uuid==<uuid>"
• --filter="spec.project_uuid==<uuid>"

endorctl api list -r <resource-kind>

Project

• This is the logical root of all the other information for a given project.
• Contains information about the source code location of a project, such as a Git repository, or a package manager package name.
• Does not have a parent and is not associated with a context.
• The object name is the HTTP clone URL, for example: "https://github.com/definitelytyped/definitelytyped.git".

For more information, see the ProjectService REST API documentation.

Repository

• Contains information about the source code for a project.
• Child of a Project and, just like the Project, does not belong to a context.
• There is at most one Repository per Project, but a Project may not have a Repository if there is no source code.
• The object name is the same as the Project.

For more information, see the RepositoryService REST API documentation.

RepositoryVersion

• Contains information about a specific version of a Repository.
• Has the Project as the parent.
• There are often multiple RepositoryVersions per project.
• Each RepositoryVersion is associated with a Context.
• The object name is the corresponding branch name, tag, or SHA, for example: "main".

For more information, see the RepositoryVersionService REST API documentation.

PackageVersion

• Contains information about a specific version of a package and its dependencies.
• Does not have a parent (for historical reasons), but is associated with a Context and connected to the Project via spec.project_uuid.
• The object name is the corresponding package version name in the format <ecosystem>://<package-name>@<version>, for example: "mvn://org.webjars.npm:types__json-schema@1.2.3".

For more information, see the PackageVersionService REST API documentation.

Resolution errors

Details about any dependency resolution or call graph generation errors for a package version are stored in spec.resolution_errors. There are three categories of resolution errors, each with a separate field that can contain up to one resolution error:

1. Unresolved - Details in spec.resolution_errors.unresolved if there was an error computing the unresolved dependencies.
2. Resolved - Details in spec.resolution_errors.resolved if there was an error resolving the dependency versions.
3. Call graph - Details in spec.resolution_errors.call_graph if there was an error generating the call graph.

Each resolution error has a status_error field and may also contain details about the target, the operation that failed, and a description of the error. The following status errors are supported:

Below is an example resolution error in the Resolved category:

{
  "spec": {
    "resolution_errors": {
      "resolved": {
        "description": "failed to discover dependency: unable to resolve dependencies for 'requirements': unable to get direct dependencies: unable to install modules to extract dependencies: unable to resolve package version: ResolveModuleVersion: error in pypi json api for: torch, exact version: 1.9.0+cpu, err: package not found in the repository: unable to resolve dependency version: unable to discover dependencies, unable to discover dependencies",
        "operation": "python:resolvedDependencies:discover",
        "status_error": "STATUS_ERROR_DEPENDENCY",
        "target": "pypi://requirements@main"
      }
    }
  }
}

DependencyMetadata

• Different from other common resource kinds as it represents the relationship between two PackageVersions: The importer and the dependency.
• There is one DependencyMetadata object for every dependency for every PackageVersion.
• Has the importer PackageVersion as the parent and exists in the same Namespace and Context as the parent.
• The object name is the same as the dependency PackageVersion.
• Combine the object name (meta.name) and the parent UUID (meta.parent_uuid) to get a unique key.
• Connected to the Project via spec.importer_data.project_uuid.
• Details about the relationship are stored in spec.dependency_data, for example:
  • spec.dependency_data.direct
  • spec.dependency_data.reachable
  • spec.dependency_data.scope

For more information, see the DependencyMetadataService REST API documentation.

LinterResult

• Contains the results of scans using 3rd party programs such as gitleaks or Semgrep.
• Has a RepositoryVersion or PackageVersion as the parent.
• Belongs to the same Context as the parent.
• Connected to the Project through spec.project_uuid.
• The object name is the name of the rule that created the result, for example: "gen-shady-links".
• The result origin is stored in spec.origin, for example: "LINTER_RESULT_ORIGIN_SECRETS_SCANNER".

For more information, see the LinterResultService REST API documentation.

Metric

• Contains the output of the analytics processing.
• Has a PackageVersion, RepositoryVersion, or Repository as the parent.
• Belongs to the same Context as the parent.
• Connected to the Project via spec.project_uuid.
• The object name is the name of the analytic that created the metric, for example: "package_version_scorecard".

For more information, see the MetricService REST API documentation.

Metric types

There are many different types of Metrics. The specifics are stored under spec.metric_values.<key>.<value>, for example: spec.metric_values.scorefactor.score_factor_list. Some Metrics have more than one key-value field under spec.metric_values. The following table lists all supported Metric types along with the corresponding paths to the Metric specific data under spec.metric_values.

Finding

• Contains details of a problem that needs to be fixed.
• Has a PackageVersion, RepositoryVersion, or Repository as the parent.
• Belongs to the same Context as the parent.
• Connected to the Project via spec.project_uuid.
• There are many different types of Findings and new types can be created by custom Finding Policies.
• The object name is the Finding type, for example: "outdated_release". For more information, see Finding names and metadata below.
• The object description contains a more specific description of the Finding, for example: "Outdated Dependency @babel/plugin-syntax-async-generators@7.8.4".
• Additional finding type specific data is stored in spec.finding_metadata, for example: spec.finding_metadata.vulnerability.
• PackageVersion Findings often involve both the root PackageVersion and a dependency PackageVersion. The following details about the dependency PackageVersion are available directly in the Finding object:
  • spec.target_dependency_name, for example: "@babel/plugin-syntax-async-generators"
  • spec.target_dependency_package_name, for example: "npm://@babel/plugin-syntax-async-generators@7.8.4"
  • spec.target_dependency_version, for example: "7.8.4"
  • spec.finding_metadata.dependency_package_version_metadata
• The UUID of the DependencyMetadata for the dependency is stored in spec.target_uuid.
• There is one Finding object for every PackageVersion that includes a dependency with a given problem. If 10 PackageVersions include a dependency with a vulnerability then there will be 10 findings for the vulnerability.

For more information, see the FindingService REST API documentation.

Finding names and metadata

The following table lists all supported values for the Finding meta.name field along with an example value for the corresponding meta.description and an explanation.

Finding categories

The following finding categories are supported as possible values in the spec.finding_categories list. All findings must have at least one category.

Finding tags

The following finding tags are supported as possible values in the spec.finding_tags list.

Exceptions

There are two ways that a finding can be exempt from triggering action policies (a.k.a. admission and notification policies).

1. Dismiss finding

   Set the spec.dismiss field to true to exclude the specific finding from action policies. This value persists as long as the finding exists, which is as long as the parent object remains the same. If this is a PackageVersion finding and the root package version number changes, then a new PackageVersion object is created and it gets a new set of findings, so the value of dismiss does not persist.

2. Add an exception policy

   Exception policies allow you to set any criteria you want to mark findings as exempt and can be applied across all projects, a sub-set of projects, or a specific project, within a tenant. Based on the criteria you set, the exception can persist across multiple package versions.

   Findings matched by one or more exception policies have the FINDING_TAGS_EXCEPTION tag and the corresponding policy object uuids are listed under spec.exceptions.policy_uuids.

Action policies

Findings matched by one or more action policies (a.k.a. admission and notification policies) contain the corresponding policy object uuids in spec.actions.policy_uuids. They also carry a tag corresponding to the specific action, for example, FINDING_TAGS_CI_WARNING, FINDING_TAGS_CI_BLOCKER, or FINDING_TAGS_NOTIFICATION.

ScanResult

• Contains details of a scan such as:
  • Configuration
  • Host environment details
  • Runtime statistics
  • Findings
  • Policies triggered
  • Error logs
  • Exit code
  • Scan status
• Has the Project as the parent.
• Belongs to the same Context as the scan.

For more information, see the ScanResultService REST API documentation.

Scan status

The following scan statuses are supported: