Monitoring or supervisory scans

• 1: Deploy Endor Labs GitHub App
• 1.1: Scanning capabilities
• 1.2: Rescan projects
• 1.3: Technical limitations of the Endor Labs GitHub App
• 1.4: Deploy Endor Labs GitHub App (Pro)
• 2: Deploy Endor Labs Azure DevOps App
• 3: Deploy Endor Labs GitLab App
• 4: Set up Jenkins pipeline for supervisory scans

graph TD
    A(["Endor Labs App"]) -->|<span style='font-size: 12px'>Continuous monitoring</span>| B["Customer Repositories"]
    A -->|<span style='font-size: 12px'>Initiate scans every 24h or on-demand</span>| C["Endor Labs Cloud
    <span style='font-size: 12px'>Customer data destroyed after scans</span>"]
    B -->|<span style='font-size: 12px'>Clones and scans repositories</span>| C
    C -->|<span style='font-size: 12px'>Pass scan data</span>| D(["Endor Labs Platform
    <span style='font-size: 12px'>Generate findings from scan results</span>"])
    subgraph "<span style='font-size: 18px'>Supervisory scan workflow<span>"
    A
    B
    C
    D
    end

1 - Deploy Endor Labs GitHub App

Endor Labs provides a GitHub App that continuously monitors users’ projects for security and operational risk. You can use the GitHub App to selectively scan your repositories for SCA, secrets, RSPM, or CI/CD tools. GitHub App scans also establish baselines that are subsequently used during CI scans.

Endor Labs GitHub App scans your repositories every 24 hours and reports any new findings or changes to release versions of your code.

If you want to use PR remediations as part of your monitoring scan, you need to use GitHub App (Pro).

Prerequisites for GitHub App

Before installing and scanning projects with Endor Labs GitHub App, make sure you have:

• A GitHub cloud account and organization. If you don’t have one, create one at GitHub.
• Administrative permissions to your GitHub organization. Installing the Endor Labs GitHub App in your organization requires approval or permissions from your GitHub organizational administrator. If you don’t have the permissions, use the command line utility, endorctl, while you wait for the approval.
• Endor Labs GitHub App requires:
  • Read permissions to Dependabot alerts, actions, administration, code, commit statuses, issues, metadata, packages, repository hooks, and security events.
  • Write permissions to checks and pull requests to check the pull requests automatically and surface policy violations to developers as pull request comments.
  • Subscribe to check run, check suite, and pull request events.

Install the GitHub App

To automatically scan repositories using the GitHub App:

1. Sign in to Endor Labs.

2. Choose Projects and click Add Project.

3. From GITHUB, choose GitHub App

4. Click Install GitHub App.

   You will be redirected to GitHub to install the GitHub App.

5. Click Install.

6. Select a user to authorize the app.

7. Select the organization in which you want to install the app.

8. Select whether to install and authorize Endor Labs on all your repositories or select the specific repositories that you wish to scan.

   

9. Review the permissions required for Endor Labs and click Install and Authorize.

   If the button to install says Install and Request instead of Install and Authorize, you don’t have permission to install the GitHub App. Use the endorctl command line interface or select Install and Request to notify your organizational administrator of your request to install.

10. Choose a namespace and click Next.

    

11. Based on your license, select and enable the scanners.

    

    The following scanners are available.

    • SCA: Perform software composition analysis.
    • CI/CD: Scan the repository and identify all the CI/CD tools used in the repository.
    • RSPM: Scan the repository for misconfigurations.
    • Secret: Scan the repository for exposed secrets.
    • SAST: Scan your source code for weakness and generate SAST findings.
    • AI Models: Scan your repository and discover AI models in your source code.

12. Select PULL REQUEST SCANS to set preferences for scanning pull requests submitted by users.

    

    • Select Enable Automatic Pull Request Scanning to automatically scan the PRs submitted by users.
    • Select Enable Pull Request Comments to enable GitHub Actions to comment on PRs for policy violations.
    • Select Include archived repositories to scan your archived repositories. By default, the GitHub archived repositories aren’t scanned.
    • Set the Scanning Preferences to:

      • Quick Scan to gain rapid visibility into your software composition. It performs dependency resolution but does not conduct reachability analysis to prioritize vulnerabilities. The quick scan enables users to swiftly identify potential vulnerabilities in dependencies, ensuring a smoother and more secure merge into the main branch.

      • Full Scan to perform dependency resolution, reachability analysis, and generate call graphs for supported languages and ecosystems. This scan enables users to get complete visibility and identifies all issues dependencies, call graph generation, before merging into the main branch. Full scans may take longer to complete, potentially delaying PR merges.

See GitHub scan options for more information on the scans that you can do with the GitHub App. 13. Click Continue.

You have successfully installed the GitHub App.

Manage GitHub Apps on Endor Labs

You can edit or delete the GitHub App integrations.

1. Sign in to Endor Labs.
2. Select Manage > Integrations from the left navigation menu.
3. Click Manage next to GitHub under Source Control Managers.
4. Click the ellipsis on the right side, and select Edit Integration.
5. Based on your license, select and enable from the available list of SCANNERS.
6. Choose PULL REQUEST SCANS to set preferences for scanning pull requests submitted by users.

   • Select Enable Automatic Pull Request Scanning to automatically scan the PRs submitted by users.

   • Select Enable Pull Request Comments to enable GitHub Actions to comment on PRs for policy violations.

   • Select Include archived repositories to scan your archived repositories. By default, the GitHub archived repositories aren’t scanned.

   • Set the Scanning Preferences to:

     • Quick Scan to gain rapid visibility into your software composition. It performs dependency resolution but does not conduct reachability analysis to prioritize vulnerabilities. The quick scan enables users to swiftly identify potential vulnerabilities in dependencies, ensuring a smoother and more secure merge into the main branch.

     • Full Scan to perform dependency resolution, reachability analysis, and generate call graphs for supported languages and ecosystems. This scan enables users to get complete visibility and identifies all issues dependencies, call graph generation, before merging into the main branch. Full scans may take longer to complete, potentially delaying PR merges. The changes are applicable from the next scanning cycle.

7. Use Reset to clear your selection.
8. To delete a GitHub App integration, click the ellipsis on the right side, and select Delete Integration.
9. To manually trigger a scan, click Rescan Org. Endor Labs GitHub App scans your repositories every 24 hours, use Rescan Org to manually schedule outside the 24-hour period.
10. Click Scan More Repositories to go to Projects page, from which you can add more repositories to scan through the GitHub App.

Set up package repositories

You can improve your experience with the GitHub App by setting up package repositories. This will help you create a complete bill of materials and perform static analysis. Without setting package repositories, you may not be able to get an accurate bill of materials. See Set up package manager integration for more information.

1.1 - Scanning capabilities

With the Endor Labs GitHub App, you can enhance the security of your repository through the following types of scans.

Scan complete repository

The Endor Labs GitHub App automatically scans your repositories every 24 hours for potential security issues and operational risks, providing up-to-date information about your projects’ security posture.

• You can use the GitHub App to selectively scan your repositories for Software Composition Analysis (SCA), secrets, Repository Security Posture Management (RSPM), or CI/CD tools.
• While the automated scan happens every 24 hours, you can manually trigger a rescan outside this schedule from the Endor Labs user interface. See Rescan projects.
• After each scan, the GitHub App reports any new findings or changes to release versions of your code. Review the scan results from the Endor Labs user interface.

Scan PRs

After scanning the complete repository, it’s important to address the pull requests submitted by users. Administrators can enable a fully automated scanning process for all pull requests and merges initiated into the main branch.

To automatically scan the PRs, set the pull request preferences during the installing of the GitHub App or edit the integration preferences afterwards.

Whenever a PR is created against a repository, the Endor Labs GitHub App performs an incremental scan to detect any changes in resolved dependencies that may introduce new vulnerabilities. These incremental scans are CI runs and are not monitored. You can see the results of the scan on GitHub.

Based on your prefrences, it performs a quick scan or a full scan before merging the PRs into the main branch.

• Quick Scan performs dependency resolution but does not conduct reachability analysis to prioritize vulnerabilities. The quick scan enables users to swiftly identify potential vulnerabilities in dependencies, ensuring a smoother and more secure merge into the main branch.
• Full Scan performs dependency resolution, reachability analysis, and generate call graphs for supported languages and ecosystems. This scan enables users to get complete visibility and identifies all issues related to dependencies and call graph generation, before merging into the main branch. Full scans may take longer to complete, potentially delaying PR merges.

1.2 - Rescan projects

Endor Labs enables you to rescan your GitHub projects. When you make a code change or upgrade a dependency, rescanning your GitHub projects ensures the integrity and security of your software.

To enable periodic scanning of your GitHub projects, install the GitHub App from Endor Labs. For more information, see Install the GitHub App.

Endor Labs automatically triggers a rescan of your GitHub projects every 24 hours. However, you can manually trigger a rescan. Follow these steps:

1. Sign in to Endor Labs and click Manage Projects.
2. Select a project configured for automated scanning using the GitHub App.
3. Click Rescan Project to start rescanning.

1.3 - Technical limitations of the Endor Labs GitHub App

The Endor Labs GitHub App provides visibility across a GitHub organization, but it has technical limitations that do not account for the unique requirements of your application.

Here are some of these limitations.

Custom package build steps

Endor Labs requires executing custom build steps outside of standard package manager commands to build software packages and get an accurate bill of materials and perform static analysis. Sometimes, a complete bill of materials may not be generated, or static analysis may not be performed if custom steps are required for your software to build. Applications that require custom build steps may need to be implemented in a CI environment to successfully get an accurate bill of materials.

Custom resource profiles

Large applications may require significant memory allocations to perform static analysis on a package. The services scanning the GitHub App use 16 GB of memory by default. Applications that require more memory may not obtain vulnerability prioritization information using the GitHub App. Scan large applications in a CI environment using a runner with sufficient resource allocations.

Authentication for private software components

Private software components hosted in an internal package repository may require authentication credentials to create a complete bill of materials or perform static analysis.

If your authentication information to your private package repository is hosted outside the repository, you will need to configure a package manager integration. See Set up package manager integration for more details. If your package repository is inaccessible from the public internet, you can work with Endor Labs to evaluate options.

1.4 - Deploy Endor Labs GitHub App (Pro)

Endor Labs GitHub App (Pro) is an enhanced version of the Endor Labs GitHub App that supports PR remediation to fix vulnerabilities. See PR remediation for more information.

Prerequisites for GitHub App (Pro)

Before installing and scanning projects with Endor Labs GitHub App (Pro), make sure you have:

• Administrative permissions to your GitHub organization. Installing the Endor Labs GitHub App (Pro) in your organization requires approval or permissions from your GitHub organizational administrator. If you don’t have the permissions, use the command line utility, endorctl, while you wait for the approval.
• Endor Labs GitHub App (Pro) requires the following permissions:
  • Read access to Dependabot alerts, actions, administration, checks, code, commit statuses, issues, metadata, packages, pull requests, repository hooks, and security events.
  • Read and write access to checks, contents, and pull requests.

Install GitHub App (Pro)

To automatically scan repositories using the GitHub App and create automatic PRs to fix vulnerabilities:

1. Sign in to Endor Labs.

2. From the left sidebar, choose Projects and click Add Project.

3. From GITHUB, choose GitHub App

4. Select Enable Automated Pull Requests.

   

5. Click Install GitHub App (Pro).

   You will be redirected to GitHub to install the Endor Labs App (Pro).

6. Click Install.

7. Select a user to authorize the app.

8. Select the organization in which you want to install the app.

9. Select whether to install and authorize Endor Labs on all your repositories or select the specific repositories that you wish to scan.

10. Review the permissions required for Endor Labs and click Install and Authorize.

    If the button to install says Install and Request instead of Install and Authorize, you don’t have permission to install the GitHub App. Use the endorctl command line interface or select Install and Request to notify your organizational administrator of your request to install.

11. Choose a namespace and click Next.

    

12. Based on your license, select and enable the scanners.

    

    The following scanners are available.

    • SCA: Perform software composition analysis.
    • CI/CD: Scan the repository and identify all the CI/CD tools used in the repository.
    • RSPM: Scan the repository for misconfigurations.
    • Secret: Scan the repository for exposed secrets.
    • SAST: Scan your source code for weakness and generate SAST findings.
    • AI Models: Scan your repository and discover AI models in your source code.

13. Select PULL REQUEST SCANS to set preferences for scanning pull requests submitted by users.

    

    • Select Enable Automatic Pull Request Scanning to automatically scan the PRs submitted by users.
    • Select Enable Pull Request Comments to enable GitHub Actions to comment on PRs for policy violations.
    • Select Include archived repositories to scan your archived repositories. By default, the GitHub archived repositories aren’t scanned.
    • Set the Scanning Preferences to:

      • Quick Scan to gain rapid visibility into your software composition. It performs dependency resolution but does not conduct reachability analysis to prioritize vulnerabilities. The quick scan enables users to swiftly identify potential vulnerabilities in dependencies, ensuring a smoother and more secure merge into the main branch.

      • Full Scan to perform dependency resolution, reachability analysis, and generate call graphs for supported languages and ecosystems. This scan enables users to get complete visibility and identifies all issues dependencies, call graph generation, before merging into the main branch. Full scans may take longer to complete, potentially delaying PR merges.

See GitHub scan options for more information on the scans that you can do with the GitHub App. 14. Click Continue. You have successfully installed the GitHub App (Pro).

Endor Labs GitHub App (Pro) scans your repositories every 24 hours and reports any new findings or changes to release versions of your code. It can also raise a PR with a fix based on your remediation policy. Ensure that you configure automated PR scans in your environment. See Automated PR scans for more information.

Manage GitHub Apps on Endor Labs

You can edit or delete the GitHub App integrations.

Edit GitHub App (Pro)

To edit the GitHub App integration:

1. Sign in to Endor Labs.
2. Select Manage > Integrations from the left navigation menu.
3. Click Manage next to GitHub under Source Control Managers.
4. Click the three verticals dots on the right side of the GitHub App (Pro) that you want to edit, and select Edit Integration.
5. Based on your license, select and enable from the available list of scanners. You can also choose to update the pull request scan options.
6. Click Save. The changes are applicable from the next scanning cycle.
7. Use Reset to clear your selection.

Migrate GitHub App

You can migrate your GitHub App (Pro) to standard GitHub App (or from standard to Pro).

1. Sign in to Endor Labs.

2. Select Manage > Integrations from the left navigation menu.

3. Click Manage next to GitHub under Source Control Managers.

4. Click the three vertical dots on the right side of the GitHub App (Pro) that you want to edit, and select Migrate To Standard App.

5. Click Migrate.

   You will be redirected to GitHub.

6. Click Configure.

7. Select a user to authorize the app.

8. Select Configure in the organization in which you want to migrate the app.

9. Select whether to install and authorize Endor Labs on all your repositories or select the specific repositories that you wish to scan.

   Warning

   When you migrate from one app to the other, select the same set of repositories as before to preserve the currently scanned projects and findings after the migration.

10. Choose the namespace and click Next.

    Warning

    You must choose the same namespace as your existing GitHub App installation.

11. Select and enable the scanners you require.

12. Select the preferences for scanning pull requests, if required.

13. Click Continue.

Delete GitHub App (Pro)

To delete a GitHub App integration, click the three vertical dots on the right side, and select Delete Integration.

You are to taken to the GitHub App page, where you can uninstall the app from your GitHub organization.

Manually trigger scans

To manually trigger a scan, click Rescan Org. Endor Labs GitHub App scans your repositories every 24 hours, use Rescan Org to manually schedule outside the 24-hour period.

Add more GitHub repositories

Click Scan More Repositories to go to Projects page, from which you can add more repositories to scan through the GitHub App.

Set up package repositories

You can improve your experience with the GitHub App by setting up package repositories. This will help you create a complete bill of materials and perform static analysis. Without setting package repositories, you may not be able to get an accurate bill of materials. See Set up package manager integration for more information.

Technical limitations of the GitHub App (Pro)

The Endor Labs GitHub App (Pro) has the same limitations as the GitHub App. See Limitations for more information.

2 - Deploy Endor Labs Azure DevOps App

Endor Labs provides an Azure DevOps App that continuously scans Azure repos in your projects for security risks. You can selectively scan your repositories for SCA, secrets, SAST, or CI/CD tools.

When you add an Azure DevOps project to an Endor Labs namespace, Endor Labs scans all the Azure repos contained in the project. As a best practice, we recommend that you add only one Azure project to one Endor Labs namespace so that the Azure repos of that project are mapped to an Endor Labs namespace.

Prerequisites for Azure DevOps App

Ensure the following prerequisites are in place before you install the Endor Labs Azure DevOps App.

• An Azure DevOps cloud account and organization. If you don’t have one, create one at Azure DevOps.
• Endor Labs Azure DevOps App requires read permissions to in your project. You can grant these permissions by providing read access to the Code category when you create an Azure DevOps personal access token for Endor Labs.

Install the Azure DevOps App

To automatically scan repositories using the Azure DevOps App:

1. Sign in to Endor Labs.

2. Select Projects from the left sidebar and click Add Project.

3. From AZURE, select Azure DevOps App.

   

4. Enter the host URL of your Azure project.

   The URL must be in the format, https://dev.azure.com/<ORG_NAME>/<PROJECT_NAME>.

5. Enter your personal access token from Azure.

   You must have at least read permissions in the Code category for your Azure DevOps personal access token.

6. Click Scanners and select the scan types to enable.

   • SCA- Perform software composition analysis.
   • Secret - Scan Azure repos for exposed secrets.
   • CI/CD - Scan Azure repos and identify all the CI/CD tools used.
   • SAST - Scan your source code for weakness and generate SAST findings.

   The available scan types depend upon your license.

   

7. Click Create.

Endor Labs Azure DevOps App scans your Azure repos every 24 hours and reports any new findings or changes to release versions of your code.

Manage Azure DevOps Apps on Endor Labs

You can edit or delete the Azure DevOps App integrations.

To edit the Azure DevOps App integration:

1. Sign in to Endor Labs and select Manage > Integrations from the left navigation menu.
2. Click Manage next to Azure under Source Control Managers.
3. Click on the three vertical dots next to the integration, and select Edit Integration. You can update your personal access token.
4. Click SCANNERS and based on your license, select and enable from the available list of scanners.
5. Click Save. The changes are applicable from the next scanning cycle.

To delete an Azure DevOps App integration, click on the three vertical dots next to the integration, and select Delete Integration.

To manually trigger a scan, click Rescan Org. Azure DevOps App scans your repositories every 24 hours, use Rescan Org to manually schedule outside the 24-hour period.

Click Scan More Repositories to go to Projects, where you can add more projects to scan through the Azure DevOps App.

3 - Deploy Endor Labs GitLab App

Endor Labs provides a GitLab App that continuously monitors users’ projects for security and operational risk. You can use the GitLab App to selectively scan your repositories for SCA, secrets, SAST, and CI/CD tools.

When you use Endor Labs GitLab App, Endor Labs creates namespaces based on your organization hierarchy in GitLab.

The namespaces created by the Endor Labs GitLab App are not like regular namespaces and are called managed namespaces. These namespaces are named after subgroup slugs in GitLab.

Limitations of GitLab groups in Endor Labs namespace

Ensure that you consider the following limitations when you use the GitLab monitoring scan.

• GitLab supports up to 20 levels of subgroup nesting, while Endor Labs currently supports a maximum of 10 levels, assuming the installation is created at the tenant level. If a GitLab installation is created within a nested namespace, such as tenant.namespace1.namespace2, the available nesting depth for subgroups in GitLab is reduced. In this case, Endor Labs can only support up to eight levels of nested subgroups.
• Endor Labs supports GitLab groups with a maximum of 64 characters
• Endor Labs does not support dot(.) in group paths and will skip the creation of namespace for such groups.

Managed Namespaces in Endor Labs

Managed namespaces are always reflective in terms of structure and content in the SCM platform.

• You cannot delete managed namespaces.

• You cannot delete projects present within managed namespaces.

• You cannot add projects or create namespaces within managed namespaces.

• You cannot create any new Endor Labs App installation within the managed namespaces.

  For example, you cannot create an Endor Labs GitHub App installation within a namespace that was created by the Endor Labs GitLab App.

Any modifications to the namespaces have to be done at the SCM side. The changes that you make to the namespaces and projects are reflected in Endor Labs after a rescan.

If your SCM has the following hierarchy:

graph TD
    HC[HappyCorp]

    %% Main divisions
    Web[Web]
    Mobile[Mobile]
    Desktop[Desktop]

    %% Web subgroups
    WA[Alpha]
    WB[Beta]
    WG[Gamma]

    %% Mobile subgroups
    MD[Delta]
    ME[Epsilon]
    MZ[Zeta]

    %% Desktop subgroups
    DP[Pi]
    DR[Rho]
    DS[Sigma]

    %% Main connections
    HC --> Web
    HC --> Mobile
    HC --> Desktop

    %% Web connections
    Web --> WA
    Web --> WB
    Web --> WG

    %% Mobile connections
    Mobile --> MD
    Mobile --> ME
    Mobile --> MZ

    %% Desktop connections
    Desktop --> DP
    Desktop --> DR
    Desktop --> DS

    class HC main
    class Web,Mobile,Desktop division

Endor Labs creates HappyCorp as the parent namespace with Web, Mobile, and Desktop as the child namespaces. The namespace HappyCorp will be under the Endor Labs namespace (for example, HappyEndor) that you create.

Each of these child namespaces have further child namespaces as follows:

• Web: Alpha, Beta, Gamma
• Mobile: Delta, Epsilon, Zeta
• Desktop: Pi, Rho, Sigma

The following diagram shows the organization of namespaces in Endor Labs.

graph TD
    EN[HappyEndor]
    HC[HappyCorp]

    %% Main divisions
    Web[Web]
    Mobile[Mobile]
    Desktop[Desktop]

    %% Web subgroups
    WA[Alpha]
    WB[Beta]
    WG[Gamma]

    %% Mobile subgroups
    MD[Delta]
    ME[Epsilon]
    MZ[Zeta]

    %% Desktop subgroups
    DP[Pi]
    DR[Rho]
    DS[Sigma]

    %% Main connections
    EN --> HC
    HC --> Web
    HC --> Mobile
    HC --> Desktop

    %% Web connections
    Web --> WA
    Web --> WB
    Web --> WG

    %% Mobile connections
    Mobile --> MD
    Mobile --> ME
    Mobile --> MZ

    %% Desktop connections
    Desktop --> DP
    Desktop --> DR
    Desktop --> DS

    class HC main
    class Web,Mobile,Desktop division

Prerequisites for GitLab App

Before installing and scanning projects with Endor Labs GitLab App, make sure you have:

• A GitLab cloud account and organization. If you don’t have one, create one at GitLab.
• Endor Labs GitLab App requires a GitLab personal access token with at least read_api permission.

Install the GitLab App

1. Sign in to Endor Labs.

2. Select Projects from the left sidebar and click Add Project.

3. From GITLAB, select GitLab App.

4. Enter the GitLab organization URL in the format: https://gitlab.com/{group}/{subgroup1}/....

   You need to enter at least the root group. For example, https://gitlab.com/group1.

   You can provide the host URL up to any subgroup level. For example, https://gitlab.com/group1/subgroup1/subgroup2/subgroup3.

   Endor Labs creates namespaces for groups and subgroups and maps projects to these namespaces.

   If the GitLab installation is created at the tenant level, Endor Labs supports up to 10 levels of GitLab group nesting. If the installation is created within a nested namespace under the tenant, the supported nesting depth decreases by one level for each additional level of nesting.

5. Enter the GitLab personal access token.

   The personal access token must have at least the read_api permission.

6. Select the scan types to enable.

   • SCA- Perform software composition analysis.
   • Secret - Scan GitLab projects for exposed secrets.
   • CI/CD - Scan GitLab projects and identify all the CI/CD tools used.
   • SAST - Scan GitLab projects to generate SAST findings.

   The available scan types depend upon your license.

7. Click Create.

Endor Labs GitLab App scans your GitLab projects every 24 hours and reports any new findings or changes to release versions of your code.

Manage GitLab App on Endor Labs

You can edit or delete the GitLab App integrations.

To edit the GitLab App integration:

1. Sign in to Endor Labs and select Manage > Integrations from the left navigation menu.
2. Click Manage next to GitLab under Source Control Managers.
3. Click on the three vertical dots next to the integration, and select Edit Integration. You can update your personal access token and choose the scanners.
4. Click Save. The changes are applicable from the next scanning cycle.

To delete a GitLab App integration, click on the three vertical dots next to the integration, and select Delete Integration. When you delete the integration, it will also delete all child namespaces, projects and references associated with the auto generated root group namespace. It also deletes any manually created namespaces and projects under auto generated namespace.

To manually trigger a scan, click Rescan Org. GitLab App scans your repositories every 24 hours, use Rescan Org to manually schedule outside the 24-hour period.

Click Scan More Repositories to go to Projects, where you can add more projects to scan through the GitLab App.

When you create a new installation of the GitLab app, it cannot be in the same root group. For example, if a GitLab installation exists in gitlab.com/group1/sg1, you cannot create another installation under group1 like gitlab.com/group1/sg2/. You need to create the with a different root group. For example, gitlab.com/group2/sg2.

4 - Set up Jenkins pipeline for supervisory scans

Use the Endor Labs Jenkins pipeline to scan all the repositories in your organization and view consolidated findings. This pipeline runs on your organization’s Jenkins infrastructure and enables administrators to run organization-level supervisory scans easily. It is designed to work in GitHub Cloud and GitHub enterprise server environments.

The Jenkins pipeline carries out the following actions.

• Pulls the Endor Labs Docker image required to perform the scan.
• Synchronizes GitHub organization repositories to a specified namespace on the Endor Labs platform.
• Retrieves the project list or the GitHub repositories for the given tenant’s namespace.
• Groups the projects into batches to optimize scan execution.
• Runs endorctl scans on each batch of projects simultaneously.

Scan the repositories in your organization

The Jenkins Pipeline script is available in the github-org-scan-docker.groovy file.

To scan the repositories in your organization:

1. Generate Endor Labs API credentials
2. Configure GitHub cloud or GitHub enterprise server credentials
3. Configure the Jenkins job

Configure GitHub credentials

Configure the required credentials needed to access GitHub and Endor Labs in the Jenkins pipeline script. You can configure these values from the Jenkins user interface.

• GITHUB_TOKEN- Enter the GitHub token that has permission to access all the repositories in the organization.
• ENDOR_LABS_API_KEY- Enter the Endor Labs API key that you generated.
• ENDOR_LABS_API_SECRET- Enter the Endor Labs API secret generated while creating the Endor Labs API key.

Configure GitHub cloud credentials

Configure the following GitHub cloud parameters in the Jenkins pipeline script.

Required Parameters for GitHub cloud

• AGENT_LABEL- This is a string parameter. Enter the label used to identify the Jenkins agents. The Jenkins job will run on the agents that have this label.
• GITHUB_ORG- This is a string parameter. Enter the organization name in GitHub.
• ENDOR_LABS_NAMESPACE- This is a string parameter. The namespace of your organization tenant in Endor Labs.

Optional Parameters for GitHub cloud

• ENDOR_LABS_API- This is a string parameter. This is only required if the tenant namespace is configured on the Endor Labs staging environment.
• ADDITIONAL_ARGS- This is a string parameter. Use this field to pass any additional parameter to the endorctl scan.
• NO_OF_THREADS- This is a string parameter. Enter the number of Jenkins agents that can be used in parallel for the endorctl scan. If you have 10 Jenkins agents configured with the given AGENT_LABEL, you can enter this value as 9, 1 agent is used for the main job. If not specified, this value defaults to 5.
• ENDORCTL_VERSION- This is a string parameter. Specify the version of the endorctl Docker container. Defaults to the latest version.
• SCAN_TYPE- This is a string parameter. Set this to git to scan commits or github to fetch info from the GitHub API. Defaults to [git, analytics].
• SCAN_SUMMARY_OUTPUT_TYPE- This is a string parameter. Use this field to set the desired output format. Supported formats: json, yaml’, table, summary. Defaults to table.
• LOG_LEVEL- This is a string parameter. Use this field to set the log level of the application. Defaults to info.
• LOG_VERBOSE- This is a string parameter. Use this field to make the log verbose.
• LANGUAGES- This is a string parameter. Use this field to set programming languages to scan. Supported languages: c#,go, java, javascript, php, python, ruby, rust, scala, typescript. Defaults to all supported languages.
• ADDITIONAL_ARGS- This is a string parameter. Use this field to pass any additional parameters to the endorctl scan.

Configure GitHub enterprise server credentials

Configure the following GitHub enterprise server parameters in the Jenkins pipeline script.

Required Parameters for GitHub enterprise server

• AGENT_LABEL - This is a string parameter. Enter the label used to identify the Jenkins agents. The Jenkins job will run on the agents that have this label.
• GITHUB_ORG - This is a string parameter. Enter the organization name in GitHub.
• ENDOR_LABS_NAMESPACE - This is a string parameter. The namespace of your organization tenant in Endor Labs.
• GITHUB_API_URL - This is a string parameter. Enter the API URL of the GitHub enterprise server. This is normally in the form of <FQDN of GitHub Enterprise Server>/api/v3. For example, https://ghe.endorlabs.in/api/v3.

Optional Parameters for GitHub enterprise server

• ENDOR_LABS_API - This is a string parameter. This is only required if the tenant namespace is configured on the Endor Labs staging environment.

• GITHUB_DISABLE_SSL_VERIFY - This is a boolean parameter. This should be used when you want to skip SSL Verification while cloning the repository.

• GITHUB_CA_CERT - This is a multi-line string parameter. This should be used to provide the content of the CA Certificate (PEM format) of the SSL Certificate used on the GitHub Enterprise Server.

• PROJECT_LIST - This is a multi-line string parameter. This should be used to provide a list of projects to scan.

• SCAN_TYPE - This is a string parameter. Set this to git to scan commits or github to fetch info from the GitHub API. Defaults to [git, analytics].

• SCAN_SUMMARY_OUTPUT_TYPE - This is a string parameter. Use this field to set the desired output format. Supported formats: json, yaml*, table, summary. Defaults to table.

• LOG_LEVEL - This is a string parameter. Use this field to set the log level of the application. Defaults to info.

• LOG_VERBOSE - This is a string parameter. Use this field to generate verbose logs.

• LANGUAGES - This is a string parameter. Use this field to set programming languages to scan. Supported languages: c#, go, java, javascript, php, python, ruby, rust, scala, typescript. Defaults to all supported languages.

• ADDITIONAL_ARGS - This is a string parameter. Use this field to pass any additional parameters to the endorctl scan.

• PROJECT_LIST - This is a multi-line string parameter. List of projects to scan. Even though all projects are synchronized, scans run only on the provided projects.

• SCAN_PROJECTS_BY_LAST_COMMIT - This is a string parameter. This parameter is used to filter projects based on the date of the last commit. Enter a number (integer) value for this parameter. The value of 0 means that projects won’t be filtered based on last commit date. Any positive integer is used to calculate the duration in which a commit will add the project for further scanning. If a project did not have a commit in that interval, it will be skipped. If a proper SSL Certificate (a certificate issued by a well-known CA) is not used for GitHub Enterprise, the sync-org command fails and Endor Labs cannot fetch the projects or repositories to scan from the GitHub enterprise server. You can use this field to provide the list of projects or repositories to scan one per line. For example:

          https://github-test.endorlabs.in/pse/vuln_rust_callgraph.git
          https://github-test.endorlabs.in/pse/vulnerable-golang.git
          https://github-test.endorlabs.in/pse/java-javascript-vulnerable-repo.git
          https://github-test.endorlabs.in/pse/multi-lang-repo.git
  

• EXCLUDE_PROJECTS - This is a multi-line string parameter. Use this parameter to list projects or repositories to exclude from the scan.

• NO_OF_THREADS - This is a string parameter. Enter the number of Jenkins agents that can be used in parallel for the endorctl scan. If you have 10 Jenkins agents configured with the given AGENT_LABEL, you can enter this value as 9. If not specified, this value defaults to 5.

Configure the Jenkins job

Use the following procedure to configure the Jenkins pipeline and scan the repositories in your organization.

1. Sign in to Jenkins
2. Configure an Endor Labs API Key and GitHub credentials correctly for your environment.
3. Click + New Item, to create a new Jenkins job.
4. Enter the name of the new pipeline
5. Select Pipeline and click OK.
6. Select This project is parameterised and add the parameters based on your requirements.
7. From the Pipeline section, for Definition, select Pipeline script from SCM
8. For SCM select Git
9. For the Repository URL, enter either git@github.com:endorlabs/jenkins-org-scan.git or https://github.com/endorlabs/jenkins-org-scan.git.
10. For Credentials, enter the credentials required for cloning the repository entered in the previous step.
11. In Branches to build, enter */main.
12. For Script Path, enter github-org-scan-docker.groovy.
13. Select Lightweight checkout.
14. Click Save.

The Jenkins pipeline is highly customizable and adaptable to various GitHub environments and scanning requirements. It streamlines the process of running endorctl scans on your repositories efficiently.