✨ Skip checks that don't apply to the current repo type

[JamieMagee]

What kind of change does this PR introduce?

Feature.

• PR title follows the guidelines defined in our pull request documentation

What is the current behavior?

All checks run on every repo regardless of platform. Checks like Dangerous-Workflow and Token-Permissions scan .github/workflows/* and parse them with actionlint. On Azure DevOps repos these checks find nothing useful but still emit results, sometimes errors, sometimes bogus scores.

What is the new behavior (if this is a feature change)?

GetEnabled reads the repos field from checks.yaml and drops checks that don't list the repo's platform. The Repo interface has a new Type() method returning GitHub, GitLab, Azure DevOps, or local.

11 checks are tagged as supporting Azure DevOps based on which client methods are implemented.

• Tests for the changes have been added (for bug fixes/features)

Which issue(s) this PR fixes

NONE

Special notes for your reviewer

The repos field in checks.yaml was documentation-only until now. This makes it load-bearing: if a check is missing a platform in its repos field, it won't run there.

Does this PR introduce a user-facing change?

Yes. Azure DevOps users will see fewer irrelevant checks in their results.

Checks are now skipped when their declared repo types (in checks.yaml) don't match the repo being scanned. Prevents misleading results from GitHub-specific checks on Azure DevOps repos.

[codecov]

Codecov Report

❌ Patch coverage is 70.73171% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.78%. Comparing base (353ed60) to head (58d8e69).
⚠️ Report is 341 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5000      +/-   ##
==========================================
+ Coverage   66.80%   69.78%   +2.98%     
==========================================
  Files         230      252      +22     
  Lines       16602    15736     -866     
==========================================
- Hits        11091    10982     -109     
+ Misses       4808     3873     -935     
- Partials      703      881     +178     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Copilot]

Pull request overview

This PR makes check selection aware of the repository’s hosting platform by introducing a Repo.Type() API and filtering enabled checks based on the repos field in docs/checks/internal/checks.yaml. This reduces misleading or error-prone results from forge-specific checks (notably for Azure DevOps repos).

Changes:

• Add clients.RepoType and a new Type() RepoType method to the clients.Repo interface, implemented by GitHub/GitLab/Azure DevOps/local repos.
• Extend policy.GetEnabled to accept a repo type and skip checks whose checks.yaml repos field doesn’t include that type.
• Update checks.yaml repo-type tags and add/adjust tests and call sites to pass repo type where available.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
policy/policy.go	Adds repo-type-aware filtering via isSupportedRepoType during check enablement.
policy/policy_test.go	Adds coverage for repo-type filtering behavior (Azure DevOps + permissive empty type).
pkg/scorecard/scorecard.go	Passes repo.Type() into policy.GetEnabled during a run.
pkg/scorecard/scorecard_test.go	Updates mocks to expect Repo.Type() calls.
docs/checks/internal/checks.yaml	Makes repos field load-bearing and tags additional checks for Azure DevOps.
cron/internal/worker/main.go	Passes repo.Type() into policy.GetEnabled for cron execution.
cmd/serve.go	Updates policy.GetEnabled call signature (currently passes empty repo type).
cmd/root.go	Updates policy.GetEnabled call signature (currently passes empty repo type).
clients/repo.go	Introduces RepoType enum and adds Type() to the repo interface.
clients/mockclients/repo.go	Updates generated mock to include Type().
clients/localdir/repo.go	Implements Type() returning local.
clients/gitlabrepo/repo.go	Implements Type() returning GitLab.
clients/githubrepo/repo.go	Implements Type() returning GitHub.
clients/azuredevopsrepo/repo.go	Implements Type() returning Azure DevOps.

Comments suppressed due to low confidence (2)

docs/checks/internal/checks.yaml:370

• repos now includes Azure DevOps, but the description states the check is limited to GitHub and relies on GitHub user profile Company. Please reconcile this (update the description to match Azure DevOps behavior, or remove Azure DevOps from repos) so users aren’t misled and repo-type filtering is correct.

    repos: GitHub, Azure DevOps
    short: Determines if the project has a set of contributors from multiple organizations (e.g., companies).
    description: |
      Risk: `Low` (lower number of trusted code reviewers)

      This check tries to determine if the project has recent contributors from
      multiple organizations (e.g., companies). It is currently limited to
      repositories hosted on GitHub, and does not support other source hosting
      repositories (i.e., Forges).

docs/checks/internal/checks.yaml:543

• repos now includes Azure DevOps, but the description explicitly says the check is limited to GitHub and only detects GitHub apps / GitHub workflow usage. Please update the description to reflect Azure DevOps support (what signals are checked there), or drop Azure DevOps from repos if support is not complete.

    repos: GitHub, Azure DevOps
    short: Determines if the project uses static code analysis.
    description: |
      Risk: `Medium` (possible unknown bugs)

      This check tries to determine if the project uses Static Application Security
      Testing (SAST), also known as [static code analysis](https://owasp.org/www-community/controls/Static_Code_Analysis).
      It is currently limited to repositories hosted on GitHub, and does not support
      other source hosting repositories (i.e., Forges).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[jeffmendoza]

@JamieMagee Can you take a look at the three unresolved copilot comments and consider if they are appropriate? The first one looks good (caching the parsed data).

[Copilot]

Pull request overview

This PR makes check selection platform-aware by filtering enabled checks based on the repository’s hosting platform (as declared in docs/checks/internal/checks.yaml), preventing GitHub-specific checks from running on non-GitHub repos (notably Azure DevOps).

Changes:

• Extend policy.GetEnabled to accept a repoType and skip checks whose repos list in checks.yaml doesn’t include that platform.
• Add Repo.Type() clients.RepoType to the clients.Repo interface and implement it across repo types and mocks; wire repo.Type() into key call sites.
• Update checks.yaml to declare Azure DevOps support for 11 checks and add unit tests for repo-type filtering.

Show a summary per file

File	Description
policy/policy.go	Adds repo-type-based filtering using docs-derived repos metadata.
policy/policy_test.go	Adds coverage for Azure DevOps filtering and permissive behavior when repo type is empty.
pkg/scorecard/scorecard.go	Passes repo.Type() into policy.GetEnabled during runs.
pkg/scorecard/scorecard_test.go	Updates mocks to expect Repo.Type() calls.
docs/checks/internal/checks.yaml	Updates repos: declarations to include Azure DevOps for multiple checks.
cron/internal/worker/main.go	Filters enabled checks by repo.Type() for cron worker execution.
cmd/serve.go	Filters enabled checks by repo.Type() for server execution path.
cmd/root.go	Updates GetEnabled call signature (still passes empty repo type).
clients/repo.go	Introduces RepoType and adds Type() RepoType to clients.Repo.
clients/mockclients/repo.go	Updates generated mock to include Type().
clients/localdir/repo.go	Implements Type() as local.
clients/gitlabrepo/repo.go	Implements Type() as GitLab.
clients/githubrepo/repo.go	Implements Type() as GitHub.
clients/azuredevopsrepo/repo.go	Implements Type() as Azure DevOps.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (5)

policy/policy.go:160

• docs.Read() errors are silently ignored when building repoTypeLookup. If the embedded checks docs can't be read/unmarshaled, repo-type filtering is effectively disabled (all checks treated as supported), which can reintroduce the misleading results this change is trying to prevent. Consider returning an internal error when repoType != "" and docs.Read() fails (or at least surfacing/logging it) instead of failing open.

	if repoType != "" {
		if d, err := docs.Read(); err == nil {
			repoTypeLookup = make(map[string][]string)
			for _, c := range d.GetChecks() {
				repoTypeLookup[strings.ToLower(c.GetName())] = c.GetSupportedRepoTypes()
			}
		}

docs/checks/internal/checks.yaml:150

• Branch-Protection is now marked as supporting Azure DevOps, but the short/description text still specifically references "GitHub's branch protection" and links to GitHub-only settings. Either update the documentation to be forge-agnostic / describe Azure DevOps equivalents, or remove Azure DevOps from repos if the check isn't actually applicable there.

This issue also appears in the following locations of the same file:

• line 289
• line 362
• line 535

    repos: GitHub, GitLab, Azure DevOps
    short: Determines if the default and release branches are protected with GitHub's branch protection settings.
    description: |
      Risk: `High` (vulnerable to intentional malicious code injection)

      This check determines whether a project's default and release branches are
      protected with GitHub's [branch protection](https://docs.github.com/github/administering-a-repository/defining-the-mergeability-of-pull-requests/about-protected-branches) 
      or [repository rules](https://docs.github.com/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/about-rulesets) settings.

docs/checks/internal/checks.yaml:307

• Code-Review is now marked as supporting Azure DevOps, but the description explicitly says it checks for "an approval on GitHub". Also, the implementation detects GitHub reviews via AssociatedMergeRequest.Reviews/MergedBy (see checks/raw/code_review.go), which Azure DevOps commits don't appear to populate, so enabling this on Azure risks incorrect scores. Either update the check+docs to correctly support Azure DevOps reviews, or remove Azure DevOps from repos for this check.

    repos: GitHub, Azure DevOps
    short: Determines if the project requires human code review before pull requests (aka merge requests) are merged.
    description: |
      Risk: `High` (unintentional vulnerabilities or possible injection of malicious
      code)

      This check determines whether the project requires human code review
      before pull requests (merge requests) are merged.

      Reviews detect various unintentional problems, including vulnerabilities that
      can be fixed immediately before they are merged, which improves the quality of
      the code. Reviews may also detect or deter an attacker trying to insert
      malicious code (either as a malicious contributor or as an attacker who has
      subverted a contributor's account), because a reviewer might detect the
      subversion.

      The check determines whether the most recent changes (over the last ~30 commits) have 
      an approval on GitHub
      or if the merger is different from the committer (implicit review). It also

docs/checks/internal/checks.yaml:372

• Contributors is now marked as supporting Azure DevOps, but the description still says it is "currently limited to repositories hosted on GitHub" and describes using the GitHub user profile Company field. With repo-type filtering now enforced, this documentation needs to be updated to reflect Azure DevOps behavior (or remove Azure DevOps from repos if support isn't intended).

    repos: GitHub, Azure DevOps
    short: Determines if the project has a set of contributors from multiple organizations (e.g., companies).
    description: |
      Risk: `Low` (lower number of trusted code reviewers)

      This check tries to determine if the project has recent contributors from
      multiple organizations (e.g., companies). It is currently limited to
      repositories hosted on GitHub, and does not support other source hosting
      repositories (i.e., Forges).

      The check looks at the `Company` field on the GitHub user profile for authors of

docs/checks/internal/checks.yaml:543

• SAST is now marked as supporting Azure DevOps, but the description still says it is "currently limited to repositories hosted on GitHub" and the implementation looks for GitHub check runs and .github/workflows/* GitHub Actions usage. With repo-type filtering now enforced, either update the docs+implementation to properly support Azure DevOps pipelines, or drop Azure DevOps from repos to avoid misleading Azure results.

    repos: GitHub, Azure DevOps
    short: Determines if the project uses static code analysis.
    description: |
      Risk: `Medium` (possible unknown bugs)

      This check tries to determine if the project uses Static Application Security
      Testing (SAST), also known as [static code analysis](https://owasp.org/www-community/controls/Static_Code_Analysis).
      It is currently limited to repositories hosted on GitHub, and does not support
      other source hosting repositories (i.e., Forges).

• Files reviewed: 14/14 changed files
• Comments generated: 1

[Copilot]

Pull request overview

This PR makes check enablement forge-aware by filtering checks based on the repos field in docs/checks/internal/checks.yaml, so GitHub-specific checks don’t run (and mis-score/error) on other repo types like Azure DevOps.

Changes:

• Extend clients.Repo with a Type() RepoType method and thread repo type into policy.GetEnabled to filter checks by supported platforms.
• Cache docs/checks.Read() via sync.Once to avoid repeated embedded-YAML unmarshalling, and build a per-call lookup map for repo-type support.
• Update checks.yaml to mark additional checks as supporting Azure DevOps and add/adjust tests for filtering behavior.

Show a summary per file

File	Description
policy/policy.go	Adds repo-type-aware filtering to enabled-check selection using check docs lookup.
policy/policy_test.go	Adds test cases covering Azure DevOps filtering and permissive behavior for empty repo type.
pkg/scorecard/scorecard.go	Passes repo.Type() into policy.GetEnabled during runs.
pkg/scorecard/scorecard_test.go	Updates mocks to expect Repo.Type() calls.
docs/checks/internal/checks.yaml	Expands repos declarations to include Azure DevOps for 11 checks.
docs/checks/impl.go	Caches parsed check docs using sync.Once to avoid repeated YAML parsing.
cron/internal/worker/main.go	Passes repo.Type() into policy.GetEnabled for cron runs.
cmd/serve.go	Passes repo.Type() into policy.GetEnabled for server requests.
cmd/root.go	Updates GetEnabled call signature (passes empty repo type).
clients/repo.go	Introduces RepoType and adds Type() to the exported clients.Repo interface.
clients/mockclients/repo.go	Regenerates mock to include Type() method.
clients/localdir/repo.go	Implements Repo.Type() returning local.
clients/gitlabrepo/repo.go	Implements Repo.Type() returning GitLab.
clients/githubrepo/repo.go	Implements Repo.Type() returning GitHub.
clients/azuredevopsrepo/repo.go	Implements Repo.Type() returning Azure DevOps.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (4)

docs/checks/internal/checks.yaml:236

• The CI-Tests entry now lists GitLab/Azure DevOps in repos, but the description text below still states the check is “currently limited to repositories hosted on GitHub” and “does not support other source hosting repositories”. Please update the documentation to match the declared supported repo types (or adjust repos if support is still GitHub-only).

This issue also appears in the following locations of the same file:

• line 359
• line 466
• line 532

  CI-Tests:
    risk: Low
    tags: supply-chain, testing
    repos: GitHub, GitLab, Azure DevOps
    short: Determines if the project runs tests before pull requests are merged.
    description: |
      Risk: `Low` (possible unknown vulnerabilities)

      This check tries to determine if the project runs tests before pull requests are
      merged. It is currently limited to repositories hosted on GitHub, and does not
      support other source hosting repositories (i.e., Forges). This check only 

docs/checks/internal/checks.yaml:370

• The Contributors entry now lists Azure DevOps in repos, but the description text below still says the check is “currently limited to repositories hosted on GitHub” and doesn’t support other forges. Please update the documentation to match the declared supported repo types (or adjust repos accordingly).

  Contributors:
    risk: Low
    tags: source-code
    repos: GitHub, Azure DevOps
    short: Determines if the project has a set of contributors from multiple organizations (e.g., companies).
    description: |
      Risk: `Low` (lower number of trusted code reviewers)

      This check tries to determine if the project has recent contributors from
      multiple organizations (e.g., companies). It is currently limited to
      repositories hosted on GitHub, and does not support other source hosting
      repositories (i.e., Forges).

docs/checks/internal/checks.yaml:478

• The Pinned-Dependencies entry now lists Azure DevOps in repos, but the description text below still states the check is “currently limited to repositories hosted on GitHub” and doesn’t support other forges. Please update the documentation to match the declared supported repo types (or adjust repos accordingly).

  Pinned-Dependencies:
    risk: Medium
    tags: supply-chain, security, dependencies
    repos: GitHub, Azure DevOps, local
    short: Determines if the project has declared and pinned the dependencies of its build process.
    description: |
      Risk: `Medium` (possible compromised dependencies)

      This check tries to determine if the project pins dependencies used during its build and release process.
      A "pinned dependency" is a dependency that is explicitly set to a specific hash instead of
      allowing a mutable version or range of versions. It
      is currently limited to repositories hosted on GitHub, and does not support
      other source hosting repositories (i.e., Forges).

docs/checks/internal/checks.yaml:543

• The SAST entry now lists Azure DevOps in repos, but the description text below still states the check is “currently limited to repositories hosted on GitHub” and doesn’t support other forges. Please update the documentation to match the declared supported repo types (or adjust repos accordingly).

  SAST:
    risk: Medium
    tags: supply-chain, security, testing
    repos: GitHub, Azure DevOps
    short: Determines if the project uses static code analysis.
    description: |
      Risk: `Medium` (possible unknown bugs)

      This check tries to determine if the project uses Static Application Security
      Testing (SAST), also known as [static code analysis](https://owasp.org/www-community/controls/Static_Code_Analysis).
      It is currently limited to repositories hosted on GitHub, and does not support
      other source hosting repositories (i.e., Forges).

• Files reviewed: 15/15 changed files
• Comments generated: 2