14 min read

White-Box Testing
Intermediate
A03 A09

SAST and Scanning Tools

A static analyser is a compiler front end with a query language bolted on. It parses your code into a model, lets a rule describe a shape in that model, and reports matches. Everything the tool cannot find follows from that sentence — and so does every tool's licensing, because the expensive part is the model, not the query.

The three tools are not interchangeable

Semgrep is pattern matching over syntax trees: fast, writable by a developer in an afternoon, and blind across file boundaries. CodeQL builds a relational database and lets you write recursive queries over it: slow, steep, and the only one of the three that answers "does this value reach that sink through four layers". SonarQube Community Build is a quality platform with security rules attached, and its free edition deliberately omits the feature teams care about most.

How a SAST engine works

Every engine in this space does the same three things, in the same order. It parses the language into an abstract syntax tree or a control-flow graph. It adds a semantic layer — types, resolved imports, and for taint analysis a model of which functions are sources, which are sinks and which are sanitisers. Then a rule, written in the tool's own language, is evaluated over that model.

The consequences are predictable. If the tool cannot parse a language, it finds nothing — which is why "we run SonarQube" is not coverage. If the taint model does not know a framework's sanitiser, every correctly escaped value is a false positive, which is why framework-specific rule sets matter more than rule count. And if the analysis is intraprocedural, a value that travels through three helper functions is invisible, which is the single largest source of missed findings in a real codebase.

Tool Licence What is free What it will not find
Semgrep CE LGPL-2.1, rules under the Semgrep Rules License Unlimited local scans, single file at a time Cross-file and cross-service data flow
CodeQL Proprietary; free on public repositories Public repos and OSS projects Private repositories without a paid plan; no PHP or Scala
SonarQube Community Build LGPL-3.0 Local and self-hosted analysis Branch and pull-request analysis — that is the paid split
Checkmarx One Commercial Not applicable Nothing structural, but procurement is the gate
gitleaks MIT Unlimited Secrets that were never committed, and custom formats without a rule
osv-scanner Apache-2.0 Unlimited Anything without an advisory — a fresh takeover is invisible

The licence boundary is part of the finding

Two of these have a gap that a team discovers mid-rollout. CodeQL on GitHub is free for public repositories; on a private repository it needs a paid plan, and the CLI licence is restricted to academic and open-source use. SonarQube Community Build is genuinely free and genuinely without branch or pull-request analysis — the paid editions exist because that is the feature people want. Check both before writing the adoption plan, because retrofitting one later means re-running the whole rollout.

Semgrep: rules you can write

Semgrep's proposition is that a security engineer can express a convention in the language they are reviewing. A rule is a pattern that looks like the code it matches, with metavariables for the parts that vary. That is enough for the majority of real findings — a dangerous function called with an untrusted argument, a framework escape hatch, an unescaped template tag — and it is why the tool became the default first scanner.

semgrep/express.yml
rules:
  - id: express-unescaped-ejs
    languages: [javascript, typescript]
    severity: ERROR
    message: >
      Unescaped EJS interpolation (<%-) renders request data as markup.
      Use <%= unless the value is trusted, sanitised HTML.
    metadata:
      cwe: ["CWE-79"]
      owasp: ["A05:2025 - Injection"]
      confidence: HIGH
    patterns:
      - pattern: <%- $X %>
      - pattern-not: <%- include($Y) %>

  - id: go-sql-string-concat
    languages: [go]
    severity: ERROR
    message: Query built by concatenation; bind parameters instead.
    metadata:
      cwe: ["CWE-89"]
    patterns:
      - pattern-either:
          - pattern: $DB.Query("..." + $X + "...")
          - pattern: $DB.Query(fmt.Sprintf(...))
      - pattern-not-inside: |
          func $F(...) {
            ...
            $DB.Query($Q, ...)
            ...
          }

Two limits define the tool. Community Edition analyses one file at a time, so a value passed between files is invisible to a taint rule — the multi-file analysis is a commercial feature. And the rules themselves are not open source: the engine is LGPL-2.1, but the rule syntax and registry rules are covered by the Semgrep Rules License, which restricts redistribution. The Opengrep fork exists because of that split, and it is worth knowing which one a repository's CI is actually running.

semgrep-cli.sh
# install without a package manager dependency
python3 -m pip install semgrep

# scan the tree with the registry rule sets plus the repository's own rules
semgrep scan --config p/owasp-top-ten --config ./semgrep/ --error

# diff-aware: only report what this branch introduced
semgrep scan --config p/security-audit --baseline-commit origin/main

# SARIF for code scanning, when the scanner is not the GitHub-native one
semgrep scan --config p/security-audit --sarif --output semgrep.sarif

# validate a custom rule against its own test file before merging it
semgrep --test --config ./semgrep/

CodeQL: queries over a database

CodeQL extracts the codebase into a relational database — tables for expressions, calls, types, control flow — and rules are written as queries over it. That model is what makes cross-file data flow possible, and it is also why the tool is slow, why each language needs a purpose-built extractor, and why the language list is shorter than Semgrep's: currently ten languages, and neither PHP nor Scala is among them.

codeql.sh
# CodeQL CLI: databases are built per language, then queried
codeql database create ./db   --language=javascript   --source-root=.   --command="npm ci && npm run build"

# the suite must be qualified: a bare suite name does not resolve
codeql database analyze ./db   codeql/javascript-queries:codeql-suites/javascript-security-extended.qls   --format=sarif-latest   --output=codeql.sarif

# upload is a separate, authenticated step
gh api   -X POST   /repos/{owner}/{repo}/code-scanning/sarifs   -f sarif="$(base64 -w0 codeql.sarif)"

The suite path is the step everyone gets wrong

A bare suite name such as javascript-security-extended.qls does not resolve. The suite must be qualified with its query pack — codeql/javascript-queries:codeql-suites/…. On GitHub-hosted code scanning, default setup avoids the problem entirely; the hand-rolled CLI path is where it bites.
Default setup versus an advanced workflow

Default setup is maintained by GitHub, updates the query pack for you, and covers the standard query suites. An advanced workflow is a checked-in file that runs the CLI, which you need when you must build the project with a specific toolchain, scan a monorepo per language, or add custom queries. Choose advanced when the build command is non-default; choose default otherwise, because it is one less thing to maintain and it does not break when a runner image changes.

SonarQube Community Build

SonarQube was renamed at the community tier: what used to be the Community Edition is now the Community Build, under LGPL-3.0, self-hosted or run locally. It is the broadest single-tool view of a codebase — duplication, complexity, coverage and security rules in one report — which is why organisations adopt it first and then discover the gap.

That gap is branch and pull-request analysis. The Community Build analyses a branch as a standalone project, so there is no "new issues in this pull request" view and no quality gate on a merge. Every commercial edition exists above that line. If the requirement is a gate on a pull request, the free tier of this tool cannot provide it, and planning around that on day one is cheaper than discovering it in month two.

Secrets, dependencies and the gaps

SAST looks at code as written today. Two classes of defect live outside that frame and need their own scanner: credentials committed and later deleted, which remain in git history forever, and vulnerable dependencies, which are not defects in your code at all. The 2025 OWASP Top 10 promoted the second to A03 Software Supply Chain Failures, and it deserves its own gate rather than a line in the SAST report.

Gap Scanner Licence Failure mode
Secrets in history gitleaks MIT Only the formats it has rules for
Secrets, verified TruffleHog AGPL-3.0 Verification makes network calls; the licence matters for redistribution
Dependency advisories osv-scanner, npm audit, pip-audit Apache-2.0, bundled, Apache-2.0 No advisory means no finding, however malicious the package
Infrastructure as code Checkov, Trivy, Semgrep IaC rules Apache-2.0, Apache-2.0 Cloud-provider coverage varies by resource type
Design and logic flaws Nothing automated — Requires a human who understands the business

Wiring it into CI

A scanner that runs after merge produces a report nobody reads. The workflow below is the minimum that gates a pull request while keeping the existing backlog from blocking every build: scan on the pull request, upload SARIF so the findings appear inline on the diff, and run the slow full-history and dependency scans on a schedule.

.github/workflows/sast.yml
name: sast
on:
  pull_request:
  push:
    branches: [main]
  schedule:
    - cron: '0 3 * * 1'

permissions:
  contents: read
  security-events: write

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Semgrep
        run: |
          python3 -m pip install semgrep
          semgrep scan --config p/owasp-top-ten --config ./semgrep/ \
            --sarif --output semgrep.sarif --error || true

      - name: Upload SARIF
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: semgrep.sarif
          category: semgrep

      - name: Secrets in full history
        uses: gitleaks/gitleaks-action@v2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - name: Dependencies
        run: |
          npm ci
          npx osv-scanner --lockfile=package-lock.json

The scanner is code execution in your pipeline

A scanner parses untrusted input with a large parser, and a custom rule is executable configuration. Pin actions by commit SHA rather than tag, run the scan with a read-only token, and never give the scanning job write access to the repository it is scanning. The upload step needs security-events: write and nothing else.

Triage workflow

Triage is the step that decides whether the programme survives. Four outcomes, and each one has to be recorded or the same finding is re-investigated in three months:

Outcome When What gets recorded
True positive The path is real and reachable Ticket with the path, owner and severity
False positive The sanitiser or binding is real A suppression with a reason, and a rule improvement if possible
Unreachable Real defect, no route reaches it Low severity, with the reason — not a closure
Duplicate The same sink reached from two rules Link to the original, and remove the overlapping rule
Check Expected Where to look
Every scanner runs on the pull request, not only on the default branch A result that appears after merge is a report, not a gate The workflow triggers
The gate is scoped to new findings A baseline or diff-aware mode, so an existing backlog does not fail every build The scan step configuration
Results land in one place with a stable format SARIF uploaded to code scanning, or a single aggregator — not three vendor dashboards nobody opens The upload step and its permissions
Secrets scanning covers history, not just the working tree A full-history scan on a schedule, and a pre-commit hook for the fast path The scheduled workflow and the hook
The rule set is versioned with the repository Custom rules reviewed like code, with a test case each — a rule that fires on nothing is worse than no rule The rules directory and its tests

Evidence Collection

A scanner finding is a hypothesis. Before it becomes a ticket it needs three things attached:

The scanner, version and rule id: A finding that cannot be reproduced by rerunning the tool is a finding that will be dismissed. Pin the tool version in CI so the rerun is possible.

The data path, if the scanner shows one: Semgrep and CodeQL both emit a path when they have one. Attach it, and check by hand that every step is on a real route.

The reason it is not a false positive: One sentence, naming the framework or the config that does not mitigate it. This is the field that decides whether the ticket gets fixed.

Bedrock Questions Before Filing

No authoritative false-positive rate exists for SAST, and every published percentage is a benchmark artefact. Judge each finding on its path instead:

  • Does the taint path terminate at a real sink: Many findings stop at a function named sanitize that sanitises nothing. Read it.
  • Is the path reachable from a route: Dead code, test fixtures and CLI-only utilities produce findings that will never be exploitable. Say so, and lower the severity rather than closing it.
  • Does the framework already handle it: Spring, Rails, Django and ASP.NET all bind parameters and escape output in their default path. A finding that assumes raw concatenation against an ORM is usually wrong.
  • Is the rule simply out of date: Framework versions move. A rule written for Spring 5 firing on a Spring 7 codebase is maintenance debt in the rule set, not a finding.

Scanning Programme Checklist

The tool is the easy part. These are the properties a scanning programme needs to stay useful after month three.

  • ☐ Every language in the repository has a scanner that parses it — including IaC, Dockerfiles and shell
  • ☐ Secrets scanning runs over full history on a schedule, not only over the diff
  • ☐ Dependency scanning runs on the lockfile, and on every base image
  • ☐ Generated code and vendored dependencies are excluded deliberately, and the exclusion is documented
  • ☐ Custom rules exist for the conventions this codebase keeps getting wrong
  • ☐ Every custom rule has a positive and a negative test case
  • ☐ The default rule set is pinned by version, not by "latest"
  • ☐ Triage decisions are recorded, so the same false positive is not re-investigated every quarter
  • ☐ The gate blocks new findings at a chosen severity, and only new ones
  • ☐ Results are visible where developers already work — the pull request, not a dashboard
  • ☐ A finding has an owner and a due date, derived from severity rather than from the reporter
  • ☐ The scanner has no write access to the repository it scans

Programme Notes

Most scanning programmes fail on adoption rather than on detection.

Why benchmark percentages are not your false-positive rate

The OWASP Benchmark and similar corpora measure a scanner against synthetic cases with one defect each. Real code has framework wrappers, generated layers and three sanitisers in series, and every one of those changes the rate — usually in the direction the vendor does not publish. There is no authoritative cross-tool false-positive percentage, and quoting one from a blog post is how a programme sets an expectation it cannot meet. Measure your own rate after a month, on your own code, and use that number.

Scanner Practice Targets

Repositories with known findings you can use to calibrate a tool against your own review rather than against its marketing.

OWASP Benchmark Custom Lab medium
Annotated true and false positives per CWECompare two scanners on the same corpusStudy which findings a rule cannot express
OWASP Juice Shop Custom Lab easy
Run Semgrep and CodeQL against a modern Node codebaseTriage the output against known challengesWrite one custom rule for a defect you found by hand
Damn Vulnerable Web Application Custom Lab easy
PHP: demonstrate that CodeQL does not cover itCompare Semgrep coverage with the challenge listPractise suppression with reasons