10 min read

Operations

Operating the Control Set

A control that nobody owns, that cannot be evidenced and that has never been reviewed is a configuration, not a control. This page is the operating model for the rest of the section: who owns each layer, what proves it exists, how an exception is granted and closed, what an agent incident looks like in the telemetry that actually exists, and what to ask a model provider before trusting a summary of its terms.

Who Owns Which Control

Ownership is assigned per layer, never per tool. A policy that arrives through Intune belongs to the team that owns the endpoint estate even when a security team wrote its values, and a rule that lives in a GitHub ruleset belongs to the organisation owner who can change it. Name a role rather than a team, because a layer with two accountable owners has none.

Layer Accountable owner Artefact that proves it exists
Device baseline and policy delivery Endpoint engineering Assignment report plus policy diagnostic output from a sampled device in each ring
Editor and extension allowlist Developer platform The allowlist in source control, and the applied value read back from a machine
Assistant configuration Platform security The managed configuration file at its version, and the console-side organisation deny
Repository platform Source control owner Ruleset export, push-protection settings, default workflow token setting
Pipeline CI owner Workflow policy and the allowed-actions configuration
Network and data boundary Network security Egress ruleset, and a DLP policy that names the developer endpoints it covers

Read the right-hand column as a test rather than a description. If nobody can produce the artefact for a layer, the layer is a claim, and the first incident review will establish which layers were real. Ownership also has to be assigned for the exception register itself, because the register is the only place where the estate records what it deliberately does not control.

Evidence to Capture

Evidence is a dated artefact that can be reproduced, not a screenshot of a settings page whose values have since changed. For each control, capture the output of the thing that reads the setting back: Developer: Policy Diagnostics for a device in each ring, the assignment report for the policy itself, the managed configuration file and its version, the allowlist as committed, the ruleset export, the egress rule, and the DLP policy's coverage list.

Two controls need more care than a screenshot. Where OpenTelemetry export is configured for the editor, record whether content and identity capture are switched on: both are off by default, so a default-configured export proves a session occurred and says nothing about what was asked. Where the evidence is an audit-log query, record the filter and the time window with the result, because the answer will be re-requested by someone else.

Keep the artefacts with the change that produced them. A policy diagnostic from before the last rollout is evidence of a previous state, and an assessor who is handed one without a date has been handed nothing. Where a control is delivered by configuration management, the commit is the evidence and the device read-back is its proof.

Exceptions and Break-Glass

An exception names the control being waived, the reason, the compensating control, the data involved and the date it expires. It is approved by the layer owner and reviewed by security, and the expiry is a closure date rather than a reminder: an exception that renews itself because nobody looked is a permanent waiver with paperwork. Time-box it to the life of the work that needs it, not to the length of an audit cycle.

Break-glass needs a different design, because a managed deny cannot be lifted locally. Anything an engineer might legitimately need in a hurry, such as a push, a package publish or a directory outside the project, belongs in an ask rule that a second person can approve rather than in the deny layer that no local change can reach. Decide in advance who can approve, log the approval, and review every use at the next cycle. If the answer to an incident is that the control had to be disabled, the control was drawn in the wrong layer.

Detecting an Agent Incident

Four sources carry most of the signal, and each answers a different question. The GitHub audit log is the platform record: filter on action:copilot for feature and policy activity, and on actor:Copilot for work attributed to the agent. Retention is 180 days, so the export cadence is part of the control rather than an administrative preference: an incident discovered after the window on a non-exporting estate has no platform history at all.

Endpoint logs answer what ran: process creation under the developer's account, file writes outside the repository, and the client process itself starting. Egress logs answer where the work went, and they are the only source that shows a session against a provider nobody approved, unless the model is local, in which case there is no egress to see. OpenTelemetry export from the editor answers when a session happened, and at its defaults it does not answer what the session was asked. Policy diagnostics answers none of these questions: it reports configuration, and configuration is what you intended, not what occurred.

Local prompts and client session data are not in the audit log, so a credential pasted into a chat window leaves no platform-side record, and a session against a local model may leave only endpoint artefacts. The shape of an agent incident in the logs you do have is usually volume and timing: a burst of commits under one identity, a broad read across directories the engineer does not normally touch, a package install that no manifest explains, or a push at an hour the account has never pushed before. Treat those as leads, not as detections, and correlate with the endpoint record before concluding anything.

When a Secret Leaks

The order matters more than the speed. First establish whether the secret was ever valid, because a placeholder pasted into a prompt is a process failure and a live token is an incident; then revoke, then rotate. Revoking first is deliberate: rotation alone leaves the old value usable for as long as the issuer honours it, and an agent that already copied the value elsewhere is unaffected by a new one in the repository.

Then read the bypass record rather than the alert. A push-protection bypass requires a reason, raises an alert and writes an audit-log event, so the interesting question is which bypasses were granted around the commit and who granted them. Scope the exposure through history, branches, forks and CI logs, and check whether the agent could have read the file at all: reads of .env and .env.* ask by default, and an approval can turn that ask into an allow that persists for the project.

The defaults are worth knowing before an incident rather than during one. Push protection is off by default for repositories and on by default for users on public repositories on GitHub.com, so a private repository may have blocked nothing at all. That asymmetry is the reason the bypass review and the secret-scanning alert queue both belong in the quarterly checklist rather than in the incident runbook alone.

Reviewing Provider Terms

Ask four questions of every provider, in writing, and get the answer from the provider's own published terms rather than from a colleague's summary: whether prompts and code are used for training, what the retention period is and how deletion works, where processing and storage take place, and which subprocessors are involved and how changes to that list are notified. Add the tier to every question, because a consumer plan and an enterprise agreement are different contracts over the same endpoint.

No answer here is safe to repeat from memory, including the one that reads most settled. A provider's statement about training use applies to the surface it describes, under the terms in force when you read it, and not to every product that company ships. Cite the provider's Trust Center or data-processing terms and the date you read them. Do not state a vendor's retention or training default as settled on the strength of a blog post or a colleague's summary, and do not assume that a local model removes the question: it changes which contract applies.

Quarterly Review Checklist

One pass per quarter, with the output filed against the register rather than discussed in a meeting.

  • Re-read the policy surface for each editor in use, and record what changed since the last pass.
  • Run policy diagnostics on a sample of devices per ring, and compare the applied values with the intended ones.
  • Diff the deployed assistant configuration against the copy in source control.
  • Confirm the extension allowlist still matches what is installed, and that nothing outside it is permitted.
  • Review open exceptions against their expiry dates, and close the ones the work no longer needs.
  • Check that audit-log exports cover the whole quarter with no gaps, and spot-check one agent-attributed commit.
  • Re-read the provider terms for the models in use, and record the date and the answer.
  • Pick one editor nobody governs, test what the compensating controls actually stop, and record the result.

The last item is the one that keeps the rest honest. A control set tested only where it is expected to work is a description of intent; a control set tested against the editor nobody manages is a description of the estate.

The Audit Pack

What to hand over when someone asks whether the control set exists. Each item is an export or a read-back, not a screenshot of a page.

Policy state: The policy diagnostic output from a sampled device in each ring, dated, showing applied and not-applied values and the account policy gate state.

Delivery record: The assignment report for the policy, and the configuration-management commit that deployed the assistant configuration.

Configuration versions: The managed configuration file at the version deployed, and the extension or model allowlist as committed.

Platform settings: A ruleset export, the push-protection settings per repository class, and the default workflow token setting.

Activity record: An audit-log extract filtered on action:copilot and actor:Copilot for the period under review, with the filter and window recorded alongside it.

Exception register: Open exceptions with owner, compensating control and expiry, plus the closures since the last review.

False Positives When Reading Agent Activity

Most of what looks like agent misbehaviour in these sources is the estate working as designed. Rule each of these out before escalating.

  • Bulk commits: A mechanical refactor or a generated migration produces a burst of commits under one identity that matches the volume signature of an agent loop.
  • Approved egress: Traffic to a provider on the allowlist during working hours is the control working, not an exception to it.
  • Saved approvals: A saved allow pattern is configuration. It belongs in the review of the configuration, not in the incident queue.
  • Template files: Secret-scanning alerts raised by example values in documentation, fixtures and test data are the classic false positive of push protection.
  • Agent-attributed commits: An agent-attributed commit means the agent produced the change, not that no human read it; the review record is the evidence, not the attribution.
  • Configuration evidence: A diagnostic showing a policy as not applied may mean the policy is malformed rather than that the device drifted, and both are worth a ticket of their own.

Closing the Loop

The corrective work is ownership, observability and expiry — the three properties that decay first.

Assign and record

  • • Name one accountable owner per layer and file the artefact they owe
  • • Move every exception into the register with a closure date
  • • Record which controls have no artefact yet, and treat that as the backlog

Make it observable

  • • Schedule audit-log exports inside the 180-day window
  • • Decide deliberately whether editor telemetry captures content and identity
  • • Test one ungoverned editor each cycle and write down what the compensating controls stopped

Keep it accurate

  • • Re-read provider terms and record the date and the answer
  • • Re-run policy diagnostics per ring after every rollout
  • • Retire approvals and exceptions that outlived the work that justified them