Skip to content
Open menuClose menu

Draftv0.2Open for reviewer comment

Testing change management when AI writes or approves code

Seven tests an IT auditor can run with git and the code host's audit log. No product is needed.

About this document

A draft procedure for IT auditors, proposing how to test a change-management control when AI assistants write code, or bots and agents approve or merge it. It sets out seven vendor-neutral tests that run with git, the code host's audit log and standard tools, and maps them to SOC 2, ISO 27001, SOX ITGC and PCI DSS change controls. It is not an audit standard or an opinion on any control.

Who it is for

External and internal IT auditors testing SOC 2, ISO 27001, SOX ITGC or PCI DSS change controls, and the teams preparing for them.

What's inside

  1. T0Repositories are tiered by risk
  2. T1Population includes every change, however it was made
  3. T2Every actor is identified as a person, a bot or an agent
  4. T3The approver is independent of the author
  5. T4A person reviewed the change where policy requires one
  6. T5The controls themselves stayed on
  7. T6AI involvement is declared where policy requires it

Preview

Open in browser

Versions

VersionDateWhat changedSizeSHA-256PDF
Draft v0.2Adds "Sample size": test what the data can prove on every change (T1, T2, T3 and T5 across the whole population by analytics once the data source is validated), sample only what needs judgement (around 25 to 60 items, weighted towards high-risk repositories and agent-authored changes), and investigate every exception individually.175 kBf6243405…8ee8bbDownload PDFOpen in browser
Draft v0.1Adds "Known limit, by design": AI authorship cannot be verified, and the procedure does not depend on it. Question 1 now asks about T0 to T5 and whether T6 is worth keeping as a policy test.166 kB0d637a6e…452273Download PDFOpen in browser
Draft v0First published version163 kB154e6c53…edc6bfDownload PDFOpen in browser

A link you received always shows that version. A later version is a new file and a new row here; no file is ever replaced.