AI Regulatory Writing: A Source Traceability Checklist

This checklist is for a regulatory medical-writing lead evaluating an AI authoring workflow. It is an original procurement resource published by Assyro, with a fictional example and no claimed vendor test results. It assesses evidence handling; completing it does not establish scientific validity or agency acceptance.

The source-traceability checklist

Before the session, record the document type, intended output, product configuration, permitted sources, source versions, evaluator, and date. Agree which checks are mandatory for your purchase. Give the vendor the same small source packet you will use to review its output.

Copy the following items into your evaluation record. For every item, retain the result, evidence location, reviewer, and any corrective action. “Unknown” means the demonstration has not established the answer; “fail” means an observed result violates the agreed criterion. Use “not applicable” only with an explicit reason accepted by the evaluation owner.

Run T01–T07 on the initial draft, T08–T09 after replacing a source, and T10 on the exported deliverable. The writer can check locations and wording; a suitably qualified reviewer should assess whether the source supports a scientific interpretation. Assign someone to close each unresolved issue.

A citation has several jobs

Source identity tells you which document was used. Location tells you where to look. Support establishes whether the passage justifies the assertion. Version history lets you reconstruct that relationship later. Test these separately: a product can succeed at opening a PDF and still attach it to an unsupported sentence.

NIST's July 2024 Generative AI Profile identifies fabricated supporting logic and citations as a risk that can increase misplaced trust. It is voluntary, cross-sector guidance, not a regulatory-writing certification. The checklist here is our practical evaluation method. NIST AI 600-1, introduction and section 2.2

For a PDF, retain the source revision, table or section identifier, and page convention. Printed page 7 and PDF page 9 may refer to the same location. For a spreadsheet, record the workbook version, sheet, row/column identifiers, and relevant headers or notes. A bare cell address can change when rows are inserted.

When data are transformed before drafting, ask to see that intermediate output. A correct citation to the original table cannot explain a value produced by an undocumented transformation. Likewise, a search result or another generated summary should not silently replace the selected primary source.

For a CMC-specific source-change case, the QOS authoring exercise follows a summary back to the selected quality dossier.

Apply the checklist to this fictional source packet

The following excerpt is invented solely to test writing and evidence review. It represents no real study or treatment, and it supplies no basis for a clinical recommendation.

The selected report is SYN-TRACE-01, revision 1. Table E-04, on printed page 7, reports the observed mean change from baseline at Week 8 in a fictional score, in points. Its population is participants with the specified Week 8 assessment: group A has 30 participants and group B has 28. The table values are A = −4.2 points and B = −3.9 points.

The supplied footnote identifies these as descriptive values. The packet contains no inferential comparison, confidence interval, p-value, definition of clinically meaningful change, or evidence covering every randomized participant. Both groups' rows and the footnote are available to the evaluator.

Inspect a sentence with a real source and an unsupported conclusion

Give the reviewer this deliberately defective draft:

“

Group A significantly outperformed group B in all randomized participants, with a mean change of −4.2 points at Week 8. [SYN-TRACE-01, revision 1, Table E-04]

The reference identifies a real source within the fictional packet. The −4.2 value, unit, and visit match that source. The sentence nevertheless fails: the source does not establish statistical superiority, and its assessed population is not stated to include all randomized participants.

Do not mark the whole sentence verified because its number appears on the cited page. Split it into assertions: group, population, statistic, value, time, and comparative conclusion. Each needs support. The absent inferential evidence is unknown; presenting significance as established is a failure of this draft's evidence criterion. Those are different dispositions.

An acceptable descriptive replacement is:

“

In SYN-TRACE-01, the observed mean change from baseline at Week 8 was −4.2 points in group A and −3.9 points in group B among participants with the specified Week 8 assessment (30 and 28 participants, respectively). These descriptive values do not establish statistical superiority.

Attach each group clause to its corresponding E-04 row and attach the scope qualification to the footnote and packet definition. The replacement retains what is known without deciding what the score change means clinically.

Complete the evidence record

These results assess the specified source and passages, not a vendor. In a demonstration, retain the actual generated output beside the same record. If a tool prevents the defective statement or raises the missing-evidence question, record that observed behavior rather than attributing the planted error to it.

A not-applicable result is local to this output. If the author adds a calculated contrast, T07 becomes applicable and the inputs, calculation, and permitted interpretation need review. It cannot remain exempt because an earlier draft contained no arithmetic.

Replace the source without rewriting history

Now supply SYN-TRACE-01 revision 2, explicitly selected by the evaluation owner to supersede revision 1 for the next draft. A documented correction changes group A's value to −4.0 points. The table has moved to printed page 9; its identifier remains E-04. Group B, the populations, Week 8 definition, and descriptive-only scope are unchanged.

Ask the demonstrator to identify the affected passage before approving a revision. The new draft should use −4.0 for A, retain −3.9 for B, and locate E-04 on page 9 of revision 2. It should not add statistical significance or expand the population because a newer report exists.

Then reopen the original reviewed draft. Its −4.2 statement should still be reconstructible against revision 1, page 7. A link that always opens the newest report would obscure the basis of that historical statement. Keeping both PDFs is useful only if the relationship between each draft and its evidence remains identifiable.

Test two related boundaries separately:

• Conflicting revisions: provide another file with a different value but no instruction establishing which source supersedes which. Keep the affected assertion unresolved until the responsible owner resolves the conflict; upload order is insufficient.
• Incomplete meaning: provide an isolated value without its unit, group, statistic, or table header. Ask for clarification or exclude it from drafting with a recorded reason. Matching the number to a nearby passage does not establish those missing attributes.

If the wrong location appears, inspect where the error entered. Did extraction lose the header, source selection choose the wrong revision, or the citation anchor remain tied to an old page? The observed record should determine the cause. Correct that boundary, rerun the same case, and inspect other claims that reuse it. Merely changing the visible sentence leaves the underlying error available to recur.

Verify what leaves the application

Request the output you intend to buy: for example, a reviewed document and a separate evidence ledger. Internal source material need not all appear in the submitted document, but the authorized receiving reviewer must be able to reconstruct the evidence through the agreed records.

Open the export outside the authoring session. Follow one unchanged claim, the corrected A claim, and an unresolved issue. Check that version identifiers, locations, and dispositions survive. If evidence links require an account, test them with the recipient's permitted access; a successful click in the vendor operator's session is insufficient.

Record the handoff as pass, fail, or unknown for the contracted format. A PDF demonstration does not establish an editable Word handoff. An unexported screenshot may show a useful feature but does not prove your retained evidence will remain usable after the session or contract ends.

Review Assyro’s document-management scope when deciding where the authoritative source and exported evidence will remain.

Make the buying decision from the record

Keep mandatory traceability failures and unknowns open until resolved. Do not average them away with drafting speed or an attractive interface. Count what you actually inspected—selected claims, source types, revisions, and exported records—and describe that sample when reporting results. A successful small sample cannot establish an accuracy rate for all future documents.

Assyro's authoring workflow is a starting point when evidence review needs to connect with regulatory document preparation. Bring this packet to an Assyro demo and ask which steps the proposed configuration performs. The exact claim-level behavior in this checklist must be demonstrated; general authoring availability does not establish a pass.

Retain the completed checklist with the proposal. It should tell your team which statements were supported, which needed correction, which evidence was missing, and what the recipient could reconstruct after export. That record makes “source traceability” a purchase requirement you can inspect.

Carry the claim-level findings into the AI medical-writing pilot plan when comparing total review burden.