EDMS Proof of Concept: Test Scenarios and Scorecard

An EDMS proof of concept should establish whether a defined configuration can perform your required document workflow, including exceptions. Use the same permitted sample and acceptance criteria for each candidate. Qualify mandatory requirements first; compare preferences only after that gate passes.

The checklist below is a procurement resource. It is not a complete validation protocol, regulatory approval or proof that every future document will work. Its purpose is to replace a general product tour with inspectable evidence for a bounded buying decision.

1. Freeze the scope and evidence record

Before the demonstration, record the product, edition, build or release, configuration, environment and proposed licenses. Identify the process owner and the person authorized to accept evidence. Name the record classes and the workflow being evaluated; a generic request to show “document management” is too ambiguous.

Use sanitized or synthetic documents and authorized test accounts. Keep production records, real restricted content and destructive operations outside the exercise unless they are explicitly permitted and controlled. A sandbox can demonstrate important behavior, but any differences from the proposed production configuration must remain visible.

Copy this result record for every test:

Pass means the defined criterion was observed with evidence. Fail means an observed result contradicts it. Unknown means the required evidence is missing or the test was not completed. N/A needs an approved scope reason. A blank result is Unknown.

2. Establish the regulatory boundary separately

For FDA-regulated electronic records, assess Part 11 applicability using the underlying record requirements and how the organization relies on the electronic record. FDA's scope guidance also describes enforcement discretion for specified provisions while retaining underlying obligations. A vendor's product label does not make that assessment for you. FDA Part 11 scope and application guidance.

Where applicable, closed-system controls in section11.10 address matters including intended performance, copies, retrieval, access and audit history. Those topics inform relevant procurement checks, but this abbreviated checklist does not reproduce all requirements or determine the complete assurance work for your use. 21 CFR11.10.

Have the responsible quality and records owners define applicable criteria. Use the computerized-system validation guide for the separate configured-use assessment. A successful demonstration is evidence to consider, not permission to bypass that assessment.

3. Prepare one representative document packet

The following packet is fictional and can be recreated with synthetic content. Adapt it to your own requirements; it is not a vendor import format.

• DOC-101 revision2 is effective; revision3 is a draft.
• DOC-202 revision1 requires review by two named roles.
• DOC-303 is a historical record with an agreed evidence package.
• SITE-A and SITE-B each have a similarly named procedure with different applicability.
• One sample file deliberately lacks a mandatory metadata value.
• Author, approver, ordinary reader and excluded test-user roles are defined.

Prepare expected source IDs, revision labels, status, applicability and the agreed content checks before import. Include a representative table, image or special character if your real records depend on those features. A clean text-only sample cannot establish fidelity for more complex content.

Record the packet version. If it changes during a demonstration, identify what changed and which results need repeating. Otherwise two vendors may appear to have completed the same exercise while actually processing different inputs.

For a minimal reproducible starting packet, create these three synthetic files and a matching manifest. The text is a fidelity marker, not a manufacturing instruction:

For a Site A effective-document request, the expected content is the 10 µg / EX-01 / 1.25 row. Returning 20 µg shows draft substitution; returning 30 µg shows wrong-site selection. Retain each file and its source manifest before import so the expected answer does not come from the system under evaluation. This tiny packet tests selected identity, state and text preservation only; add the other required packet items and representative complex formats before accepting the full scope.

4. Run the mandatory scenarios

Mark rows mandatory only where they belong in the approved scope. Split a row into separate test records when different roles, document classes or configurations can produce different outcomes.

If electronic signatures are part of the purchase, add a dedicated protocol for the applicable signature requirements and the actual signing workflow. A generic approval button is not a complete signature assessment. Similarly, backup restoration, retention execution and integration behavior may need separate exercises and supplier evidence beyond the demonstration window.

Ask the vendor to identify any administrator-only action, manual repair or additional application used. A successful result can still depend on unquoted services or unavailable permissions. Capture that dependency before treating the requirement as commercially covered.

5. Apply the gate before preference scoring

A candidate is not qualified while an applicable mandatory requirement is Fail or Unknown. An approved N/A is an exclusion with a reason, not a positive test result. Keep the scope decision and acceptance owner in the record.

For example, a supplier may show excellent authoring but be unable to produce the agreed historical export. If historical retrieval or exit is mandatory, the candidate remains on hold. A high usability score cannot compensate for missing records.

Distinguish a product limitation from incomplete evidence. “The required export has not been demonstrated” is different from “the product cannot export it.” Assign the next action: request evidence, remediate and retest, revise scope through the authorized owner, or reject the candidate for the stated use.

6. Use this preference scorecard only for qualified candidates

The following weights are an illustrative starting point, not an industry benchmark. Change them before testing to match the buyer's priorities. Record the observation behind every rating; unknown evidence cannot receive an assumed score.

Use the same four rating definitions for all candidates: 0 does not meet the stated preference; 1 meets it only with significant accepted burden; 2 meets the defined preference; 3 meets the pre-agreed preferred outcome. Define what burden and preferred outcome mean for each task before the demonstration. These ratings supplement mandatory qualification; they do not replace it.

Preference score = sum of each weight × rating ÷3. The maximum is100 because the weights sum to100. Report the result to a sensible precision and retain the underlying observations. If any rated preference lacks evidence, report the score as incomplete instead of substituting zero or a favorable estimate.

For a fictional qualified candidate with ratings 2,3,2,3, the calculation is20+25+16.67+20, or 81.67/100. If the administration observation warrants1 instead of3, the total falls to 65/100. That sensitivity shows why the documented operating burden matters; these are not scores for named products.

7. Work through one completed result and its failures

For POC-02, the fictional expected result is DOC-101 revision2, effective. The tester's retained retrieval record shows revision2 with the correct identity and status. Record Pass, the evidence location and the configuration used. This establishes that bounded result, not all future version handling.

Now the same exercise returns revision3 merely because it was uploaded most recently. Record Fail. Assign the version-selection rule to the relevant configuration or integration owner, correct it and repeat the affected retrieval cases. Do not repair the evidence by relabeling the returned draft as effective.

In a third run, the screen displays the right title but no version identity and the supplier has not provided another way to verify it. Record Unknown. Request the missing evidence; do not infer a pass from the filename.

An N/A example is a controlled-print test excluded from a scope that does not use printed controlled copies, with the records owner approving that boundary. The exclusion would not justify skipping version retrieval, which remains necessary for the stated packet. Keep exclusions specific.

8. Close findings and reconcile the proposal

For every unresolved item, record the owner, requested evidence or correction, due date and acceptance decision. Repeat the directly affected tests after a change and inspect neighboring requirements that share the changed configuration. Preserve earlier results so the team can understand why the decision changed.

Finally reconcile the successful demonstration with the proposal. Confirm applications, roles, environments, services and ongoing responsibilities. Separate standard capability from configured behavior and from implementation work still to be delivered. A future promise is not present evidence, even if it appears in the commercial discussion.

Retain the packet, requirements, results, exceptions, preference calculations and scoped proposal as the decision package. For broader requirements, use the QMS document-control guide. If evaluating Assyro for a specific regulatory-document task, bring this packet to a scoped demonstration and apply the same evidence rules. Assyro publishes this guide; the scorecard does not grant it an automatic pass.