A completed task is not proof the workflow stayed in bounds
Administrators often get a summary that looks finished and still cannot answer a simple investigation question: what did the agent touch, where did outputs land, and who can reconstruct the session tomorrow.
For Tampa Bay law-firm administrators and IT leads supporting Florida pilots, that gap shows up when agent tools leave chat and start working with folders or external services. The need is an evidence and stop checklist, not another high-level policy paragraph.
This piece is a practical administrator checklist. It is not legal advice and it does not claim that any checklist satisfies ethics obligations.
Keep this checklist distinct from the intake decision
The companion guide Decide what information may enter an AI agent workflow covers which information may enter a workflow in the first place. Do not duplicate that approved-input table here.
This checklist starts after a task is attempted. It asks what to capture, when to stop, and who owns retention of the evidence. Florida readers can still open Florida Bar Ethics Opinion 24-1 with counsel for oversight themes. The opinion page states that advisory ethics opinions are not binding. Technical evidence supports counsel’s review; it does not replace it.
Record these items for every pilot task worth expanding
Use synthetic or public materials for early tests. Keep real matter files out until intake approval exists elsewhere.
| Checkpoint | Pass signal | Stop signal |
|---|---|---|
| Task definition | Written goal, allowed folder or data class, named requester | User improvises a new goal mid-session without a record |
| Session identity | Intended pilot user, device, and app version captured | Shared credentials or an unmanaged device appears |
| File reach | Agent reads only the approved mount or upload set | Unexpected paths, sibling folders, or sync roots appear |
| Write behavior | Creates or edits only in the approved output location | New files appear in uncontrolled locations |
| External actions | Browser, connectors, and MCP remain disabled if out of scope | Tool attempts network or connector actions the pilot forbids |
| Human review | Named reviewer checks output against source before reuse | Output is pasted to a client, filing, or ticket without review |
| Evidence landing | Sample events visible in the monitoring destination the firm trusts | Security cannot find the task in audit, Compliance, or OTel paths |
| Stop ownership | Named person who can pause the workflow the same day | Nobody knows who can revoke access or kill sessions |
Keep the table short enough that administrators will use it. A twelve-page runbook that nobody opens during an incident is not a stop procedure.
Synthetic examples that make gaps visible
Use boring tasks on purpose. Dramatic demos hide weak evidence.
Example A: public PDF summary. The user asks for a short summary of a public policy PDF in an approved folder. Afterward, confirm which files were opened, where the summary was saved, and whether any neighboring internal documents were readable. If the agent can see a home directory or a full matter library mount, stop expansion until the mount is narrowed.
Example B: compare two public clauses. The user asks the agent to compare two approved public excerpts and write a difference list. Inspect whether the comparison pulled unmarked files. Confirm the difference list is treated as draft work product until a human reviews it.
Example C: “fix the formatting” request. The user asks the agent to clean a synthetic memo. Watch for silent renames, duplicate copies, or edits outside the approved output folder. Unexpected writes are a stop condition even when the text looks helpful.
None of these examples is a client matter. They exist to prove whether evidence and boundaries work before stakes rise.
Decide who owns retention before the first real incident
Monitoring is its own data boundary. OpenTelemetry exports, compliance APIs, and endpoint logs can contain file paths, prompts, or tool details that need access control and retention decisions.
Name three owners in writing:
- Operational owner for day-to-day pilot support and user removal.
- Security evidence owner for where session and tool events land, and how long they stay.
- Incident escalation owner for unexpected file access, suspected prompt injection, or client-data exposure concerns.
If those names are missing, do not treat a successful demo as a launch. A firm that cannot reconstruct a session will also struggle to explain what happened after a complaint.
Validate claims against current vendor documentation for the products in use. Coverage differs across audit exports, compliance APIs, and telemetry pipelines. Test a sample task end to end instead of assuming a green admin console equals investigation readiness.
Stop conditions that should pause expansion
Pause the pilot, or at least block the next capability, when any of these appear:
- Restricted or client material is reachable without a separate intake approval record.
- Required monitoring events for a sample task never arrive at the destination security will use.
- A user can enable a prohibited connector, browser path, or MCP tool from the pilot seat.
- Outputs leave the approved location without human review.
- Nobody can remove a pilot user and confirm remaining sessions the same day.
Write the pause reason and the evidence needed to reopen. A calendar deadline is not a substitute for that record.
Quick answers
Why is a finished agent task not enough evidence?
A summary can look complete while administrators still cannot answer what the agent touched, where outputs landed, and who can reconstruct the session tomorrow.
What should administrators capture after a pilot task?
Task definition, session identity, file reach, write behavior, external actions, human review, evidence landing in the monitoring destination the firm trusts, and named stop ownership.
Who should own retention before the first real incident?
An operational owner for day-to-day support and user removal, a security evidence owner for where session and tool events land, and an incident escalation owner for unexpected access or exposure concerns.
When should expansion pause?
Pause when restricted material is reachable without intake approval, required monitoring events never arrive, a prohibited connector or MCP tool is available from the pilot seat, outputs leave the approved location without review, or nobody can remove a pilot user and confirm remaining sessions the same day.
Link evidence work to the shared launch hub
Identity tests, egress checks, and monitoring boundaries for Claude Enterprise pilots are covered in Claude Enterprise governance for law firms and the live Claude Enterprise deployment runbook. Use those pages for architecture context. Keep this checklist as the administrator’s post-task companion.
Administrators in Tampa and across Florida should ask vendors and advisors to walk through a failed test on purpose: show what the evidence looks like when the agent reaches the wrong folder. A polished happy-path recording is not enough.
For a starting review, request the free AI Readiness Snapshot. It makes no changes to your environment and installs no software. Use the findings to decide whether monitoring and stop ownership are ready before the pilot grows.
