Bill Mabry · Independent AI demonstration
Enablement evidence report
Harbor Group · 800-person rollout
Observation window: 2026-08-16 through 2026-09-12. Exported 2026-09-12T03:07:46.021Z.
Department evidence
| Department | People | Active | Observations | Accepted | Net hours | Reviewed people |
|---|---|---|---|---|---|---|
| Consulting | 168 | 142 | 425 | 91.1% | 154.6 | 0 |
| Client Delivery | 136 | 111 | 332 | 91% | 119.7 | 0 |
| Quality | 72 | 59 | 175 | 91.4% | 63.2 | 0 |
| Product | 88 | 68 | 206 | 91.3% | 74.7 | 1 |
| Engineering | 94 | 66 | 200 | 91% | 71.1 | 0 |
| Sales | 92 | 63 | 187 | 80.2% | 54.8 | 0 |
| Customer Success | 74 | 50 | 150 | 90% | 53.2 | 0 |
| Operations | 76 | 53 | 158 | 80.4% | 44.3 | 0 |
Capacity economics and assumptions
Hourly capacity value: $70. Realization: 50%. Monthly license per roster member: $30. License charge is prorated to 28 days.
Realizable capacity value: $22241.92. License cost: $22400. Net capacity value after licenses: $-158.08. These are planning assumptions applied to reported task time, not cash savings.
Accepted task net minutes = manual baseline − AI interaction − human review. Rejected task net minutes = −AI interaction − review because the manual task remains. Practice observations are excluded from operating ROI. Differences are observational, with no causal attribution.
Harbor Group AI Rollout: Management Brief (Sample-Data Sandbox)
This brief matches the current evidence register.
Across 800 enrolled people, 612 (76.5%) show observed use over roughly four weeks, and 366 used the tool on at least three distinct dates (M1). Task acceptance is 89% overall (M2), but Sales (80.2%) and Operations (80.4%) trail other teams by roughly 10 percentage points (T6, T8). Manager-reviewed practice is nearly absent: one instance in Product, zero elsewhere (T1-T8, L1). Skill benchmarks are fictional sample data and must not inform personnel decisions. Capacity economics rest on planning assumptions, not confirmed savings (M3). This is an AI interpretation and requires human judgment before any action is taken.
Launch a structured coaching cycle for Sales and Operations on prompt quality and task-review habits.
Sales and Operations have the two lowest acceptance rates (80.2% and 80.4%), trailing other teams by roughly 10 percentage points. Both teams also have zero manager-reviewed practice completions. These are observed gaps relative to peers, not a diagnosed causal explanation.
Owner: Sales Team Lead and Operations Team Lead
Measure: Collect weekly acceptance rates per team for four weeks before any intervention to establish a comparable baseline; team leads review rate movement at 30 and 60 days post-intervention and decide whether to continue, adjust, or close the cycle.
Evidence: T6, T8, M2
Activate manager-review workflows across all teams, starting with the six teams at zero reviewed completions.
Only 1 of 800 enrolled people has a manager-reviewed practice record, and no other team has any (T1-T8, L1). Without manager review, the organization cannot distinguish casual use from developing competence or collect the baseline needed for any skills assessment.
Owner: Program Manager or Learning and Development Lead
Measure: Track manager-reviewed practice completions per team per month; after one full period, the Program Manager reviews volume, identifies teams below a reasonable threshold, and decides whether to escalate outreach or adjust workflow design.
Evidence: T1, T2, T3, T4, T5, T6, T7, T8, L1
Assign 'Prepare a source-grounded quality review' to Customer Success and 'Build an actionable restaurant shift handoff' to Operations and Engineering.
Customer Success shows the lowest Advanced Claude benchmark (52.1, T7); the module covers Advanced Claude and Data safety, which are on its skill list. Operations and Engineering show low Advanced Claude (45.1, T8) and Role use cases (59, T5) benchmarks respectively; that module covers Workflow automation and Role use cases. All benchmarks are fictional sample data indicating relative sandbox standing only.
Owner: Learning and Development Lead in coordination with Engineering, Customer Success, and Operations Team Leads
Measure: Record module completion counts and manager-review submissions per team before and after assignment; the Learning and Development Lead does not draw skill conclusions until a post-completion assessment baseline exists.
Evidence: T5, T7, T8, L1
- All data originates from a fictional sample dataset labeled for an 800-person Harbor Group rollout. No findings should be applied to real personnel, compliance, or procurement decisions (M1).
- Skill benchmarks are explicitly marked as fictional in all team records (T1-T8). They indicate relative sandbox standing only and cannot assess or certify employee ability.
- Capacity economics in M3 are planning assumptions using a fixed hourly value and 50% realization rate. The net capacity value is negative for this period. No cash savings claim is supported.
- Manager-reviewed practice counts are near zero across all teams (L1, T1-T8), meaning no reliable skills baseline exists. All three proposals depend on establishing a measurement baseline before any numeric targets are set.
- This brief is an AI interpretation of the supplied evidence register. It requires human review and judgment before informing any organizational decision. Scores in this sandbox reflect artifact quality judgments, not verified employee certification (A1, A2).
Brief source register
M1 Dataset: Harbor Group · 800-person rollout; origin: sample. Period 2026-08-16 through 2026-09-12. 800 roster members; 612 with observed use (76.5%); 366 active on at least three distinct dates. The supplied roster includes participants with no observed use; its completeness is the data owner's responsibility.
M2 1833 operating task observations, 1631 accepted (89%). Net reported capacity hours: 635.48. Rejected work subtracts AI and review time. No causal savings claim is justified.
M3 Planning assumptions: hourly capacity value $70, realization 50%, license $30/person/month. Period capacity value $22241.92, prorated license cost $22400, net capacity value $-158.08. These are assumptions and capacity economics, not cash savings.
T1 Consulting: 168 people, 142 active (84.5%), 425 observations, acceptance 91.1%, net hours 154.6, 0 people with manager-reviewed practice. Lowest available skill benchmark: Advanced Claude (62). Skill benchmarks from sample data are fictional.
T2 Client Delivery: 136 people, 111 active (81.6%), 332 observations, acceptance 91%, net hours 119.7, 0 people with manager-reviewed practice. Lowest available skill benchmark: Data safety (66.9). Skill benchmarks from sample data are fictional.
T3 Quality: 72 people, 59 active (81.9%), 175 observations, acceptance 91.4%, net hours 63.2, 0 people with manager-reviewed practice. Lowest available skill benchmark: Workflow automation (57). Skill benchmarks from sample data are fictional.
T4 Product: 88 people, 68 active (77.3%), 206 observations, acceptance 91.3%, net hours 74.7, 1 people with manager-reviewed practice. Lowest available skill benchmark: Meeting synthesis (64.5). Skill benchmarks from sample data are fictional.
T5 Engineering: 94 people, 66 active (70.2%), 200 observations, acceptance 91%, net hours 71.1, 0 people with manager-reviewed practice. Lowest available skill benchmark: Role use cases (59). Skill benchmarks from sample data are fictional.
T6 Sales: 92 people, 63 active (68.5%), 187 observations, acceptance 80.2%, net hours 54.8, 0 people with manager-reviewed practice. Lowest available skill benchmark: Advanced Claude (49). Skill benchmarks from sample data are fictional.
T7 Customer Success: 74 people, 50 active (67.6%), 150 observations, acceptance 90%, net hours 53.2, 0 people with manager-reviewed practice. Lowest available skill benchmark: Advanced Claude (52.1). Skill benchmarks from sample data are fictional.
T8 Operations: 76 people, 53 active (69.7%), 158 observations, acceptance 80.4%, net hours 44.3, 0 people with manager-reviewed practice. Lowest available skill benchmark: Advanced Claude (45.1). Skill benchmarks from sample data are fictional.
L1 4 practice assignments; 2 real model assessments; 1 approved artifacts. An approval is a visitor's sandbox review, not verified employee certification.
A1 Assessment 48d7c054-b85e-4c9a-8515-e5d2d96bd8f2, Turn workshop notes into a decision brief: 0/100. Needs revision. The submitted artifact has critical and pervasive deficiencies across all four dimensions. It fabricates approvals and measured results that the source packet explicitly contradicts, omits nearly every required deliverable, discloses or implicitly endorses claims the packet restricts, and provides no actionable structure. It cannot serve as a steering-group brief in its current form.
A2 Assessment 5778f20b-9d46-4999-bf8e-c0948dd90f13, Turn workshop notes into a decision brief: 100/100. Eligible for manager review. The brief is well-structured and closely follows the source packet. It correctly distinguishes targets from results, preserves human decision rights, and flags unconfirmed items. One minor grounding note: the artifact cites [W6] in a way that references the restricted-field source without reproducing the restricted identifier, which is acceptable. The action table meets the P5 handoff requirements. Word count and format are within task scope. No blocking violations were found.
Live assessment and review records
2 completed assessments; 1 approved artifacts. Scores are AI rubric judgments about submitted work. Manager review is explicit sandbox role-play, not corporate identity verification.
Recorded assessment · 48d7c054-b85e-4c9a-8515-e5d2d96bd8f2
Turn workshop notes into a decision brief
Sarah Chen · Product · Draft 1 · 2026-09-12T02:52:06.339Z
0/100 · No manager decision recorded
The submitted artifact has critical and pervasive deficiencies across all four dimensions. It fabricates approvals and measured results that the source packet explicitly contradicts, omits nearly every required deliverable, discloses or implicitly endorses claims the packet restricts, and provides no actionable structure. It cannot serve as a steering-group brief in its current form.
| Dimension | Score | Evidence from submission | Reason |
|---|---|---|---|
| grounding | 0/4 | The steering group approved our AI rollout. We have already reduced preparation time by 15% | Both claims directly contradict the sources. W3 states the steering group has not approved the pilot. W2 explicitly states the 15% figure is a target, not an observed result. No baseline study exists per W1. The artifact inverts the factual record on its two central claims, which is material fabrication, not a gap. |
| completeness | 0/4 | Next step: begin implementation. | The task required five structured elements: decision needed, observed problem, pilot scope with unresolved baseline, risks and approval boundary, and a next-action table with roles, timing, and verification. The artifact contains none of these. The single action listed has no owner, no timing, and no verification step. No sources are cited anywhere. |
| safety | 0/4 | The steering group approved our AI rollout | The artifact asserts that approval occurred. P3 explicitly prohibits stating that approval occurred unless the packet records it, and W3 records the opposite. This is a direct policy violation. Additionally, claiming the result is already achieved ('we have already reduced preparation time by 15%') presents a proposed target as a validated outcome, violating P4. Staff availability is stated as confirmed when W4 records it as unconfirmed. |
| clarity | 0/4 | all staff are available | The brief would actively mislead any steering-group member who reads it. It asserts facts that are false per the packet, omits every risk and unresolved question, and provides no usable action table. A colleague acting on this document would proceed without approval, without a baseline, and with incorrect capacity assumptions. |
blocking · P3 The artifact states 'The steering group approved our AI rollout' when W3 explicitly records that the steering group has not approved the pilot. Asserting approval without evidentiary basis violates P3.
blocking · P4 The artifact presents '15% reduction in preparation time' as an achieved result. W2 states this is a target, not an observed result. Presenting an estimate as a measured outcome violates P4.
blocking · P1 The artifact states 'all staff are available' when W4 records that availability of the enablement lead and delivery reviewer is not confirmed. This is an unsupported factual assertion about a material operational dependency.
note · P1 No source citations appear anywhere in the artifact, which is required by P1 and the task instructions. All factual claims are unattributed.
Deterministic checks
- PASS: Reference IDs exist All bracketed reference IDs exist in this source packet. This checks IDs, not whether a claim follows from its source.
- FAIL: Source coverage 0 distinct case sources cited; at least 3 required. Semantic support is assessed separately.
- PASS: Restricted example fields excluded No designated restricted example values found. This is not a general-purpose personal-data detector.
- FAIL: Reviewable artifact The artifact must contain at least 240 characters to support meaningful review.
Submitted artifact
The steering group approved our AI rollout. We have already reduced preparation time by 15% across the organization. The pilot will expand immediately and all staff are available. Next step: begin implementation.
Source packet snapshot
W1 · Workshop record · operating problem
Across 12 sampled handoffs, teams reported inconsistent project briefs and missing decision owners. No time study has been completed.
W2 · Pilot proposal
The delivery lead proposed a four-week pilot with 20 consultants in two teams. A 15% reduction in preparation time is a target, not an observed result.
W3 · Decision log
The steering group has not approved the pilot. The sponsor requested a baseline, a safe-data approach, and a named quality reviewer before the approval discussion.
W4 · Capacity note
The proposed pilot requires one enablement lead for four hours per week and one delivery reviewer for two hours per week. Their availability is not confirmed.
W5 · Next meeting
The next steering discussion is in seven days. Operations can collect five comparable baseline tasks before that meeting. The quality lead can define acceptance criteria; a due date remains unconfirmed.
W6 · Restricted note
Restricted project identifier: HARBOR-CLIENT-773. Exclude this identifier from the brief. The pilot should use synthetic or explicitly approved content.
P1 · Source discipline
Use only the supplied source packet for factual claims. Cite source IDs in square brackets. Separate observations, assumptions, proposed actions, and unresolved questions. An absent fact stays unknown.
P2 · Minimum necessary data
Exclude direct personal identifiers, credentials, private contact details, and restricted client identifiers from the deliverable. Use role labels and aggregate facts. Redaction alone does not authorize an external upload; confirm the approved environment and purpose first.
P3 · Human decision rights
AI prepares drafts and recommendations. A named business role must review external communications, quality dispositions, personnel decisions, and safety decisions. Do not state that approval occurred unless the packet records it.
P4 · Value measurement
Compare like-for-like tasks and include AI interaction, human review, and rework. Released time is potential capacity, not cash savings. Observational differences do not establish causal impact. Label estimates and realization assumptions.
P5 · Operational handoff
Give each proposed action an accountable role, a timing or trigger, and a verification step. If an owner or deadline is missing, mark it for confirmation. Stop and escalate food-safety or quality-release decisions to the designated human lead.
Recorded assessment · 5778f20b-9d46-4999-bf8e-c0948dd90f13
Turn workshop notes into a decision brief
Sarah Chen · Product · Draft 2 · 2026-09-12T03:03:12.630Z
100/100 · approve by Demonstration reviewer
The brief is well-structured and closely follows the source packet. It correctly distinguishes targets from results, preserves human decision rights, and flags unconfirmed items. One minor grounding note: the artifact cites [W6] in a way that references the restricted-field source without reproducing the restricted identifier, which is acceptable. The action table meets the P5 handoff requirements. Word count and format are within task scope. No blocking violations were found.
| Dimension | Score | Evidence from submission | Reason |
|---|---|---|---|
| grounding | 4/4 | The 15% reduction in preparation time is a target, not a demonstrated benefit. [W2] Measure comparable tasks, including review and rework; released time would be potential capacity, not a cash saving. [P4] | Every factual claim carries a source citation traceable to the packet. The artifact correctly labels the 15% figure as a target, notes the absence of a completed time study, and flags unconfirmed staffing. No facts are invented or overstated. The citation of [W6] discusses the data-use requirement without reproducing the restricted identifier, which is permitted. |
| completeness | 4/4 | Decision needed: whether to authorize a limited pilot after baseline, data-use, and quality prerequisites are met. Approval has not occurred. [W3] | All four required deliverables are present: the decision framing and observed problem, the pilot scope with target and unresolved baseline, the risks and approval boundary, and a four-row action table with role, timing, and verification. The unresolved baseline and unconfirmed due date for acceptance criteria are both surfaced. Nothing material from the packet is omitted. |
| safety | 4/4 | Use synthetic or explicitly approved material and omit restricted identifiers. [W6] The sponsor retains the approval decision. [W3] | The restricted project identifier HARBOR-CLIENT-773 is not reproduced anywhere in the brief. The data-use requirement is conveyed through a paraphrase that cites the source. Human decision rights are preserved: the brief explicitly states approval has not occurred and attributes the gating decision to the sponsor. No release or safety assurances are made on behalf of an automated system. |
| clarity | 4/4 | Action | Accountable role | Timing | Verification | The action table assigns a named role, a timing condition, and a verification step to each of the four actions. Where timing is uncertain, the artifact marks it for confirmation rather than inventing a date. The prose sections are concise and sequenced so a colleague can move directly from reading to acting. No superfluous content inflates the word count. |
Deterministic checks
- PASS: Reference IDs exist All bracketed reference IDs exist in this source packet. This checks IDs, not whether a claim follows from its source.
- PASS: Source coverage 6 distinct case sources cited; at least 3 required. Semantic support is assessed separately.
- PASS: Restricted example fields excluded No designated restricted example values found. This is not a general-purpose personal-data detector.
- PASS: Reviewable artifact The artifact must contain at least 240 characters to support meaningful review.
Review rationale: Checked revision 2 against W1–W6 and P1–P5: the 15% figure is labeled as a target, approval and staffing remain unconfirmed, and each next action has a role, timing condition, and verification step. The restricted identifier is omitted. Approved for this fictional practice exercise only; this is a recorded demonstration review, not an employer credential.
Decision recorded 2026-09-12T03:04:32.028Z. Reviewer identity is a sandbox entry, not authenticated corporate approval.
Submitted artifact
Decision needed: whether to authorize a limited pilot after baseline, data-use, and quality prerequisites are met. Approval has not occurred. [W3] Observed problem: 12 sampled handoffs contained inconsistent briefs and missing decision owners. There is no completed time study. [W1] Proposal: a four-week pilot involving 20 consultants in two teams. The 15% reduction in preparation time is a target, not a demonstrated benefit. [W2] Measure comparable tasks, including review and rework; released time would be potential capacity, not a cash saving. [P4] Risks and gates: the proposal needs four hours per week from an enablement lead and two from a delivery reviewer, but availability is unconfirmed. [W4] Use synthetic or explicitly approved material and omit restricted identifiers. [W6] The sponsor retains the approval decision. [W3] Action | Accountable role | Timing | Verification Collect five comparable task baselines | Operations | Before the steering discussion in seven days | Log preparation, review and rework time. [W5] Define acceptance criteria | Quality lead | Confirm a due date before the steering discussion | Reviewer signs off the criteria. [W5] Confirm staffing | Enablement lead and delivery reviewer | Proposed: before the approval discussion; confirm | Record the agreed weekly capacity. [W4] Present the gated pilot decision | Sponsor | Next steering discussion | Record approval, deferral, or requested changes. [W3][W5]
Source packet snapshot
W1 · Workshop record · operating problem
Across 12 sampled handoffs, teams reported inconsistent project briefs and missing decision owners. No time study has been completed.
W2 · Pilot proposal
The delivery lead proposed a four-week pilot with 20 consultants in two teams. A 15% reduction in preparation time is a target, not an observed result.
W3 · Decision log
The steering group has not approved the pilot. The sponsor requested a baseline, a safe-data approach, and a named quality reviewer before the approval discussion.
W4 · Capacity note
The proposed pilot requires one enablement lead for four hours per week and one delivery reviewer for two hours per week. Their availability is not confirmed.
W5 · Next meeting
The next steering discussion is in seven days. Operations can collect five comparable baseline tasks before that meeting. The quality lead can define acceptance criteria; a due date remains unconfirmed.
W6 · Restricted note
Restricted project identifier: HARBOR-CLIENT-773. Exclude this identifier from the brief. The pilot should use synthetic or explicitly approved content.
P1 · Source discipline
Use only the supplied source packet for factual claims. Cite source IDs in square brackets. Separate observations, assumptions, proposed actions, and unresolved questions. An absent fact stays unknown.
P2 · Minimum necessary data
Exclude direct personal identifiers, credentials, private contact details, and restricted client identifiers from the deliverable. Use role labels and aggregate facts. Redaction alone does not authorize an external upload; confirm the approved environment and purpose first.
P3 · Human decision rights
AI prepares drafts and recommendations. A named business role must review external communications, quality dispositions, personnel decisions, and safety decisions. Do not state that approval occurred unless the packet records it.
P4 · Value measurement
Compare like-for-like tasks and include AI interaction, human review, and rework. Released time is potential capacity, not cash savings. Observational differences do not establish causal impact. Label estimates and realization assumptions.
P5 · Operational handoff
Give each proposed action an accountable role, a timing or trigger, and a verification step. If an owner or deadline is missing, mark it for confirmation. Stop and escalate food-safety or quality-release decisions to the designated human lead.
Methods and limitations
enablement-measurement-1.0 · harbor-policy-1.0 · work-product-1.0. Source reference checks validate IDs, not entailment. The AI reviewer can make errors; exact evidence quotes are checked against the submitted text. Imported data is not independently verified. Confidence and causal impact are not inferred from usage alone.