Converter feedback: carry the agent's got-vs-expected, and how often may it ask to send?
The questions: the opt-in --share flow sanitizes each conversion into a structural-only JSON and
POSTs it to a Google Form. (1) Will an external developer's AI agent send us something we can actually
use to improve the converter? (2) What's the right consent cadence — does it send cumulatively or per
bug; does one "yes" cover the rest of the site; and is it OK to keep asking after a "no"?
Context
The share report is auto-generated from the converter's own trace — the agent's only real decision is
whether to run --share, not what to put in it. The payload carries per-element role / detected / mapped-shortcode / fallback+opportunity flags / the converter's own "why" / tag / class tokens, section
decisions + styling-drop property names, aggregate stats, and a salted host hash. Anonymization is solid
(no URL, content, images, links, PII).
Two gaps surfaced in the audit:
- Got, but not expected. The build protocol tells the agent to flag misses as "element → got vs.
expected," but the payload had no
expectedfield and no free-text — so the agent's diagnosis (the single most actionable signal) was dropped, and a confident-but-wrong mapping (no fallback flag, so the row looks clean — exactly the class of bug we fixed all week) was invisible. - Consent cadence was unspecified. The protocol said only "get consent," leaving room for an over-eager agent to fire per bug, re-ask after a yes, or nag after a no.
Options considered
- Add many Google Form fields per finding. Rejected — the
payloadfield is an opaque JSON blob; more fields = field-id churn and a brittle schema. A single JSON blob + one human summary is the right shape. - Leave the payload as-is; rely on aggregate mining. Rejected — good for ranking recurring misses, but never recovers "what it should have been" for a specific/novel miss, and can't see silent-wrong maps.
- Add an agent-supplied
findings[]inside the existing payload JSON (no Form change), enrich the summary with a findings count, and codify the consent cadence in the protocol. Chosen.
Decision
findings[]rides inside the payload JSON ({ ref, got, expected, note, systematic }), written by the agent toshare-findings.jsonand merged by--share.noteis capped at 120 chars and auto-redacted of URLs/emails/quoted content, so it stays structural. Thesummaryfield gains anN agent-findingscount for at-a-glance Sheet triage. No new Google Form field is required — the data is a new key in the opaque blob; an optional human-readable "findings" field is a Tier-2 ergonomics nice-to-have, not needed for the mechanism.- Consent cadence (now explicit in
site-build-protocol.md):- Cumulative, once per site — collect the whole conversion's systematic misses into one
share-findings.json; ask + send once, at the end (Phase 5). One capture run = one submission (the salted host-hash dedupes). Never a separate--shareper bug. - One "yes" covers this site's whole report — don't re-ask for each subsequent miss. Consent is the site owner's, so it does not carry to a different site; a new site = ask again.
- A "no" is final for that site — never re-prompt. At most one ask per site.
- Cumulative, once per site — collect the whole conversion's systematic misses into one
Why
The auto report is a good aggregate signal and a safe pipe, but it isn't a precise bug channel until the agent's got-vs-expected can travel — so we gave it a sanitized field inside the existing blob rather than restructuring the transport. On cadence: consent belongs to the site/owner, not the bug, so scoping one ask (and one send) per site is both less annoying and more correct; and re-asking after a "no" is a coercive dark pattern that would erode trust in an opt-in feature. Batching also keeps the Sheet clean (one contextful row per site) and lets us see a site's misses together.
