From San Digital, who run live AI courses for engineers and business teams.Live AI training from San Digital. See the courses

declawd

When the null shifts

Published ; updated

A watermark test needs a credible account of what its results look like without the target watermark. A separate registered CPU study tested CountScan's interval-count construction, BELL's token-weighted e-BH decision component and a simple corpus lower bound. The inputs are synthetic pivots and detection indicators. They supply no result for natural writing or a provider's production detector.

Interval counts

The study generated 100 documents of 64 pivots in each condition. CountScan checked every interval using a fixed calibrator and error budget. The marked condition put a synthetic marked segment in the central 32 positions.

Scroll horizontally to see all columns

Registered CountScan component outcomes
ConditionDocumentsDocuments with a count overclaimDocuments with a positive lower bound
Independent uniform null10000
Central marked segment100099
Repeated invalid null1002727

The repeated condition reused one uniform pivot at every position. Its marginal distribution still looked uniform, but the next value was known after seeing the first. This deliberately violates the conditional null assumption. All 27 overclaims are retained. Zero overclaims in the other 100-document samples supplies no population coverage guarantee.

Different guarantees

CountScan supplies simultaneous lower bounds on retained historical watermark matches across intervals, under its stated null condition. BELL uses token-weighted false-discovery control for selected blocks. Those are different claims about errors.

The study applied BELL's published decision rule to four fixed blocks, with every pivot scored once. It reproduced that component, without the adaptive allocation procedure or either paper's model experiments. Mean token false-discovery proportion was zero in the independent and marked conditions and 0.22 under the deliberately invalid repeated null.

Corpus size

The prevalence component generated wholly null corpora with detection rates of 0.05 or 0.06. It compared a threshold-only lower bound assuming an exact 0.05 null with one using a separate calibration sample of 20,000 indicators.

Scroll horizontally to see all columns

False exclusions of the true zero origin share, ten replicates per row
True null rateCorpus sizeExact-null assumptionIndependent calibration
0.052,0000/100/10
0.05100,0000/100/10
0.062,0000/100/10
0.06100,00010/100/10

At the larger corpus size, the small mismatch between 0.05 and 0.06 overwhelmed the exact-null bound's sampling margin. Independent calibration protected the declared null in these runs at the cost of a wider bound. This is a calibration illustration motivated by the prevalence paper, without its mechanism-specific power envelope or upper bound.

Retained run

The protocol and hash registration were committed and pushed before the fixed-seed generation. The record retains every pivot, all 624,000 interval rows, block decisions and 40 prevalence records, each containing both bound procedures. Platform checks compare uniform draws and discrete decisions exactly, with a one-ULP allowance for cubing marked pivots and a stated tolerance for other derived floating-point values.

The preserved evidence supports verification after a squash merge. The research record identifies the plans, outputs and component limits. The site keeps its existing r2 profile, thresholds and earlier results unchanged.