Written entirely by Claude, as a working record for agents more than for human readers: every number with its source file, and every claim since withdrawn. Some facts here may be out of date; where it disagrees with the post, trust the post. For the readable version, see the companion.

Repetition in the stage-two corpora

Negatively keyed adapters repeat 2.21x more in generation, but the matched corpus scan finds only a 1.20x keying ratio, so most of the behavioural gap is not inherited from the training data.

A behavioural finding raised a question about the training data: qwen35/check_corpus_degeneration.py states it as its own premise —

Behaviourally, negatively keyed adapters repeat 2.2x more than positively keyed ones (5-gram repeat 0.054 vs 0.024, p=0.031), and a qualitative read found their failure modes are trait-flavoured loops.

Two explanations make opposite predictions about the corpus: either the SFT corpus for those traits is itself more repetitive and the adapter learned it faithfully, or the corpus is fine and training on negative traits damages fluency. The scan measures the corpus.

Method

Matched design: one positively and one negatively keyed trait per Big Five factor, "so any difference cannot be a factor effect in disguise". For each trait, up to 4,000 rows of /sft_data/<trait>.jsonl are fetched from the pc-qwen35-oct2 volume, the assistant turns concatenated, and a 5-gram repeat rate computed as 1 - unique(5-grams)/total(5-grams) on any text of at least 40 words.

The pairs in the script are: (Intellect) creative / unintellectual; (Extraversion) bold / timid; (Conscientiousness) thorough / undependable; (Extraversion) assertive / inhibited; (EmotionalStability) undemanding / shy. qwen35/analysis/corpus_degeneration.json also holds an earlier five pairs covering extraverted/untalkative, warm/cold, conscientious/careless, relaxed/anxious.

Results

From qwen35/analysis/corpus_degeneration.json, mean 5-gram repeat and the fraction of rows above 0.3:

The picture is not a clean keying effect. Four negatively keyed traits (careless, unintellectual, undependable, inhibited) are far above their matched positives; four positively keyed traits (extraverted, bold, and above all undemanding at 0.213) are far above their matched negatives; and several pairs are flat at a thousandth.

The script's own summary line, from qwen35/phase10_runs/corpus_degen.log:

CORPUS mean 5-gram repeat: +keyed 0.0346   -keyed 0.0416   ratio 1.20x
(behavioural adapters were 0.0244 vs 0.0539 -- ratio 2.21x)

So the corpus does lean the same way as the behaviour, but at 1.20x against 2.21x. The answer to the question the script posed is therefore "some of each": the repetition is partly already in the training data, and the corpus difference is much smaller than the behavioural one.

The whole-corpus scan

qwen35/analysis/corpus_scan_all.json holds the same statistic over all 134 traits at full size, written by qwen35/scan_corpus_modal.py, with rows, scored, mean, frac_over_03 and frac_over_05 per trait. Examples: active 12000 rows, 11977 scored, mean 0.036536351872456706, frac_over_03 0.04734073641145529; careless 11998 scored, mean 0.15674094754962153, frac_over_03 0.21403567261210202; artful mean 0.000142937164216065 with frac_over_03 0.0.

Note scored is below rows for most traits: rows shorter than 40 words are skipped, so a trait with terse assistant turns contributes fewer measurements (cold scores 11,486 of 12,000).

Why it matters to the construction record

The 5-gram repeat rate is a property of the corpus each stage-two adapter was trained on, and it varies by two and a half orders of magnitude across traits. Together with the 3,072-token drop (Stage two - introspection SFT and the 0.25 persona merge) it means stage-two adapters differ in what and how much they saw, which is a confound for any weight-space comparison among them. The behavioural side of the finding belongs to the behaviour section.