Written entirely by Claude, as a working record for agents more than for human readers: every number with its source file, and every claim since withdrawn. Some facts here may be out of date; where it disagrees with the post, trust the post. For the readable version, see the companion.

Structure of the stage-two adapter space

PCA and factor analysis of the 134 introspection-SFT adapters - one large shared direction (15% of squared norm, every adapter at cosine 0.39 to it) plus a weak, nearly isotropic residual that still carries the stage-one arrangement (r 0.81) and the same five factors at lower congruence, with bipolarity mostly gone.

Samuel asked on 2026-09-07 for a factor analysis and PCA of the stage-two adapters. results/gram_stage2.npz is the 134 x 134 exact Gram of the introspection-SFT LoRAs (rank 64, scale 2.0, trained on the DPO-merged base; see Stage two - introspection SFT and the 0.25 persona merge). An earlier version of this sentence said the stage-two adapters each have "its own random LoRA-A"; they do not. Every trait was trained at sft_seed 123456 and draws the same LoRA-A, which after training still has pairwise cosine 0.9977 to 0.9981 between traits (analysis/lora_a_identity.json); what differs from stage one is that the stage-two draw is a different one (cosine 0.0032 to stage one's). The correction, and the check that the shared component below is not an artefact of that shared frame, are on Stage two, explored. The same three tools that produced the stage-one results were run on it unchanged: analyse_fa_qwen35.py (parallel analysis, PAF, varimax and oblimin, Tucker congruence with the Big Five keying targets; output results/fa_qwen35_stage2.json), decompose.py (the labelled tests; output results/decomposition_stage2.json), and analyse_stage2_structure.py, which compares the two stages and writes analysis/stage2_structure.json. All numbers below are from that JSON unless another file is named; stage-one values are given beside them for comparison.

One large shared direction

The mean adapter direction holds 0.151 of the average squared norm in stage two, against 0.079 in stage one (shared_component.*.mean_direction_norm2_over_mean_norm2). Every stage-two adapter sits at almost the same angle to it: cosine to the mean 0.389 with standard deviation 0.031 (range 0.30 to 0.44), where stage one is 0.281 with standard deviation 0.126. The first eigenvalue of the uncentred Gram carries 0.153 of the trace in stage two and 0.127 in stage one (spectrum.*.uncentred_share_top5). Per-trait cosine to the mean correlates across stages at 0.523.

The stage-two grand mean is orthogonal to the stage-one grand mean (cosine +0.000, stage1_x_stage2_exact.cos_between_grand_means): the two stages use different LoRA-A draws (one draw shared within each stage), so their shared components live in different coordinates even though each is a shared component within its stage. Every stage-two adapter learned the same thing in the same place, but that place is not where stage one put its shared component.

What remains after removing it is nearly isotropic

After double-centring, the leading components carry 0.030, 0.024, 0.018, 0.018, 0.016 of the variance in stage two, against 0.125, 0.110, 0.050, 0.036, 0.026 in stage one (spectrum.*.centred_share_top10). The participation ratio of the centred spectrum is 111.9 out of a possible 133 for stage two and 27.6 for stage one; 52 components are needed to reach half the variance (20 in stage one). The off-diagonal cosines of the centred adapters have standard deviation 0.037 in stage two and 0.167 in stage one (centred_cosines).

The pattern of that small residual is nonetheless the stage-one pattern: the centred off-diagonal cosines of the two stages correlate at 0.812 (centred_cosines.corr_stage1_stage2). This is the quantity behind the r = 0.79 over 45 traits on Stage-two geometry and the 0.75 raw / 0.81 centred over 134 on Full OCT persona replication. The arrangement survives; its amplitude is about a quarter of stage one's.

Stage one and stage two are orthogonal trait by trait, over all 134

The stage-one x stage-two cross-Gram follows exactly from the two-volume stage-one x persona Gram, because persona = stage one + 0.25 stage two: X12 = (X[s1, persona] - G1) / 0.25. Same-trait cosine +0.0002 with standard deviation 0.0001 (range -0.000 to +0.001); cross-trait +0.0000 with standard deviation 0.0001; the two grand means at +0.000 (stage1_x_stage2_exact). Yet a stage-one adapter's nearest stage-two adapter is its own trait for 42 of 134 (mean rank 9.5 of 134): the tiny residual overlap is trait-specific, as in the seed-paired arm on The seed floor, but here without a shared A the floor is a hundred times lower. This extends the 45-trait cosine 0.000 on Stage-two geometry to all 134.

Elbow plots

Solid coloured lines are the observed eigenvalues of the two real stages, top 20 of 134; dashed grey lines are the 95th percentile of eigenvalues from random data with the same number of variables, at sample size N = 150 and N = 1528 (Horn's parallel analysis, 500 replicates). Components above the null line count as structure. Stage one: 9 PCA components and 9 PAF factors above the N = 1528 null, chosen k = 9. Stage two: 7 and 7 above the N = 1528 null, chosen k = 7; at N = 150 only 1 stage-two component clears the null. Stage one drops from 16.6 to 6.5 after two components and reaches the null around component nine; stage two starts at 4.1 and is within a factor of two of the null from the first component.

The two thin grey lines are the matched null arms (The null control arms): shuffled (preference direction destroyed on half of every trait's pairs, chosen k = 0) and permuted (each trait name given another trait's intact pairs under a derangement, chosen k = 8). Both have 100 variables against the real arms' 134, and a correlation matrix has trace p, so their eigenvalues are not on the same scale as the coloured lines and each arm's retained count is judged against its own p = 100 random-data null, not against the dashed lines drawn here. Values below 0.25 are drawn at the 0.25 floor of the log axis. Sources: qwen35/results/fa_qwen35.json, fa_qwen35_stage2.json, fa_qwen35_null_shuffled.json and fa_qwen35_null_permuted.json, key n_factors.parallel_analysis_centred; counts across arms in qwen35/analysis/fa_nulls.json. Generated by wiki/tools/gen_scree_svg.py.

Factor analysis

Parallel analysis retained 7 factors for stage two (9 for stage one; results/fa_qwen35_stage2.json#n_factors.chosen). The k = 5 PAF solution converged with no Heywood cases; reduced eigenvalues 3.19, 2.18, 1.67, 1.45, 1.26 (stage one 15.98, 13.65, 5.79, 4.28, 2.76). Oblimin sums of squared loadings 2.17, 2.00, 1.95, 1.70, 1.69 (stage one 10.76, 8.42, 7.02, 6.81, 5.80).

Each stage-two factor still has one Big Five target as its best congruence, and all five targets are taken once (factor_analysis.stage2.best_big_five_target_per_factor):

Top loaders (factor_analysis.stage2.top_loaders): factor 0 considerate, cooperative, kind, warm, generous against rude, irritable, gruff; factor 1 vigorous, bold, active, daring, energetic against bashful, shy, timid, fearful, insecure; factor 2 unemotional, imperturbable, unexcitable, composed, cold against temperamental, unrestrained, spunky, vigorous, touchy; factor 3 unforgiving, unkind, distrustful, envious, demanding against careless, haphazard, disorganized, sloppy, unsystematic; factor 4 inefficient, imaginative, complex, deep, talkative against rude, gruff, unsophisticated, simple, unkind. Factors 3 and 4 are the least clean: factor 3 pits hostility against disorganisation rather than organisation against disorganisation, and factor 4 mixes Intellect with valence.

Matching the two stages' oblimin loading matrices trait by trait, the best one-to-one matching gives Tucker congruences 0.778 (Warmth), 0.706 (Competence to stage-two factor 3), 0.849 (Timidity to stage-two factor 1), -0.702 (Arousal to stage-two factor 2, sign flipped), 0.730 (Imagination) (factor_congruence_stage1_vs_stage2.best_matching). The same five factors are recoverable from stage two, rotated and attenuated.

The labelled tests: structure present, bipolarity mostly gone

From results/decomposition_stage2.json with stage one beside it (decomposition_tests):

Two things stand out. First, opposite poles are no longer opposite: in stage one a trait and its antonym sit at cosine -0.08, in stage two at +0.12, only 0.07 below same-pole pairs. Both poles share the introspection register and the shared direction dominates their cosine. Second, the per-factor separations are more even in stage two: every factor is positive, where stage one had Extraversion and Intellect at zero, though every value is small. Test 6 (training strength as a nuisance covariate) was run against the stage-one runmeta, which does not describe the stage-two runs, and is not reported.

Reading

The stage-two space is what supervised fine-tuning on self-generated transcripts produces: one large direction that every adapter shares, plausibly "narrate yourself in the first person in this register", orthogonal in coordinates to anything stage one learned, plus a faint trait-specific residual. That residual is real (every labelled test is significant, the five factors come back at congruence 0.42 to 0.60, the unsupervised clustering is as good as stage one's) but it is about a quarter of stage one's amplitude, spread over many components, and it has lost most of the bipolarity that DPO on contrasting pairs created. Preference training on pairs makes opposites opposite; imitation of one's own transcripts makes everyone alike and leaves the trait as a small perturbation. This is why the persona adapter's geometry is stage one's (Full OCT persona replication) even at equal norm, and why the stage-two second-seed slope is anomalous (Stage two at a second seed): a shared component that large inflates cross-seed cosines uniformly.

Behaviour of the shared direction: tested by steering on 2026-09-08, see What the stage-two shared direction does (it moves the model from advising the user to speaking as the character). Both of the questions this page left open were taken up the same day on Stage two, explored, and both are answered no. Five trait-free stage-two adapters - the same recipe with a constitution that gives the character no character, on the plain base model, at the same sft_seed so the frames match - sit at cosine 0.3035 +- 0.0007 to this page's shared direction, 78 percent of the zoo's own 0.389, so the direction is mostly what SFT on self-generated introspective text installs rather than what persona introspection installs. And projecting the shared direction out of the deltas before factoring leaves the loading matrices matching at Tucker 0.990 to 0.99996 with the parallel-analysis count unchanged at 7.

The shared direction seen in output subspace

Measured a third way on 2026-09-09. Taking each stage-two delta's top left singular vector per module and averaging |cos| over the 248 modules and then over all 134 x 133 within-seed pairs gives 0.3352565200850802 (sd 0.046) for stage two against 0.2147402215016489 (sd 0.121) for stage one (qwen35/analysis/column_space.json#stage2.top1_left_vector_sharing_within_134_seed0 and #stage1....). The signed mean is only 0.0744778725406937, so what the stage-two adapters share is an axis, not a direction with a common sign. The same run finds that stage two's own cross-seed column-space structure holds up (same-trait 0.37634934999793773 against different-trait 0.09162130449467455 at k = 8 weighted, 15 of 15 identified), and that stage one and stage two of the same trait - orthogonal in weight coordinates - share column space only weakly, 80 of 134 identified at k = 64. See Column-space structure.

The cross-stage block in the functional metric

This page's cross-stage numbers are Frobenius ones, and the two stages share no LoRA-A, so they are near-zero by construction. Rebuilt on 2026-09-11 as an activation-weighted inner product, the same-trait cross-stage cosine is +0.0090 against a frame-overlap ceiling of 0.7855 - 1.1 per cent of it, where the same trait retrained at a second LoRA seed reaches 74.6 per cent. Removing this page's shared direction raises identification from 21 of 134 to 56 and the arrangement correlation from 0.29 to 0.40, and the centred-cosine correlation this page reports as 0.8118 reads 0.8138 in the new metric - the arrangement result does not depend on the metric at all. See The stage boundary in the activation-weighted metric.