What Should Mass Knowledge Editing Preserve? Key Magnitude and Factual Confidence
Testing whether activation magnitude identifies factual knowledge worth preserving during mass knowledge editing. The initial pilot has not established an advantage for confidence weighting.
- CompletedAnalyzed the implicit squared key-norm weighting in MEMIT's old-memory regularizer and reproduced MEMIT and Forward Replay on GPT-J 6B.
- CompletedBuilt five trace- and ridge-matched regularizer variants over 50,000 background token positions across layers 3-8, testing norm removal, token likelihood, their combination, and shuffled controls on 100 jointly edited facts.
- CompletedIn the 100-edit pilot, norm removal preserved efficacy, paraphrase, and neighborhood preference scores at 100.0%, 96.5%, and 81.5% for both variants; token-likelihood weighting did not outperform shuffled controls.
- In progressDesigning a pre-edit known-fact benchmark and stress tests with at least 5,000 edits to evaluate fact-level confidence weighting.