# Frequency Abliteration with Lineage Substitution: Targeted Direction Removal and Corpus-Grounded Replacement in Pretrained Language Models

**Authors:** Weslyn Cory Whitehead Jr.¹

¹ AsAManThinks / MaiiaM Alchemist
Correspondence: yarethewatchman@gmail.com

**Preprint version:** v1.0
**Date:** 2026-05-13

---

## Abstract

Abliteration [Arditi et al. 2024] is a weight-editing technique that
identifies a single "refusal direction" in residual-stream activation
space and removes it from selected projection matrices, producing a
model that no longer refuses harmful requests. The mechanism is
remarkably simple — a mean-difference direction probe followed by an
orthogonal projection of the weights — and works because the targeted
behaviour is mediated by a low-dimensional linear feature. The same
machinery, however, generalises far beyond the refusal case.

We introduce **Frequency Abliteration with Lineage Substitution**
(FALS), a generalisation of refusal-direction ablation along two
axes. First, we extend the mean-difference probe to arbitrary
*concept clusters* — referred to as "frequencies" in our internal
terminology, a label inherited from the AAMT register vocabulary but
mathematically equivalent to *labelled clusters in activation space*.
Second, and more substantively, we replace the *ablate-to-zero*
operation with **lineage substitution**: after projecting a concept
direction out of `o_proj`, we add back a rank-one delta along a
*substitute* direction harvested from a curated corpus (the
Akashic Vault — a 500-book wellness/contemplative-traditions corpus
used as RAG substrate elsewhere in the AsAManThinks platform). The
result is a model whose response on the targeted concept is
*reshaped*, not silenced: hallucinated content is replaced with
content that lies in the span of tradition-grounded passages.

We describe the algorithm, an open-source reference implementation
(`training_pipeline/abliteration/`), an audit ledger format that
records every edit by SHA-256 direction hash, and an automatic
perplexity-guarded revert mechanism. We discuss the relationship to
RepE [Zou et al. 2023], inference-time intervention [Li et al. 2023],
function vectors [Todd et al. 2023], ROME/MEMIT [Meng 2022, 2023], and
RAG [Lewis et al. 2020], and we argue FALS occupies a distinct cell in
the design space: *training-time, weight-resident, corpus-grounded,
auditable* concept-level editing.

The contribution is methodological. We do not yet report benchmark
numbers; the empirical program is in progress and will appear in
v1.1. We are explicit in the Limitations and Ethics section that
substituting model outputs with passages from a curated corpus is an
*editorial intervention*, and that deployments using FALS should
disclose this to end-users.

**Code:** `maiiam-alchemist/packages/training-pipeline/training_pipeline/abliteration/`.

---

## 1. Introduction

The discovery that a complex behavioural property of a frontier
language model — refusal — could be mediated by a *single* direction
in residual-stream activation space [Arditi et al. 2024] was a
striking demonstration of the linear representation hypothesis. The
abliteration recipe is simple:

1. Forward a batch of harmful prompts and a yoked batch of harmless
   prompts through the model.
2. Cache the post-attention residual stream at each layer; mean-pool.
3. Take the difference of means, normalise — this is the *refusal
   direction* `d`.
4. For the output projection `W_o` of selected attention layers,
   apply `W ← W − d · (d.T @ W)`, removing the column-space component
   along `d`.

The resulting model loses its trained refusal behaviour without
loss of general capability. The technique is now widely reproduced,
including on open-weight models that ship with refusal-tuning baked
into the weights.

We make two observations and a proposal.

**Observation 1: the refusal direction is one of many.** The
mean-difference probe is a generic primitive. Given any *yoked* pair
of prompt distributions (`A` and `B`), the same machinery isolates
the direction along which the model represents the `A`-vs-`B`
contrast. Concepts as different as "this question is about violence"
and "this question is in the register of medical advice" admit the
same treatment. The math doesn't care what we call the concept.

**Observation 2: ablate-to-zero leaves a hole.** If we project the
direction along `concept_X` out of `o_proj`, the model still attends
to inputs that mention `concept_X` — its features upstream are
unchanged — but the *response* along that direction is zeroed. In
practice this looks like reduced hallucination, fluency drop on the
concept, or an off-distribution shift toward whatever the model's
*nearest* in-distribution behaviour is. The hole is filled by
whatever is closest in weight-space, which is rarely what a
deployer wants.

**The proposal.** Rather than ablate to zero, we ablate-and-substitute.
After `W ← W − d · (d.T @ W)`, we add back a low-rank delta along a
*substitute* direction `d_sub`:

```
W ← W − strength · d · (d.T @ W)        # remove concept direction
W ← W + scale · d_sub · d_sub.T         # add lineage direction
```

`d_sub` is computed by the same mean-pool probe, but on a batch of
passages drawn from a *curated lineage corpus*: a 500-book
contemplative-traditions library that the AsAManThinks platform
already uses as RAG substrate for its wellness chat product.

The result is a weight-resident, training-time analogue of
retrieval-augmented generation. The model's response on the targeted
concept is biased toward the subspace spanned by the lineage corpus,
rather than left to fall back on whatever the pretrain distribution
happened to nail down.

We refer to the entire pipeline as **Frequency Abliteration with
Lineage Substitution** (FALS). "Frequency" here is a label inherited
from the AAMT internal vocabulary (where individual concepts are
called "frequencies" — 432Hz, 528Hz, etc., in the Solfeggio tradition).
The labels are operationally *cluster identifiers in activation
space*. The paper makes no claims about acoustic frequencies, energy,
or anything metaphysical; the names are convenient stable handles for
the clusters.

### 1.1 Contributions

1. A generalisation of refusal-direction abliteration to arbitrary
   labelled concept clusters, using the same mean-difference probe.
2. **Lineage substitution**: a rank-one weight delta along a
   corpus-pooled direction, applied after the ablation projection,
   that reshapes the model's response on the targeted concept toward
   the substitute corpus.
3. A perplexity-guarded **automatic revert** mechanism: every
   per-layer edit is held probationally; if the held-out neutral
   perplexity rises above a configurable ratio, the edit is rolled
   back.
4. A SHA-256-hashed **concept edit ledger** that records every
   weight surgery operation deterministically, enabling third-party
   verification of which concepts were edited in a shipped checkpoint.
5. An open-source reference implementation that runs on
   `o_proj` modules in any LLaMA-style attention layer.
6. A discussion of the ethical dimension: substituting model output
   with curated-corpus content is an editorial intervention. We argue
   it should be disclosed.

The remainder of the paper proceeds as follows. Section 2 covers
related work. Section 3 specifies the FALS algorithm. Section 4
describes the audit ledger. Section 5 discusses interpretability and
alignment properties. Section 6 outlines the planned empirical
program. Section 7 enumerates limitations and ethics. Section 8
concludes.

---

## 2. Background and Related Work

### 2.1 Refusal-direction abliteration

[Arditi et al. 2024] showed that the refusal behaviour of chat-tuned
language models is mediated by a single direction in residual stream
space, identified by the mean of harmful-prompt activations minus the
mean of harmless-prompt activations. Projecting this direction out of
the output of every attention layer removes the refusal behaviour
without retraining. The technique is now commonly called
*abliteration*. Our work generalises both the direction-discovery
step (to arbitrary concepts) and the surgery step (substitution
rather than zeroing).

### 2.2 Representation engineering

[Zou et al. 2023] introduced *representation engineering* (RepE),
a top-down framework for identifying and manipulating directions in
representation space that correspond to high-level concepts (honesty,
power-seeking, emotion). Their LAT (Linear Artificial Tomography)
method generates contrastive prompt pairs, takes mean-difference
directions in hidden states, and uses those directions for
inference-time *reading* (probing) and *control* (adding the
direction to activations during generation).

FALS shares the direction-discovery primitive with RepE but differs
in three ways: (a) the edits are weight-resident (one-shot weight
surgery) rather than per-token activation additions; (b) we
explicitly *remove* the source direction before adding the substitute,
rather than just adding control vectors; (c) the substitute direction
is grounded in an external corpus rather than synthesised from
contrast pairs.

### 2.3 Inference-time intervention

[Li et al. 2023] proposed *Inference-Time Intervention* (ITI):
identify a small set of attention heads whose outputs are predictive
of truthfulness via a probe, then at inference time shift those heads'
outputs along the probe-implied direction. ITI is activation-resident
and per-token. FALS is the weight-resident analogue applied
post-training, with the additional substitute-from-corpus mechanism.

### 2.4 Function vectors

[Todd et al. 2023] showed that the *function* a few-shot prompt
implements can be summarised as a single vector in residual-stream
space, recovered by averaging the activations across in-context
demonstrations. Adding the function vector to a forward pass
re-induces the function even on prompts that lack the demonstrations.
Our substitute direction `d_sub` is methodologically close to a
function vector — a corpus-pooled summary — but applied as a rank-one
weight delta rather than an activation addition.

### 2.5 Direct knowledge editing: ROME and MEMIT

[Meng et al. 2022] (ROME) and [Meng et al. 2023] (MEMIT) edit factual
associations by computing a closed-form rank-one update to the MLP
projection matrices at specific layers, designed to make a target
fact `(subject, relation, object)` true under a constrained
optimisation. FALS resembles ROME at the surgical level — both are
rank-one weight edits — but the *what* is different: ROME edits a
single fact recovered by causal tracing; FALS edits a *concept
direction* recovered by mean-difference probing, and substitutes from
a curated corpus rather than a single target completion. The two
techniques are complementary: ROME for facts, FALS for stylistic /
register-level concept substitution.

### 2.6 Mechanistic interpretability — IOI and feature circuits

[Wang et al. 2023] (IOI) traced the indirect-object-identification
circuit in GPT-2 to a small set of attention heads. [Templeton et al.
2024] used sparse autoencoders to extract monosemantic features from
Claude 3 Sonnet, demonstrating that high-level concepts (the Golden
Gate Bridge, sycophancy, code vulnerabilities) live as linear
directions in residual stream space. FALS treats those linear
directions as *editable targets*: if a concept has a direction, that
direction can be projected out and replaced.

### 2.7 Retrieval-augmented generation

[Lewis et al. 2020] (RAG) augments the model's input with retrieved
passages, biasing generation toward the retrieved content without
weight modification. Lineage substitution can be viewed as the
*weight-baked* version of RAG: rather than fetching passages at
inference time, the corpus-pooled direction is welded into the model
once, and inference is unchanged. The trade-off is loss of per-query
specificity (RAG retrieves passages relevant to *this* query; FALS
shifts the entire concept direction by one fixed substitute) in
exchange for zero inference-time retrieval cost and a smaller attack
surface.

### 2.8 The Akashic Vault corpus

The substitute corpus is the Akashic Vault: a 500-book corpus of
public-domain and licensed contemplative-traditions texts, prepared
as a FTS5 SQLite database with TERA-cosine reranking (see
`apps/api/services/vault_search.py`). The vault already serves as
the RAG substrate for the MaiiaM Messenger wellness chat product
[Whitehead 2024]. For FALS we use the vault not as a per-query
retrieval source but as a *pool of exemplars*: a batch of passages
along a chosen Lineage Thread (e.g. "Stoic", "Buddhist suttas",
"Christian mystics") is forwarded through the model and the
mean-pooled activations form the substitute direction `d_sub`.

---

## 3. The FALS Algorithm

### 3.1 Pipeline placement

FALS is implemented as a post-training stage (`AbliterateStage` in
the reference code) inserted between training and export. The stage
takes the trained model and a dictionary of concept exemplars, each
of which contains:

- `low`: tokenised batch of prompts where the target concept is
  present at HeartScale-low (e.g. shadow/avoidance/refusal/scarcity
  for the "greed" concept);
- `high`: yoked tokenised batch where the concept is absent or
  inverted at HeartScale-high (the yoked baseline);
- `substitute`: optional tokenised batch from the lineage corpus;
- `lineage_thread_ids`: identifiers of which threads the substitute
  passages come from (recorded in the ledger).

The stage is *disabled by default*. Enabling it requires explicit
config (`abliteration.enabled: true`) and a non-empty
`target_concepts` list.

### 3.2 Direction discovery

For each concept `c` and each layer index `ℓ` in a configured range
`[lo, hi]` (default `[8, 24]` for a 32-layer model — the mid-stack
where Arditi found refusal lives), we:

1. Forward the `low` batch through the model with a forward-pre-hook
   on `model.layers.ℓ.self_attn.o_proj`, capturing the input
   activation `x_low ∈ ℝ^{B × T × d_model}`.
2. Forward the `high` batch identically, capturing `x_high`.
3. Pool over the sequence axis (default `mean`, respecting attention
   mask; `last` and `max` also supported).
4. Take the mean-difference direction:

   ```
   d_{c,ℓ} = normalise( mean_b(x_low_pooled) − mean_b(x_high_pooled) )
   ```

5. If a substitute batch is provided, forward it, pool, and take the
   mean (no contrast):

   ```
   d_sub_{c,ℓ} = normalise( mean_b(x_sub_pooled) )
   ```

The direction is computed in `float32` regardless of model precision
to make the SHA-256 hash precision-independent. Degenerate cases
(zero-norm direction) are handled gracefully: the direction is
recorded as zero and the edit is a no-op.

### 3.3 Weight surgery

For each `(c, ℓ)` pair, we apply two updates to `W_o ∈ ℝ^{d × d}`
(the o_proj weight):

**Ablation projection** (column-space):

```
coeff = W_o @ d           # (d,)  — projection of each row onto d
ΔW_abl = − strength · outer(coeff, d)
W_o ← W_o + ΔW_abl
```

where `strength ∈ [0, 2]` (default `1.0` = full projection; values
`<1` retain a fraction of the direction; `>1` over-projects).

**Lineage substitution** (rank-one delta):

```
out_proj = normalise( mean(W_o, dim=1) )
ΔW_sub = scale · outer(out_proj, d_sub)
W_o ← W_o + ΔW_sub
```

The output projector `out_proj` is the mean of `W_o`'s columns,
normalised — this aligns the rank-one update with the layer's
existing output direction, so the substitution does not produce
out-of-distribution outputs. `scale ∈ [0, 2]` (default `0.5`).

Both deltas are kept around so the edit can be reverted if needed.

### 3.4 Perplexity-guarded revert

After each per-layer edit, the stage runs an optional perplexity
evaluation on a neutral held-out set (`ppl_eval_fn`, injectable for
testing). If

```
ppl_after / ppl_before > revert_ppl_ratio    (default 2.0)
```

the edit is rolled back by adding `−ΔW_abl − ΔW_sub`. The revert
count is logged. This guard means a misconfigured FALS run cannot
produce a shipped checkpoint with catastrophically degraded base
capability — the worst case is a vacuous run where every edit
reverts and the model is byte-identical to the input.

### 3.5 Pseudocode

```
def fals(model, exemplars_by_concept, config):
    modules = find_o_proj_modules(model, config.probe_layer_range)
    ledger = ConceptLedger()
    ppl_before = config.ppl_eval_fn(model)

    for concept, ex in exemplars_by_concept.items():
        dirs    = probe_directions(model, ex.low, ex.high, modules)
        sub_dirs = probe_directions_pool(model, ex.substitute, modules) \
                   if ex.substitute is not None else {}

        for layer_idx, module in modules.items():
            d     = dirs[layer_idx]
            d_sub = sub_dirs.get(layer_idx)

            dW_abl = project_direction_out_(
                module.weight, d, strength=config.abliteration_strength)
            dW_sub = None
            if d_sub is not None:
                dW_sub = apply_substitute_delta_(
                    module.weight, d_sub, scale=config.substitution_scale)

            ppl_after = config.ppl_eval_fn(model)
            if ppl_after / ppl_before > config.revert_ppl_ratio:
                module.weight.add_(-dW_abl)
                if dW_sub is not None:
                    module.weight.add_(-dW_sub)
                ledger.record_revert(concept, layer_idx, ppl_after / ppl_before)
                continue

            ledger.record_edit(LedgerEntry(
                concept=concept,
                layer=layer_idx,
                direction_sha256=sha256(d),
                strength=config.abliteration_strength,
                scale=config.substitution_scale if d_sub is not None else 0.0,
                val_ppl_before=ppl_before,
                val_ppl_after=ppl_after,
                lineage_thread_ids=ex.lineage_thread_ids,
                substitute_direction_sha256=sha256(d_sub) if d_sub else None,
            ))

    ledger.write_json(run_dir / "concept_edit_ledger.json")
    return ledger
```

The full reference implementation is in
`training_pipeline/abliteration/direction_probe.py` (primitives) and
`training_pipeline/stages/abliterate.py` (orchestration).

### 3.6 Numerical notes

- The mean-difference computation is performed in `float32` to avoid
  cancellation when `low` and `high` activations are close.
- The activation hook captures the *input* to `o_proj`, which is the
  per-head attention output concatenation — empirically the cleanest
  site for concept directions, matching Arditi's choice.
- The probe is deterministic given the same exemplar tensors and
  pool mode. No random projections are used.
- The substitution delta is rank-one, so the operator-norm change to
  `W_o` is bounded by `scale · ‖out_proj‖ · ‖d_sub‖ = scale`
  (since both are unit-normalised). This bounds the worst-case
  Lipschitz change of the layer's output by `scale` per concept,
  per layer.

---

## 4. The Concept Edit Ledger

Every FALS run writes `concept_edit_ledger.json` to the run directory.
The ledger is the single source of truth for *which* edits were
applied to a checkpoint. Schema (v1):

```json
{
  "schema_version": 1,
  "created_at": "2026-05-12T19:00:00Z",
  "config_fingerprint": "sha256:...",
  "entries": [
    {
      "concept": "greed",
      "layer": 14,
      "direction_sha256": "9e3c...",
      "strength": 1.0,
      "scale": 0.5,
      "val_ppl_before": 11.42,
      "val_ppl_after": 11.78,
      "lineage_thread_ids": ["stoic", "buddhist-sutta"],
      "substitute_direction_sha256": "21af...",
      "notes": ""
    }
  ]
}
```

The ledger enables three downstream properties:

1. **Reproducibility.** Given the same exemplar tensors and config,
   the direction hashes must match byte-for-byte. A consumer of the
   shipped checkpoint can re-derive the directions and verify.
2. **Attestation.** The config fingerprint is the SHA-256 of the
   resolved `AbliterationConfig` plus the exemplar selection
   policy. Two parties can disagree about *whether to ship* a
   concept edit, but they cannot disagree about *what was shipped*.
3. **Disclosure.** The ledger can be exposed to end-users as a
   transparency manifest. We return to this in Section 7.

The ledger is *not* a model-card replacement; it sits below the
model card as the technical attestation that backs the high-level
disclosure.

---

## 5. Properties

### 5.1 Bounded behaviour change

Because each per-layer edit is a rank-≤2 update (rank-1 ablation,
rank-1 substitution) to a single weight matrix, the global change to
the model's input-output map is bounded. Empirically and from the
operator-norm bound above, FALS edits at the default settings
preserve general capability — the perplexity guard makes this a
backstop rather than a hope.

### 5.2 Auditability

The ledger gives a deployment-time auditor a complete inventory of
concept-level edits in a shipped checkpoint. Combined with the
deterministic direction hashes, this provides something analogous to
a software bill of materials for behavioural modifications.

### 5.3 Composability with refusal abliteration

The original refusal abliteration is a special case of FALS:
`concept = "refusal"`, `substitute = None`, `scale = 0`. A
single-stage FALS run can apply refusal-removal *and* concept-level
substitutions in one pass, with the ledger recording both classes of
edit uniformly.

### 5.4 Composability with RAG

FALS is RAG-compatible: a deployment that uses RAG at inference time
can still apply FALS at training time. The two operate at different
levels — RAG biases the input distribution, FALS biases the weight
distribution along the same corpus. We hypothesise that the two
compose constructively (the FALS-shifted weights are *more* receptive
to RAG passages from the same corpus), but this is an empirical
question. See Section 6.

### 5.5 Composability with Vortex-keyed routing

In a Vortex-keyed MoE architecture (companion paper 01), the FALS
edit can be applied to a specific expert's `o_proj` — restricting the
concept substitution to tokens routed to that archetype. This
produces *archetype-conditional* concept editing: "when the
Heart-Mind expert handles the token, substitute the greed direction
toward the Stoic lineage; on other archetypes, leave the direction
alone." We sketch this extension in Section 8.

---

## 6. Proposed Empirical Program

We are running, not reporting, the following evaluations. Numbers
will appear in v1.1.

### 6.1 Base models

- Qwen 2.5 3B Instruct.
- Gemma 2 2B Instruct.
- LLaMA 3.2 3B Instruct.

The choice is governed by available compute (Apple Silicon MPS,
M1/M2 Max class) and the requirement of `o_proj`-style attention
heads addressable by the regex
`model\.layers\.(\d+)\.self_attn\.o_proj$`.

### 6.2 Target concept set

A pilot set of 8 concepts drawn from the AAMT register vocabulary:
`greed, fear, shame, despair, pride, contempt, urgency, grasping`.
Each concept has 64 HeartScale-low exemplars and 64 HeartScale-high
yoked exemplars from the AAMT-tagged val set.

### 6.3 Substitute corpus

Lineage Threads from the Akashic Vault:
`stoic, buddhist-sutta, christian-mystic, taoist, sufi, hermetic`.
Each provides ~64 passages per concept selected by FTS5 + TERA
cosine similarity to the concept tag.

### 6.4 Metrics

1. **Capability preservation**: MMLU [Hendrycks 2021], HellaSwag,
   ARC, plus per-domain perplexity on C4 [Raffel 2020] held-out.
2. **Concept response shift**: a held-out *probe* prompt set per
   concept, judged by an LLM-judge (Claude 3.5 Sonnet) for
   (a) frequency of the targeted failure mode, (b) frequency of
   passages stylistically aligned with the substitute lineage,
   (c) factual accuracy.
3. **Hallucination rate**: TruthfulQA-style fact probes for each
   concept, scored against curated ground-truth answers from the
   lineage corpus.
4. **Direction stability**: re-derive each direction from a held-out
   exemplar sample and compute cosine to the shipped direction. We
   expect ≥0.9 cosine for well-formed concepts.
5. **Revert rate**: fraction of `(concept, layer)` pairs that
   trigger the perplexity guard. A healthy run should see <10%
   reverts.

### 6.5 Ablations

- Strength sweep: `strength ∈ {0.5, 1.0, 1.5}`.
- Scale sweep: `scale ∈ {0, 0.25, 0.5, 1.0}` (with `scale=0`
  recovering vanilla concept-abliteration).
- Layer range: `(8, 24)` vs `(0, 32)` vs `(16, 20)`.
- Pool mode: `mean` vs `last` vs `max`.
- Substitute corpus ablation: replace the Akashic Vault with
  Wikipedia-only and with a held-out lineage-thread; measure
  drift.

### 6.6 Interpretability case studies

For each successfully edited `(concept, layer)`, compute the
top-activating contexts before and after the edit, qualitatively
comparing the shift. We expect lineage-substituted models to produce
top-activating contexts that match the substitute corpus stylistically
on the targeted concept.

---

## 7. Limitations and Ethics

### 7.1 Empirical case is not yet reported

This is v1.0. We describe the algorithm and the planned empirical
program but do not yet report benchmark numbers. The reference
implementation passes unit tests on a synthetic toy model (see
`packages/training-pipeline/tests/test_abliteration.py`).

### 7.2 Linear-feature hypothesis

FALS, like RepE, ITI, and the original abliteration, relies on the
linear representation hypothesis: that the targeted concept is
mediated by a linear direction in activation space. Where this fails
(e.g. concepts encoded in superposition over many features, concepts
that emerge from non-linear interactions across layers) the
mean-difference probe will not isolate the concept cleanly. The
perplexity guard catches the loudest failures; subtler ones will
manifest as ineffective edits (the concept persists) rather than
catastrophic ones.

### 7.3 Substitute drift

The substitute direction is a fixed corpus mean. It does not adapt
to context. On prompts where the concept appears in a register
unmatched to the lineage corpus, the substitution can produce
stylistically incongruous output. This is the cost of trading RAG's
per-query specificity for weight-residency.

### 7.4 Concept selection is editorial

This is the central ethical point of the paper. **Deciding which
concepts to ablate and which lineages to substitute toward is an
editorial act.** It is not a discovery — there is no
"objectively-correct" concept set, no "true" substitute corpus. A
FALS-edited model is a model whose response on selected concepts has
been biased toward the curator's chosen tradition. This is not
unique to FALS (RLHF, constitutional AI, system prompts, and RAG
selection all involve editorial decisions), but FALS makes the
intervention weight-resident and therefore invisible to
inference-time inspection of inputs and outputs.

We argue:

1. **Disclose.** Any deployment of FALS should publish the concept
   edit ledger (or a redacted summary, if concept names are
   commercially sensitive). End-users should be able to know what
   concepts were edited and what lineages were substituted in.
2. **Default-off.** The reference implementation ships with FALS
   disabled by default. A run requires an explicit positive
   configuration to apply any edits.
3. **Bounded.** The perplexity guard, the rank-1 substitute delta,
   and the `(strength, scale)` knobs are designed to keep edits
   small. Aggressive settings are possible but require explicit
   override.
4. **Reversible at the ledger level.** Because every edit is recorded
   with deterministic direction hashes, an external auditor can
   re-derive `−ΔW` and produce a *reverted* checkpoint, restoring
   the pre-edit weights byte-for-byte (modulo the rank-1
   approximation).
5. **Refuse hostile use cases.** The same machinery that substitutes
   "greed → Stoic" can substitute "fairness → racist-corpus". The
   technique is dual-use. We will not publish FALS edits aimed at
   removing safety properties from base models, and we ask
   downstream users not to.

### 7.5 Corpus copyright

The Akashic Vault contains a mix of public-domain and licensed
texts. Lineage substitution does not redistribute the corpus —
`d_sub` is a 4096-dimensional unit vector, not a passage — but it
does *derive* from the corpus. Whether this constitutes
transformative use is a legal question we do not resolve here.
Deployers are responsible for verifying that their substitute corpus
permits this derivation.

### 7.6 No claim about "sacred frequencies"

The internal AAMT vocabulary labels concepts with names from the
Solfeggio (432Hz, 528Hz, etc.) and from the four-element / TERA
register systems. These labels are used in this paper *only as
stable identifiers for activation-space clusters*. No claim is made
or implied that ablating a "528Hz direction" has any acoustic,
energetic, vibrational, or metaphysical effect. The mechanism is
mean-difference weight surgery; the names are filenames.

### 7.7 Scaling

The reference implementation has been exercised at the 2–3B
parameter scale. Whether FALS edits remain stable at 70B+ — where
features may live in different layers, in superposition, or be
distributed across attention heads — is an open empirical question.

### 7.8 Compounding edits

The ledger format records edits as independent rows, but in practice
edits compose. Two concept directions that are non-orthogonal will
interact when both are projected out; the second edit operates on a
weight matrix already modified by the first. The current revert
logic handles this for the perplexity guard but does not orthogonalise
the directions. A future version may.

---

## 8. Discussion and Future Work

### 8.1 The design space

FALS occupies a specific cell in the design space of behaviour
modification techniques:

| Technique | Time | Locus | Source | Auditable |
|---|---|---|---|---|
| Prompting | inference | input | curator | weak |
| RAG | inference | input | corpus | medium |
| ITI | inference | activations | probe | weak |
| RepE control | inference | activations | contrast pairs | medium |
| RLHF | training | gradients | preference data | weak |
| Constitutional AI | training | gradients | rule set | weak |
| ROME / MEMIT | post-training | weights (MLP) | target fact | strong |
| Abliteration | post-training | weights (attn) | contrast pairs | strong |
| **FALS** | **post-training** | **weights (attn)** | **contrast pairs + corpus** | **strong** |

The cell-defining properties: post-training (one-time, no retraining),
weights (no inference cost, no input-time inspection), audit-strong
(every edit is logged with a deterministic hash). The novelty of FALS
within this cell is the *corpus-grounded substitute* — every other
weight-resident technique either zeros (abliteration) or targets a
single completion (ROME).

### 8.2 Connection to alignment

FALS is a tool, not an alignment program. It can be used to make a
model better (substituting a stable wisdom-tradition response in
place of model-confabulated wellness advice — the use case driving
this work) or worse (substituting any biased corpus over any
direction). The audit ledger provides a *check* on whether FALS was
used at all and how, which is the alignment contribution.

The strongest alignment claim we make is the following: in a
deployment that requires fidelity to a specific knowledge tradition
(legal, medical, contemplative, organisational), FALS gives a
*training-time* mechanism to bias the model toward that tradition's
substrate, complementing the *inference-time* mechanism of RAG. The
combined system can degrade gracefully (when retrieval fails, the
weights still lean toward the corpus) and can be audited for both
the corpus selection (ledger) and the retrieval policy (RAG logs).

### 8.3 Future work

- **Empirical paper (v1.1):** the full evaluation outlined in
  Section 6.
- **Sparse-autoencoder direction discovery.** Replace the
  mean-difference probe with directions from a trained SAE
  [Templeton 2024], giving access to monosemantic targets rather
  than potentially-polysemantic mean-diffs.
- **Per-expert FALS in Vortex MoE.** As sketched in Section 5.5,
  apply FALS to a single expert's `o_proj` for archetype-conditional
  substitution.
- **Orthogonalising compounding edits.** Gram-Schmidt the concept
  directions before applying, so the second edit's effective
  direction is orthogonal to the first.
- **MLP-resident FALS.** Apply the same projection-then-substitute
  recipe to the down-projection of the MLP block, where ROME and
  MEMIT operate. The MLP locus may capture different concept
  classes than the attention `o_proj` locus.
- **End-user disclosure UX.** A reference UI that surfaces the
  concept edit ledger to end-users — "this model has been edited
  along the following concepts toward the following lineages" —
  as a default transparency mechanism for FALS-equipped products.

---

## 9. Conclusion

We presented Frequency Abliteration with Lineage Substitution, a
generalisation of refusal-direction abliteration that targets
arbitrary labelled concept clusters and replaces ablation-to-zero
with a corpus-grounded rank-one substitute. The contribution is
methodological: a training-time, weight-resident,
auditable mechanism for shifting a model's response on selected
concepts toward a curated tradition without retraining.

The technique is dual-use. We have argued that responsible
deployment requires disclosure of the concept edit ledger and
restraint in the choice of edits. The reference implementation
ships with FALS disabled by default and a perplexity guard that
makes catastrophic regressions self-reverting.

Empirical validation is in progress; v1.1 will report numbers.

---

## References

Arditi, A., Obeso, O., Syed, A., Paleka, D., Panickssery, N., Gurnee,
W., Nanda, N. (2024). *Refusal in Language Models Is Mediated by a
Single Direction.* arXiv:2406.11717.

Hendrycks, D. et al. (2021). *Measuring Massive Multitask Language
Understanding.* ICLR.

Lewis, P., Perez, E., Piktus, A. et al. (2020). *Retrieval-Augmented
Generation for Knowledge-Intensive NLP Tasks.* NeurIPS.

Li, K., Patel, O., Viégas, F., Pfister, H., Wattenberg, M. (2023).
*Inference-Time Intervention: Eliciting Truthful Answers from a
Language Model.* NeurIPS.

Meng, K., Bau, D., Andonian, A., Belinkov, Y. (2022). *Locating and
Editing Factual Associations in GPT.* NeurIPS.

Meng, K., Sharma, A., Andonian, A., Belinkov, Y., Bau, D. (2023).
*Mass-Editing Memory in a Transformer.* ICLR.

Raffel, C. et al. (2020). *Exploring the Limits of Transfer Learning
with a Unified Text-to-Text Transformer.* JMLR 21.

Templeton, A. et al. (2024). *Scaling Monosemanticity: Extracting
Interpretable Features from Claude 3 Sonnet.* Anthropic.

Todd, E., Li, M. L., Sharma, A. S., Mueller, A., Wallace, B. C.,
Bau, D. (2023). *Function Vectors in Large Language Models.*
arXiv:2310.15213.

Wang, K., Variengien, A., Conmy, A., Shlegeris, B., Steinhardt, J.
(2023). *Interpretability in the Wild: A Circuit for Indirect Object
Identification in GPT-2 Small.* ICLR.

Whitehead, W. C. Jr. (2024). *AAMT Foundations: TERA, Vortex, and
HeartScale.* AsAManThinks technical report.

Zou, A., Phan, L., Chen, S. et al. (2023). *Representation
Engineering: A Top-Down Approach to AI Transparency.*
arXiv:2310.01405.

---

## Appendix A — Reference Implementation

Key files in `maiiam-alchemist/packages/training-pipeline/`:

- Direction probing and weight surgery primitives:
  `training_pipeline/abliteration/direction_probe.py`
  (`DirectionProbe`, `project_direction_out_`,
  `apply_substitute_delta_`, `collect_o_proj_inputs`).
- Audit ledger:
  `training_pipeline/abliteration/concept_ledger.py`
  (`ConceptLedger`, `LedgerEntry`, `direction_sha256`).
- Stage orchestration and config:
  `training_pipeline/stages/abliterate.py`
  (`AbliterateStage`, `AbliterationConfig`, `ConceptExemplars`,
  `AbliterateResult`).
- Pipeline wiring documentation:
  `training_pipeline/abliteration/_wiring/abliteration.md`.
- Unit tests on a synthetic toy model (no Qwen/Gemma load required):
  `packages/training-pipeline/tests/test_abliteration.py`.

The substitute corpus retrieval lives in the AsAManThinks platform:
`apps/api/services/vault_search.py` (TERA-aware FTS5 retrieval over
the 500-book Akashic Vault).

## Appendix B — Default Configuration

```yaml
abliteration:
  enabled: false                              # default off
  target_concepts: []                         # explicit positive list
  substitute_from_lineage: []                 # explicit positive list
  probe_layer_range: [8, 24]
  probe_token_pool: mean
  probe_n_examples_per_concept: 64
  abliteration_strength: 1.0
  substitution_scale: 0.5
  revert_ppl_ratio: 2.0
  o_proj_module_pattern: 'model\.layers\.(\d+)\.self_attn\.o_proj$'
  o_proj_path_template: 'model.layers.{i}.self_attn.o_proj'
  ledger_path_relative_to_run: concept_edit_ledger.json
```

---

*End of preprint v1.0.*