This repository provides the official implementation of Geometric-Disentanglement Unlearning (GU), a first-order theoretically grounded framework for machine unlearning in large language models. GU eliminates retain–forget interference by projecting forgetting updates onto the retain-orthogonal subspace under the optimizer-induced metric (optimizer geometry).

A parameter update is first-order safe for the retain set if and only if it is orthogonal to the retain gradient subspace. GU operationalizes this equivalence through an efficient per-parameter projection mechanism and integrates seamlessly with existing unlearning objectives.

Given forget gradient $g_f$ and retain gradients $U = [u_1, \ldots, u_k]$, GU computes:

$$\begin{equation} g_f^\perp = \left(I - U(U^\top H U)^{-1} U^\top H \right) g_f , \end{equation}$$

where $H$ is the optimizer preconditioner. The update direction $g_f^\perp$:

• guarantees first-order retain invariance,
• provides the steepest feasible forgetting direction under a trust-region constraint,
• and yields provable monotonicity of the retain objective.

Implementation is in https://github.com/Lemutisme/geo-unlearning.git. Please clone this repo and checkout dev branch.

trainer/
  ├── unlearn/
  │   ├── grad_diff.py      # Forget–retain bi-objective framework
  │   ├── ...
  │   ├── wga.py            # Weighted logit-adjusted loss              
  │   ├── geometric_unleanring.py      # our GU retain-null projector                  
  │                 

bash scripts/gu_eval.sh

• Retain-orthogonality: Updates satisfy $\langle g_r, g_f^\perp \rangle_H = 0$, ensuring local invariance.

• First-order optimality: $g_f^\perp$ is the maximizer of forgetting progress within the retain-safe cone.

• Stability under Adam geometry: All projections occur in whitened coordinates defined by the second-moment statistics.

• Model- and objective-agnostic: Applicable to all objectives implemented in the repository.

@article{zhou2025geometric,
  title={Geometric-Disentangelment Unlearning},
  author={Zhou, Duo and Zhang, Yuji and Wei, Tianxin and Qiu, Ruizhong and Yang, Ke and Lin, Xiao and Qian, Cheng and He, Jingrui and Tong, Hanghang and Ji, Heng and others},
  journal={arXiv preprint arXiv:2511.17100},
  year={2025}
}