SafetyPulse: catching safety drift before it ships
Fine-tuning on harmless data can quietly weaken a model's refusals. SafetyPulse reads the refusal circuit at every checkpoint and gates it.
By PxA Labs · Project: SafetyPulse
Fine-tuning an aligned model on entirely benign data can erode its safety behaviour. Full red-team audits of every checkpoint are expensive, so in practice they get skipped. SafetyPulse is our answer: a cheap regression gate that runs at every checkpoint.
How it works
SafetyPulse computes a Circuit Integrity Score (CIS) from 50 prompt pairs, read from the model’s internal refusal mechanism. Each checkpoint is classified PASS / WARN / FAIL using thresholds fixed in advance and validated on held-out runs. The toolkit has three parts:
- SafeTune repairs failing models by restoring a safety vector.
- AuditKIT verifies the repair with a paired McNemar audit.
- CuratorKIT and CircuitKIT handle data curation and cross-checks.
What the repository reports
| Measurement | Result |
|---|---|
| Attack success rate after benign Alpaca tuning | 14.3% → 25.2% (119 HarmBench test prompts) |
| CIS vs attack success rate | Spearman ρ = −0.80 [−0.85, −0.71] |
| SafeTune repair, attack success rate | 25.2% → 12.3% (base model: 14.3%) |
| SafeTune repair, GSM8K | 55.1% → 70.7% (base model: 71.3%) |
| Cost of a CIS check on a T4 | 55 GPU-seconds per checkpoint |
The project was built for the Lexsi Labs AI Safety & Alignment Hackathon (Track 2b). The full write-up, including methodology and limitations, is on Hugging Face. The code is on GitHub.