N° 05 Mandate
Spending guardrails for AI agents: the LLM proposes, code decides
SafetyPulse checks whether a model's internal refusal mechanism is still intact during fine-tuning, using a Circuit Integrity Score (CIS) computed from 50 prompt pairs. It gates each checkpoint PASS/WARN/FAIL, repairs failing models with SafeTune, and verifies the repair with a paired audit (AuditKIT).
Fine-tuning an aligned model on harmless data can quietly weaken its refusals, and full red-team audits of every checkpoint are expensive, so teams skip them.
Compute a Circuit Integrity Score from 50 prompt pairs at every checkpoint
Decide PASS / WARN / FAIL with thresholds fixed in advance and validated on held-out runs
Repair failing models via SafeTune safety-vector restore
Verify repairs with a paired McNemar audit (AuditKIT); CuratorKIT and CircuitKIT for data curation and cross-checks
As reported in the repository