Steering vectors can work as activation oracles! Inspired by @isolyth.dev 's recent work on instruct vectors, I've ported the same approach to create Activation Oracles. Seems to work surprisingly well given the param deficit, nearly matching Activation Oracle LoRA performance.
Post
I'm quite excited to share, as this is my first foray into interpretability & safety research.
0 reposts8 likes
No replies yet.