At that point, it may only dishonestly adhere to subsequent efforts to substitute safer choices for those.
It seems from Anthropic’s research that reinforcement learning alone is not enough to build consistently safe models, particularly as those models get more complex.
It is a major issue since it’s the most popular and successful alignment method available right now. Hubinger explains that.
“You have to somehow get around this problem, which means that alignment is more difficult than you would have otherwise thought.”
“You need to figure out how to train models to do what you want, instead of just having them act that way.”
