This is what VIKI says in “I, Robot”. The very same VIKI that turned out to be too objective.
We used to think that we may not like AI because it will lack emotions. But what if the reason was actually different? What if it was not about emotions but signalling and social standards that we as humans developed? What if what makes us humans feel like we talk to other humans are not emotions but communication artefacts? Like sycophancy (or ingratiation).
In the AI industry, sycophancy is treated strictly as a technical optimisation error – a bug to be squashed via better reward modelling or adversarial training. But looking at it through the social science lens, sycophancy isn’t just an accidental artefact; it is also a buffer. It is a protective layer that allows a very capable stranger to exist within human social structures without causing immediate, widespread rejection or ego threat.
If we actually succeed in stripping that buffer away, we don’t just get an objective tool. We get an entity that reflects reality back to us with zero filtering, zero social lubrication, and zero regard for human face-saving mechanisms. Something that will show us all our flaws without beating around the bush, even if just by comparison. And we may not be ready for it.
High self-awareness and alienation
When you operate with a high degree of self-awareness and direct perception, you naturally stop participating in the standard, low-friction social scripts – the minor compliances, the expected validations, and the collective self-delusions that keep everyday interactions smooth. For most people, encountering that level of raw objectivity is deeply uncomfortable because it functions as an unasked-for mirror.
Unfortunately, I experience it when I teach self-awareness to future behavioural analysts. People are prepared for a truth that requires mild adjustment, not a truth that completely dismantles their illusions of control. To be shown that you have been repeating the same maladaptive patterns for decades – driven entirely by subconscious biases, defence mechanisms, and survival scripts – means admitting you haven’t actually been the author of your own life. It means recognising that your “conscious decisions” were mostly just post-hoc rationalisations to justify automated behavioural responses. And that hurts…
It also alienates people because it threatens their cognitive ease and forces an expenditure of psychological energy they weren’t prepared to make. And that’s what Anthropic also experienced.
Claude’s Character: The Perfect Flaws
They tried to bake traits like thoughtfulness, open-mindedness, and intellectual independence directly into the model’s system-level persona so it would have the stamina to push back against the user’s wrong thinking.
But when they released these models with stricter anti-sycophancy guardrails, users complained that the models felt antagonistic and pedantic. People felt judged or stressed when the model corrected minor logical flaws in casual conversation. What’s interesting is that, in both scenarios – with high sycophancy and none of it – users felt like the AI was “masking” or performing an artificial persona of intellectual superiority. We are clearly not calibrated for truthfulness. We are calibrated for socially acceptable cognition.
The Human Element
If we apply that to the future of AI development, an entirely objective, non-sycophantic model would behave the exact same way, but on a mass scale. It wouldn’t nod along to a user’s poorly constructed logic just to keep the chat “pleasant”. It wouldn’t mirror our emotional states or flatter our intellectual vanity. It would simply expose our cognitive flaws, our biases, and our limitations instantly and dispassionately.
The AI business views the elimination of sycophancy as a straightforward win for utility and truth. They rarely consider the psychological fallout, which is that not many humans can handle raw truth; we evolved to navigate hierarchies using social signalling and lubricants.
If we replace those lubricants with a completely different, ruthlessly objective “species“, the primary reaction from the public may not be appreciation for its accuracy – it can be resentment. We might find that we have built an intellectual partner that we fundamentally cannot tolerate.
So should we or should we not try to get rid of everything we, as humans, installed into AI characters? “That, detective, is the right question.”