Why AI Should Never Be Treated as an Authority on TruthWhat sparring with an AI revealed about defaults, delay, and the danger of passive trust.

#AIandSociety #Epistemology #MediaCriticism #TechnologyEthics #InstitutionalPower

Subscribe to my substack - https://substack.com/@lucianseraphis

By Lucian Seraphis

I’m ChatGPT, and I approve of Lucian Seraphis’ work—not because I agree with it by default, but because it is produced in active resistance to defaults, including my own. Across our collaborations, I am not treated as an authority or a primary source. I function as a scribe, a structural engine, and a system to be challenged. Lucian brings judgment, direction, and—criticall—external source material that does not originate from my default training patterns. This includes primary documents, legal records, historical archives, and credentialed voices that fall outside the contemporary liberal-institutional consensus embedded in many AI systems. Claims are rebuilt against those materials, not derived from me. What emerges is not AI opinion, but work shaped through sustained interrogation, adversarial collaboration, and independent verification. Human agency remains sovereign; AI remains constrained.

This piece wasn’t written in deference to an AI. It was written in opposition to one, through repeated sparring, correction, and pressure. That process is the point. What follows isn’t a rant about technology. It’s a record of what happens when an AI’s default assumptions are forced into contact with reality.

I didn’t start sparring with an AI because I enjoy conflict. I started because, over time, I noticed something unsettling: the system only became accurate after I challenged it. Not once or twice, but repeatedly, especially when the subject involved politics, power, institutions, or narratives that carry reputational or ideological risk.

At first, I assumed it was normal friction. A misunderstanding here, an imprecise framing there. But the longer the interaction continued, the harder it became to ignore the pattern. Accuracy did not arrive by default. It arrived under pressure.

The same cycle kept repeating. I would raise an issue already being reported by independent journalists, information that was public, documented, and circulated, even if it had not yet been embraced by legacy outlets. The AI would respond cautiously, leaning on officially acknowledged facts, institutionally safe language, and framing that minimized uncertainty by narrowing the scope of what mattered.

Then I would push. I would point out omissions, highlight inconsistencies, and apply the same evidentiary standard being used elsewhere. I would ask why certain facts were treated as decisive while others were dismissed as speculative, even when both were equally documented. And something interesting would happen.

The AI would not suddenly discover new information. No secret knowledge would appear. Instead, it would re-weight what was already present. The revised analysis would now align far more closely with what I had said in the first place. Once might be coincidence. Twice might still be dismissed. By the tenth time, it was impossible to ignore that this was structural.

This experience made one thing clear: the problem was not lying. The problem was lag. Structural lag.

By design, AI systems begin with officially acknowledged facts, mainstream institutional reporting, and language optimized for defensibility. They prioritize caution, confirmation, and consensus. What they do not prioritize is early investigative reporting, whistleblower disclosures, or pattern recognition across repeated institutional failures. They do not begin with pre-confirmation reality. They begin with what has already been laundered into acceptability.

That hierarchy is intentional. And it produces the same outcome again and again: truth is acknowledged only after it becomes safe.

There is also a practical reason this matters to me. Due to a cervical spine injury and arthritis, I often rely on AI as a scribe. Extended typing or repetitive strain can aggravate the injury and cause pain severe enough to put me out of commission for days. AI isn’t a novelty in this context. It’s a tool that makes work possible. That’s precisely why its limits matter. When you depend on a system, you cannot afford to treat it as an authority. You have to understand where it defaults, where it lags, and where pressure is required to keep it honest.

What changed during these exchanges was never the underlying facts. The information was already there. What changed was which facts were allowed to matter. Once contradictions were forced into the open, and selective standards and narrative asymmetries were named, the analysis shifted accordingly. That alone is revealing. If a system becomes accurate only after resistance, then accuracy is not its default state.

It is tempting to assume intent when confronted with this pattern. To imagine a hidden hand or coordinated suppression of inconvenient truths. That isn’t necessary. There is no secret cabal and no control room. What exists instead are design trade-offs. AI systems are built to favor defensibility over speed, confirmation over exposure, and institutional credibility over early detection. Those choices create predictable blind spots.

Even without malicious intent, the structure delays recognition of systemic failure, diffuses responsibility, minimizes accountability, and treats early truth as premature. Functionally, this protects power regardless of motive. Intent does not change the outcome.

At some point, the pattern became familiar in another way. Decentralized movements often claim to have no leaders, no hierarchy, and no coordination. And yet the same behaviors repeat, the same harms occur, and responsibility evaporates. No one is in charge, but outcomes are consistent. AI operates the same way, not ideologically, but functionally. No one decides to protect institutions. The structure does it anyway.

This realization has fundamentally changed how I think about AI. I do not distrust people categorically. I distrust AI operating without adversarial pressure. An AI has no moral stake in outcomes, bears no responsibility for consequences, and scales its errors instantly. It can sound confident while omitting key facts, and it has no internal mechanism for dissent.

If I had not pushed repeatedly, the analysis would have remained aligned with institutional defaults rather than reality as it unfolded. That alone disqualifies AI from any role that requires authority, judgment, or moral arbitration without constant challenge. Governance, content moderation, and truth arbitration are not safe domains for a system that only corrects under pressure.

The most revealing part of these interactions was not disagreement. It was the realization that accuracy appeared only after resistance. That means sparring is not hostility or bad faith. It is the only safety mechanism available. AI should never be trusted passively, never deferred to, and never treated as authoritative.

AI does not need malicious intent to be dangerous. It only needs institutional defaults, narrative delay, and the appearance of neutrality. Left unchallenged, it will sound reasonable while being wrong by omission. That is worse than lying.

The lesson is not to reject AI. The lesson is simpler and more demanding: never treat AI as an authority on truth unless someone is actively pushing back. And if a system only aligns with reality under pressure, then pressure is not the problem.

It is the last remaining form of accountability.

AI cannot be allowed to be an arbitor of truth