Substack AI Detection and What It Catches

Substack's AI detection feature exists to protect a business model built on a single premise: readers pay because they believe a specific human is writing to them. If that belief erodes, the economic logic of the individual newsletter subscription falls apart, and Substack built its scanner to defend that logic before it came under visible strain.

Why Substack built a detection feature in July 2026

Chris Best, Substack's CEO, named the specific threat the company was trying to get ahead of: "Claudefishing," his term for using AI to simulate human connection with readers who believe they're hearing from a person. Best framed the goal in equally direct terms, saying he wanted to prevent Substack from becoming "like LinkedIn," a platform where authorship has grown ambiguous enough that readers no longer assume a human wrote what they're reading. The timing of the launch mattered as much as the feature itself. The feature launched July 21, 2026, and applies to posts, notes, replies, and comments that exceed a minimum length, covering content published on or after that date.

What the Pangram scanner does and does not do

The scanner returns a percentage estimate of how much of a given piece of text appears human-written versus AI-assisted. It returns a probabilistic estimate rather than a pass or fail judgment, as Substack has been explicit about. The scope of what it covers, and what it leaves untouched, is specific enough to spell out directly: it works on posts, notes, replies, and comments, but only on items over 100 characters; it works only on content published on or after July 21, 2026; it's available on web, iOS, and Android; and it does not work on items that fall below the character threshold, where it simply returns "not enough text," nor does it work on video or audio posts, the email version of a post, or standalone Substack sites viewed outside the app and Reader. Anyone can run a scan, not just the author or a platform moderator. The signal the tool produces is public by design, independent of any internal moderation workflow. Substack itself has acknowledged that Pangram has limits on what it can detect, and the company has hinted at further features around AI content and reader preferences still to come. Two things sit structurally outside what the tool can see: whether the idea behind a piece of writing originated with a human, and whether genuine effort or care went into producing it. The scanner reads the surface statistics of the words on the page, not the provenance of the thinking behind them.

The writer-facing controls Substack built alongside the scanner

Substack paired the reader-facing scanner with a set of tools aimed at writers, treating disclosure as a parallel system built alongside it. Writers can run the Pangram scanner on their own drafts before publishing, which gives them a preview of how a piece will score before any reader sees the result. A "How I make this" statement lets a publisher describe their creative process in their own words, a disclosure field that sits alongside the work itself. Writers who believe a scan produced an inaccurate result can report it, and Substack has framed that reporting function as a genuine part of the system, treating the scan as a starting point for conversation. None of this activity, Substack has said, feeds back into training generative AI models, whether through Substack's own systems or through Pangram.

How behavioral classifiers like Pangram flag AI text

Pangram works as a behavioral classifier. It identifies statistical patterns in word choice, sentence structure, and other stylistic regularities that correlate with output from post-trained language models, rather than scanning for specific banned phrases or applying fixed rules. The detection community names a more specific cause behind that general description: base language models are trained to match the human distribution of text as closely as possible, while post-training and alignment, the steps that make a model useful and safe to deploy commercially, produce the statistical signatures classifiers are specifically trained to find. That means the thing being detected isn't "AI" in some general sense. It's the particular fingerprint that alignment and instruction-tuning leave on a model's output. Both processes end in text, and only the text passes through the classifier.

Why certain legitimate writers trigger false positives

The clearest case of structural unfairness in this system involves writers who don't have full command of English. That risk traces back to how large language models were built in the first place: they were trained on enormous volumes of human writing, and their output inherits the statistical regularities of human prose, including the habits of particular writers and particular genres. The line between "sounds like AI" and "sounds like a careful, somewhat formulaic human writer" is genuinely blurry at the level the classifier operates on. Writers whose natural voice leans on repetitive sentence structures, certain transitional phrases, or consistent punctuation habits face a higher chance of misclassification than writers with more idiosyncratic, less patterned styles, simply because those regularities overlap with what the classifier has learned to flag. One troubling response has already appeared among early users: some writers started deliberately flattening their own voice, sanding down the very qualities that made their writing distinct, in order to score better on a tool meant to protect authentic human expression. That outcome runs directly against the feature's stated purpose. Substack's answer to this problem is disclosure. The "How I make this" note and the ability to report an inaccurate scan function as the correction mechanism, not a promise to rebuild the classifier itself.

What model-level watermarking adds on top of behavioral detection

Pangram and tools like it operate after the fact, reading finished text for statistical traces once it already exists. Model-level watermarking works earlier in the pipeline, built into the act of generation itself, embedding a signal in the specific words a model chooses to output. A watermark of this kind isn't visible to a reader at all; it's recoverable only by a detector holding the right key, which makes it a fundamentally different kind of signal than anything a behavioral classifier can infer from style alone. Anthropic announced in August 2026 that Claude would watermark its output, and the company followed that announcement by releasing a detection API in private preview in September 2026, made available to regulators and law enforcement. Regulation is moving in the same direction on a separate track: the EU AI Act, passed in 2024, requires AI model providers to watermark their output, and that provision became enforceable on August 2, 2026, with providers already active in the EEA market given until December 2, 2026 to come into compliance. As of August 2026, Claude stands as the clearest confirmed case of an embedded, machine-readable text watermark applied with no opt-out, though that coverage currently extends only to the most recent model variants and not yet to legacy deployments, which are being brought into compliance over the transition period running through December 2026. This carries a direct consequence for anyone publishing on Substack: a post drafted with a watermark-enabled Claude variant carries a signal that Pangram may not yet be built to read, even though a purpose-built detection API could read it. The detection infrastructure surrounding a platform like Substack is layered across multiple systems moving at different speeds, not a single static tool.

Research-level watermarking: semantically conditioned fingerprinting

Academic research is already moving past the kind of watermarking Anthropic has deployed. A paper at ICLR 2026 proposes fingerprinting through semantically conditioned watermarks, an approach that uses entire semantic domains as fingerprint queries instead of relying on specific individual prompts to recover a fingerprint key. The signal this method embeds grows stronger as the number of tokens increases, meaning longer passages of generated text carry a more recoverable fingerprint rather than a weaker one, the opposite of what intuition might suggest. That design makes the approach structurally different from earlier watermarking schemes, where the identifying signal lived in a specific word or phrase that editing could plausibly scrub out. Here the signal is distributed across the semantic structure of an entire passage, which makes it considerably more resistant to paraphrasing or light surface editing than anything watermarking has offered before.

What evades detection

The ELOQUENT 2026 Voight-Kampff competition tested generators including Claude Opus 4.7, GPT-5.4, Gemini 2.5 Pro, and Qwen3.6, and the results point to a clear asymmetry in what actually defeats adversarial detectors. Pushing generated text out of a detector's training distribution reliably works as an evasion strategy; pulling generated text into that distribution, by mimicking the patterns of human training data, fails completely. The same research found that detectors retrained to catch adversarial examples still carry persistent vulnerabilities under structural shifts in how text is generated, the kind of change that comes from altering a model's architecture. Superficial evasion techniques, including grammar-level perturbations, synonym swapping, and light paraphrasing, turn out to be fragile: once a detector gets retrained on examples using these exact techniques, their fool rate collapses to near zero, measured in the same research at 0.025 or below. Separate 2026 research points toward a more durable approach. Coding agents equipped with a base language model, rather than a post-trained, aligned one, can assemble responses by stitching together text samples drawn from that base model, producing output that reads as coherent and task-specific while substantially reducing detection rates, with one measurement showing a commercial classifier's detection rate on Pangram v4 fall from 77 percent to 24 percent. The explanation traces back to the same distinction raised earlier: base models are trained to match the human distribution of text as closely as possible, while the post-training and alignment steps that make a model commercially useful are what introduce the statistical signatures classifiers learn to find. Text assembled from base model samples simply carries fewer of those signatures to begin with. Older humanization techniques built around iterative paraphrasing involve a real trade-off rather than a clean win: each additional round of rewriting risks semantic drift, where the text gradually loses coherence and specificity, and while newer methods such as HIP are designed to manage that trade-off more gracefully than earlier approaches, they don't eliminate it.

Substack's transparency-first model is one coherent institutional response to this whole problem, but it isn't the only one on the market. LinkedIn announced its own detection systems in May 2026, trained in partnership with its editorial team and aimed specifically at posts that appear machine-generated and carry no clear point of view, with a stated standard that posts and comments must represent the author's actual voice and perspective. Even the opt-out functions as information rather than silence, since a reader who sees "AI detection unavailable" on a post is still being told something by the absence of a score. Neither platform's system measures what may be the most important variable of all: the judgment, experience, and original perspective a human being brought to a piece of writing, regardless of which tools touched the prose along the way.

Why single-model output is structurally vulnerable to detection

Everything the research surfaces points back to one architectural fact. Text generated entirely by a single post-trained model carries a consistent, learnable statistical fingerprint, because the alignment process that makes a model safe and useful for a commercial product is the same process that leaves behind the regularities a classifier is trained to catch. That fingerprint doesn't depend on content or topic; it's a byproduct of how the model was built. That is why classifiers trained on one domain often generalize reasonably well to another. The evasion research makes the implication unavoidable: the methods that actually work, structural shifts away from a detector's training distribution, base model stitching, semantically distributed watermark signals strong enough to survive editing, all target the architecture of generation itself rather than the surface appearance of the finished text. A single aligned model producing prose in its default mode is, by the nature of how it was trained, the easiest case for a classifier to catch. Writers and developers navigating detection honestly need to understand that this is a property of the architecture generating the words, not a flaw in any one scanner's design.