Last July, a YouTube creator I was subscribed to published a review of ProWritingAid. She knew the tool. That much was clear. She had used it enough to walk through its reports with the fluency of someone who had actually run them on her own work. She had an affiliate link in the description.
Give her credit for knowing the tool. She does. The problem isn’t that she lied. The problem is what the tool actually is, and the ambassador program has no incentive to name it. The program is designed to produce that video, the creator followed the program, and the tool is the problem.
The broader pattern is worth naming while we’re here. The ambassador video isn’t the exception. It’s the format: writing advice content produced on a schedule to justify a subscription or an affiliate arrangement. Canva presentations. Basic skills presented as revelation. The form is optimized for production speed and distribution cadence, not for the complexity of the thing it’s supposedly teaching.
Prescriptive rules applied without context are a category failure, whether the rule is a content rating or a readability score. Both measure exposure, not intent. The writer watching that video gets the surface. The evaluation requires doing the work yourself, because the content-as-commerce model has no incentive to go that deep. Going that deep takes longer. It might even recommend against the affiliate product.
So I ran my own evaluation. My background is systems engineering, not creative writing pedagogy. I was looking at something different than she was.
What I saw was a Python script with a reporting layer.
What It Actually Is
ProWritingAid is a statistical aggregator. It runs a document through predefined measurements, returns numbers, compares them against hardcoded thresholds, and produces a report. The 25-plus reports are 25 statistical measurements with a UI. There is no reading happening. There is no scene awareness. There is no recognition that the document in front of the tool is a horror-gothic fight scene versus a conversation versus a chapter summary block that was never intended to be scored at all.
The architecture is the problem. Every failure I’m about to name traces to the same missing piece: a pipeline stage that would have done the work of making the data clean before a single metric fired.
You can’t ship a measuring instrument without a preprocessing step and claim it reads prose. You can ship it and call it that. The tool does both.
Before the Receipts, a Note on the Alternative
Canary is going to show up a lot in the next six sections, so the name needs a proper introduction before the receipts start.
Canary is a writing-evaluation skill I built for Claude Code and Cowork over the past couple of days. It runs a ProWritingAid-style scored report against genre-specific thresholds, with a preprocessing pipeline in front of the scoring engine and judgment notes behind it. It is alpha. I vibecoded it on April 22 and 23 of this year and pushed the repo public at github.com/elf1024-vot/canary. The full fix-section treatment comes later in this article. For now, the relevant fact is that it exists, and that every time I name a missing piece in ProWritingAid’s architecture, I can point at the stage of Canary that fills that gap, because I had to build the gap-filler myself.
The receipts are what happens when that gap-filler isn’t there.
Six Receipts
Each of these is an architectural failure. Not a product failure. The architecture cannot do the thing. The product is the architecture delivered on schedule.
1. No Language Recognition Step
My character Blondie speaks Spanish. She code-switches mid-sentence, the way bilingual Tucson residents do, because that’s how the voice is built. “Mas perdida que un borracho en misa.” Roughly: more lost than a drunk at Mass. It’s a Las Levantadas idiom, precise in its register, and my manuscript is full of sentences like it.
ProWritingAid flagged every Spanish word as a misspelling. Not some of them. All of them. There is no language recognition step in the pipeline. There is no “before the spellchecker runs, identify what language each span is in.” The tool assumes American English and flags everything else as error.
Canary’s Recognition Order has six steps. Step two is multi-language recognition. It fires before the spellchecker does. “Gracias” is recognized as Spanish, not as a misspelled English word. That isn’t a luxury feature for writers working in multiple languages. That’s the minimum bar for not being wrong about your own characters.
2. No Scene Awareness
The same metric fires the same way regardless of what’s happening in the scene. A fight scene and a conversation get identical treatment because the tool has no mechanism to distinguish between them. Word density is word density. Sentence variety is sentence variety. The metric doesn’t know which chapter has a knife in it.
A tool that can’t tell which chapter has a knife in it isn’t reading your manuscript. It’s counting it.
3. Calibrated for Corporate Prose
Flesch-Kincaid was designed in 1948 for US Navy training manuals. Glue index was calibrated on professional prose. Sentence variety floors were built on whatever corpus the tool’s engineers chose, and the corpus was not horror-gothic fiction.
These thresholds travel under genre labels. The labels don’t recalibrate the underlying math. Calling a readability ceiling “fiction mode” does not change what the Flesch-Kincaid formula measures or what it was designed for. The Sentence Variety floor of 5.5 does not know it is reading a knife fight. It counts and compares against a standard that has nothing to do with what fiction is trying to do.
4. Tag Prescription Without Scene Awareness
ProWritingAid’s dialogue tag guidance defaults toward “said” as the correct answer across every scene type. The documentation is explicit: general writing advice is to avoid anything other than said. That is a prescription without scene awareness.
In a conversation, it’s defensible. Two people talking over coffee, “said” is often the right call. In a knife fight, it’s wrong.
Action beats are the tags in a fight scene. Gestured, moved, stood, thrashed, cut, dug, pinned. Nobody is just saying things while a knife is out. The action is the attribution, because the action is what’s happening. A tool that cannot distinguish a conversation from a fight scene cannot give valid tag guidance, and recommending “said” across both is architecturally the same mistake: applying a universal rule to prose that has no uniform register.
5. No Preprocessing Pipeline
The document goes in whole. Every production marker my manuscript system generates, ProWritingAid scores as prose.
My chapter files contain a What Happened: block. It’s a shorthand chapter summary, written in my own voice, for my own tracking. From Blondie Ch13.1:
Michelle completes an abandoned-laundry-room “ritual” meant to correct the universe by forcibly exchanging blood, and thus identity, with Blondie, luring her there under false pretenses and blocking escape with knife and restraints; when Blondie recognizes the setup as predatory escalation, she refuses, is attacked, and the confrontation turns violent. Michelle’s certainty collapses as Blondie’s superior vampiric reflexes dismantle the ritual and disarm her, and when Michelle lunges again, Blondie ends the threat decisively with an improvised stake. Michelle disintegrates to ash amid the wreckage of her pentacle and symbols, leaving Blondie alone with the irreversible knowledge that she has just destroyed another vampire to survive, the fantasy of “transformation” extinguished by lethal reality.
That is a production artifact. It is not prose. It was never meant to be scored against sentence-variety thresholds.
ProWritingAid scores it anyway. It has no mechanism to strip it, because it has no preprocessing stage.
My chapter files also contain [[INTERNAL DIALOGUE: text]] paired delimiters. These tag internal character voice with formatting distinct from prose, so my beat sheet and my manuscript can coexist in the same file. ProWritingAid scores these as prose, because it has no mechanism to handle paired delimiters as a category.
Canary’s strip engine runs before any metric fires. It reads a per-manuscript config that names what is and isn’t prose for this specific manuscript. For the Blondie run: What Happened: block stripped via the header-strip rule, [[INTERNAL DIALOGUE:]] handled as paired delimiter under Mode A, chapter title lines stripped.
Strip verification delta: 6,145 source words, 5,974 stripped words, delta 171 words. Those 171 words are scaffolding. They don’t reach the scoring engine. That is not a setting. That is a pipeline stage. ProWritingAid does not have one.
6. The Net Result
It makes your writing worse, not better.
That is the take the ambassador program has every incentive not to say. A tool recommends action beats be replaced with “said” regardless of whether your character is having a conversation or pinning someone against a washing machine with a stake. A tool lists every Spanish word your characters speak as a misspelling. A tool scores your chapter summary block as prose because it has no preprocessing stage. A writer who accepts those recommendations is making their manuscript worse.
You paid $399 for a lifetime license to a tool that cannot do the job it is marketed to do on fiction like yours. That is not a writing tool. That is a spreadsheet that read your manuscript.
What the Fix Looks Like
I built the alternative over two days. It’s called Canary. It’s alpha. I vibecoded it, pushed the repo public, and it already does things a $399 production release does not. If that’s embarrassing for anyone, it isn’t embarrassing for me.
The architecture matters more than the implementation. A real software engineer with a real development background could take what’s in the repo and build something production-worthy. What’s there now is a proof-of-concept, and the proof is that the architecture works.
The receipts above each name a missing stage. Canary has that stage. The preprocessing pipeline runs before any metric fires, the thresholds are calibrated per-genre and swappable, and the Recognition Order handles multi-language spans, coined terms, proper nouns, and production markers as distinct categories before a single metric triggers. Each receipt inverts directly into a named stage of the architecture. That’s half the fix.
The other half is four pieces the receipts didn’t cover.
Per-manuscript configuration that persists across runs. The pwa_config.json file sits next to the manuscript. Every run picks up the same rules. The writer doesn’t re-enter preferences every session, and Canary doesn’t forget what a What Happened: block is.
Chapter summary and beat sheet generation, produced by the same run that scores the prose. Canary reads the chapter, writes a narrative summary of what happened, then emits a beat sheet with word ranges per beat and structural observations. ProWritingAid does not do this. It has no mechanism to do this, because summarization is a reading task and the tool does not read. Canary returns it as a side-effect of running the report. This is designed to ensure that Canary understands what it is reading and is giving you quality results.
If the summary or beats are wrong, you know to throw out the results.
Judgment notes that distinguish craft from error. This is the piece ProWritingAid cannot replicate, because its output format has no place for it. Canary flags a metric miss and then, where the miss is a craft feature rather than a defect, names it as such. The next section demonstrates this, inside the article, on two canon chapters.
Mode separation between report and fix, and Canary never edits the original file. Mode 1 is read-only scoring. Mode 2 is grammar and spelling corrections. Mode 3 is full auto-fix under user-configured rules. In every mode, edits are written to a new file alongside the original. The manuscript the writer handed Canary is the manuscript the writer still has when Canary is done. The writer picks the mode. The tool doesn’t decide for them, and the tool doesn’t touch what they started with.
What Canary Does With a Miss
Here is the demonstration, inside this article.
Same genre (F5 Horror/Gothic). Same author. Same tool (Canary, my build). Same day (April 23, 2026). Two chapters, scored on Sentence Variety, an old standby both Canary and ProWritingAid measure, with an F5 floor of 5.5 where higher numbers mean more varied sentence structure.
Babydoll Book 2, Chapter 19.1. Las Levantadas arriving at Abuela’s. Four women sizing each other up in a sanctuary house. Conversation register, tactical-survivor POV.
| Metric | Value | Floor | Result |
|---|---|---|---|
| Sentence Variety | 3.44 | 5.5 | MISS |
Blondie Book 3, Chapter 13.1. Michelle’s attack. The stake. The washing machine. Blood on tile.
| Metric | Value | Floor | Result |
|---|---|---|---|
| Sentence Variety | 4.13 | 5.5 | MISS |
Two misses. Same metric. Two completely different chapters, same genre, same author, on the same day.
ProWritingAid looks at those numbers and tells the writer their sentences aren’t varied enough. Maybe a checkbox to “add complexity.” Maybe a pop-up flagging the paragraphs responsible. A number, a threshold, a verdict, and a recommendation.
Canary tells the writer this:
Sentence Variety 3.44, below the F5 floor of 5.5. This reflects a consistent short-clipped register carrying Babydoll’s tactical-survivor voice (“Yeah.” “Understood.” “He never stopped.”) and the fragment-heavy internal-mode spans. The register is character-consistent. Horror as a category expects denser sentence architecture; Babydoll as a POV expects clipped authority. Accepted as a craft choice.
And for Blondie:
Sentence Variety 4.13, below the F5 floor of 5.5. Sentence variety and average length both flag. The prose leans short and punchy for extended stretches, which is voice in horror/gothic but costs variety points. Review if you want to normalize, not because the report flagged a defect.
Same metric. Same MISS verdict. Two entirely different explanations, each one tied to the specific scene. A tactical-survivor POV misses the variety floor for one reason. A fight scene misses it for another. Canary knows which is which because it knows what it is looking at.
ProWritingAid would have printed 3.44 and 4.13 and moved on. That is the contrast. Not which tool got the right answer. Which tool knew enough to name its own limitations.
The number is the same. What follows the number is the difference.
Four Questions to Ask Any Writing Tool Before You Trust Its Scores
You don’t have to use Canary. The repo is public at github.com/elf1024-vot/canary if you want it. Download canary.skill, install it in Claude Code or Cowork, run it on your own manuscript. Alpha. Bring a helmet.
You also don’t have to use ProWritingAid. What you need, regardless of the tool, is a way to evaluate whether the architecture underneath is doing what you’re being told it does. Here is the checklist.
Four Questions to Ask Any Writing Tool Before You Trust Its Scores
1. Does it have a preprocessing stage?
Before any metric runs, can you tell it what is not prose?
Headers, production markers, chapter summaries, internal dialogue
tags, beat sheet lines. If the tool cannot strip these before
scoring, the numbers include them. That is not your prose score.
That is your prose plus your scaffolding.
2. Does it know what language your characters speak?
If your manuscript contains Spanish, French, Tagalog, or any
language other than corporate American English, does the tool
recognize those words before calling them misspellings? A six-step
recognition order is not a luxury for multilingual fiction. It is
the minimum bar for not being wrong about your own characters.
3. Do its thresholds know what genre they are reading?
Flesch-Kincaid was designed for US Navy training manuals. Glue
index was calibrated on professional prose. Sentence variety was
not built for horror-gothic fight scenes. If the tool applies the
same thresholds to your thriller and your literary fiction and
your corporate email, it does not have genre settings. It has
genre labels on universal settings.
4. Can it tell you when its own metric fired wrong?
A tool that reports Sentence Variety below floor for a clipped
first-person POV and moves on is not helping you. A tool that
reports the same miss and then says "the register is character-
consistent; accepted as a craft choice" is reading your
manuscript. The number is the same. What follows the number is
the difference.
Apply it to whatever tool you currently use. If the tool fails on all four, you’re holding a spreadsheet. If it fails on three, you’re holding a spreadsheet with a nicer UI.
Closing
I said in The Standards Document that ProWritingAid wouldn’t have caught the tell my wife caught. Grammar checkers catch errors. Standards documents catch tells. This is the architectural reason why.
A tool without a preprocessing stage cannot know what to ignore. A tool without scene awareness cannot know what to weight. A tool without genre-aware thresholds cannot know what “good” looks like for the prose in front of it. A tool without judgment notes cannot tell you when it’s wrong. Stack those four absences on top of each other and what you have is not a writing tool. It is a measuring instrument pointed at the wrong object, reporting exactly what it was built to report, sold to fiction writers through an ambassador program that has every incentive not to name what the instrument actually measures.
It’s not a writing tool. It’s a spreadsheet that read your manuscript.
You may also like:
- The Standards Document
- The imageprompt Field or Your Character’s Eye Color Doesn’t Change in Book 6
- Defining Character Voice