We built W10 to answer one question we kept asking by hand: does this draft read like a person wrote it? After eighteen drafts in a round, nobody on our team could still tell. Flavour fatigue sets in around the sixth one, and the first one you read sets the standard for all the rest. So we measured instead.
This post covers how the scorer works, how it changed over three weeks, and what its latest version, W10V4, can and cannot tell you. Three other posts cover how we got here: Qwen tied Opus at a tenth of the cost, so we made it our Writer, twenty rules cut the stink in our drafts and showed us what they can't fix, and a second pass by the same model took our draft from 67 to 88.
What W10 counts
W10 measures habits of prose that are cheap to count: how long sentences run and how much they vary, how often a sentence opens with something other than its subject, how often the reader is addressed as "you", whether the subject of a verb is a person or a thing, and how many sentences describe a general kind of person in the present tense with no particular place or time. The last one we call the Jaqen habit, also known as the Jaqen rule, after Jaqen H'ghar, a character who never says "I" and calls himself "a man".
The yardstick is a reference of thirteen writers, about 6,000 words each. Ten are essayists and authors with voices nobody would confuse. Three are company blogs whose 2022 posts nobody has touched since, which rules out a model as their author. We chose them for distinct styles, and we make no claim that the list is objective. The texts stay private, and only their numbers ship.
For each measure the scorer asks how far a draft sits from the writers' middle, in units of how much the writers differ from each other. Within one spread is green, within two is amber, beyond that is red. Too little counts as much as too much. The writers' own most extreme value on a measure is set aside before the range is drawn, so one writer's genre does not stretch it. Self-help writers talk to "you" and reporters use the past tense, and neither should define the range.
How it grew
Version 1 had eighteen measures and ten writers. Our first eight Writer drafts scored a median of 2.5 greens and 14 reds. Version 1.1 added the three company blogs to the reference. Later versions added measures for things that do what only minds do ("the paragraph decides"), for the Jaqen habit, for chains of the shape "do this and that happens", and for a group we call Funk, which holds the machine tics: dashes, "quietly", "not X but Y", trailing -ing phrases, announcements. The measures fell into five groups: Rhythm, Flow, Voice, Specifics and Funk. We weighted the groups equally, because nobody could defend any other weighting.
The fly in the soup
The earlier total had a flaw we only saw when we tried to cheat it. Each measure could cost a draft at most its own point, so a measure thirty spreads out cost the same as one at three. A perfect post stuffed with "you" still scored 87.
Our fix is the one statisticians use for outliers. Each measure now pays a penalty that grows with the square of its distance beyond the green zone, up to four spreads, and grows in a straight line after that. One bad habit can spoil a post, and no single measure takes the score to zero by itself. The total is 100 times e raised to minus the average penalty divided by 7.3, where 7.3 was set so a median writer lands near the top.
In W10V4 we also removed two measures that counted a habit twice. A run-based version of the "you" measure agreed with the plain one 92 percent of the time, so a draft paid for the same habit twice. W10V4 has 24 measures.
| One perfect post, one habit pushed out | Score |
|---|---|
| One measure six spreads out | 89 |
| One measure ten spreads out | 78 |
| "You" six spreads out | 75 |
| "You" ten spreads out | 54 |
| The four Specifics measures six spreads out | 63 |
What the writers score
Scored against the other twelve at 6,000 words, the thirteen writers have a median of 96, from 79 to 99. The lowest are a company blog at 79, a philosopher at 81 and another company blog at 82. The blogs lose points on "you" because their genre addresses the reader. The philosopher loses them on sentence length. We kept those scores as they are. A scorer that exists to catch habits should not be tuned until every writer clears a bar of 85.
What it does with machine drafts
On a fresh topic, publishing prices when the honest answer is "it depends", we drafted four texts of about 2,500 words, one per setup, and scored them.
| Draft | W10V4 |
|---|---|
| Our latest prompt rules with a self-review pass | 92 |
| Our current Writer, then a self-review pass | 88 |
| Our current Writer | 67 |
| Plain Opus, no system prompt | 58 |
Then the founder read excerpts of all four without knowing which was which. The order he gave matched the scorer's at the ends: the top draft sounded natural to him, and plain Opus came last. In the middle, he could not tell the reviewed draft from the unreviewed one. He would have scored all four within five to ten points of each other. The scorer spread them across 34.
Where it stops
- It measures habits, and a reader hears more than habits. The gap between a score of 58 and 92 is real, and it is smaller on the page than on the scoreboard.
- The model does most of the work. Plain Opus on a good brief wrote something its reader called great. House style and a review pass reduce the stink. They do not turn a draft into the work of a writer people seek out.
- Length matters. Rates wobble on short texts. The writers' own 1,300-word pages score lower and vary more than their 6,000-word totals. We draft test texts at 2,500 to 2,600 words now, and a text pasted twice scores exactly like the text once, because rates do not change when you repeat them.
- It is not an AI detector. It reports which habits a text has and how far they sit from writers people read on purpose.
If you run a blog, the useful output is the list of measures that sit far from the writers, because each one points at a sentence you can rewrite. The score is a summary of that list. A free version you can run on your own draft is coming to our site, along with a whitepaper that covers the whole study.