The AI Slop Review Rubric for Editors: 5 Scored Dimensions and Where to Draw the Reject Line

1Why Most AI Review Checklists Fail

Not for lack of items. The checklists in circulation are often long and some carry point systems with cutoffs. They fail for three narrower reasons.

They are not reproducible. One widely circulated rubric scores twelve criteria from 0 to 10 for a 120-point total, publishing at 84 and above, holding 60–83 for one revision and redrafting below 60. Precise-looking, and several of those twelve criteria are judgement calls like originality and tone. A scale that asks you to rate originality out of ten behaves like a mood ring: hand the same draft to two editors and you get two totals. A rubric is only an instrument if it returns the same answer twice, which means every criterion has to reduce to something countable — a ratio, a density, an enumerated list of phrases.

They do not separate fatal from fixable. Almost every published checklist treats its criteria as interchangeable point sources, so a draft can lose points for bad adjectives and for having no sources and land in the same band. Those two failures cost wildly different amounts to repair. One is a find-and-replace. The other is a reporting assignment.

They measure the wrong thing. Search for editorial guidance on AI drafts and a large share of what comes back is published by companies selling detectors or humanizers, whose recommended workflow is to score a draft for "AI-ness" and then rewrite it until the score drops. Consider what that pipeline optimises for: not a truer or better-sourced draft, just one that survives a detector. Nothing about the reporting changed. Confusing those two goals is how you publish well-disguised slop, and it is why the dimensions below measure whether a draft says anything, not whether it sounds machine-made. We have written at length about why detectors fail; the shorter version is that teaching editors to beat one is career-limiting advice.

"Editing is also a matter of judgment about what is most relevant, original, or entertaining about the piece. This judgment depends on understanding both the subject and the readership, neither of which AI can do."
— WIRED, "How WIRED Will Use Generative AI Tools"

That sets the ceiling on what any rubric can do. It narrows the field; it cannot pick the winner.

2The Rubric: Five Dimensions at a Glance

Each dimension scores 0–20, where 0 is clean and 20 is saturated. They sum to a 0–100 total. The diagram shows the pass, borderline and fail band for each dimension on that 0–20 scale, with the reject line marked.

The totals map to the engine's published verdict bands. These are the thresholds in the code, not a scale invented for this article:

Two honest notes. The middle column is what the tool outputs; the third is this article's editorial recommendation layered on top, since the engine ships no Accept/Revise/Reject button — adopt the mapping or set your own cutoffs, but write them down, because that is the part that makes it a rubric. And inputs under about thirty characters return zero, so score sections and whole drafts, never a headline.

3Each Dimension, With a Fail and a Pass

For each: what it measures, what failure looks like, what a pass looks like, and the one question to ask yourself. The fail examples are written to fail — do not copy them.

Two details in the Substance dimension are worth stealing as manual heuristics even if you never open the tool.

First, the vague-subject list. The engine counts eleven interchangeable phrases: this topic, this subject, this field, this area, this industry, this space, this domain, the world, our society, the modern world, the landscape. Three or more and the dimension takes its full substitutability penalty. You can grep your queue for that list this afternoon.

Second, a positional check that mirrors how editors actually read: the platitude scan looks only at the last two hundred characters, against endings like the possibilities are endless, only time will tell, continues to evolve, pave the way, shape the future. A platitude buried in paragraph nine is a wart. The same phrase as the closing line is a confession that the draft ran out of things to say. Read the last two sentences first. It is the cheapest triage move in the queue, and you can hand writers the full vocabulary reference so they stop sending you drafts that trip it.

4The Reject Line: What Is Fatal, What Is Fixable

Vocabulary and Cliche are the most diagnostic failures and the cheapest to fix. Substance is the least diagnostic failure and the most expensive. Those two sentences look contradictory and are not. Holding both is what separates a usable rubric from a word blacklist, and it is the thing no other review checklist says out loud.

Diagnostically, Vocabulary and Cliche are the strongest tells, and the engine weights them that way: Vocabulary at 18 or above with two or more cliche phrases firing adds a twelve-point combo bonus. The code calls these the two true-signal dimensions, because they catch a lexical fingerprint rather than proxies like sentence-length uniformity, which modern models vary well enough to dodge.

Editorially, they are trivia. Every one of those failures is a word swap. A competent writer clears a saturated Vocabulary score in twenty minutes and the draft underneath is unchanged — which is exactly what a humanizer pass does: it launders the signal and leaves the emptiness.

Substance is the inverse. It is a weak authorship signal, since plenty of human copy is vague, and it is the only failure you cannot edit your way out of. A draft where the specificity sub-check finds zero numbers, zero dates, zero multi-word proper nouns and zero URLs takes the full eight-point penalty, and it is telling you something no line edit addresses: no reporting happened. There is no fact in the document to tighten, and asking the writer to improve the paragraph is asking them to invent one — the failure mode you least want to encourage.

Two findings explain why this is the dimension to be strict about. First, you cannot rely on being told: a 2025 audit of 186,000 articles from 1,500 American newspapers found roughly 9% were partly or wholly AI-generated, and of 100 flagged articles checked by hand, exactly 5 disclosed it. Second, the thing an unsourced draft is most likely to be hiding is a citation that does not exist. Walters and Wilder checked all 636 cited works in 84 generated documents: 55% of the GPT-3.5 citations and 18% of the GPT-4 citations were fabricated outright, and among the citations that did exist, 43% and 24% respectively carried substantive errors — wrong authors, wrong titles, wrong years. An editor who checks only whether a source exists still ships a bad citation roughly a quarter of the time.

So the reject line has two parts: the total-score band, and an override. If Substance alone is in the fail band, reject regardless of the total. A draft can score a respectable 34 overall on the strength of varied sentence rhythm and clean vocabulary while containing not one verifiable fact. That is not a draft that needs editing. It is a brief that was never reported, and it goes back as a brief. The named failure types in our slop taxonomy — Generic Slop, Pseudo-Insight Slop, Fake Authority Slop, Wikipedia Rehash — are mostly Substance failures wearing different costumes, which is why giving them names helps a team argue about a draft faster.

5Three Verdicts, Not a Score

A writer cannot act on "62." They can act on a verdict. Your review ends with one of exactly three, and each has a standard sentence.

Why three and not a number? Because of the WIRED line above. A rubric can tell you a draft is padded, templated and unsourced. It cannot tell you that section three is the actual story and the lede is buried under it. The score gets you to the decision faster; it does not make it.

Note where the real policies draw their lines. AP's approved uses run to early-stage research, summarisation, transcription, translation and headline suggestions. WIRED allows AI for headline ideas but makes an editor approve the final choice, and refuses AI-written or AI-edited text outright. Neither gates on origin for its own sake; both gate on output quality under human sign-off. Same bet this rubric makes.

6What to Write in the Revision Request

A specific ask names the location, names the defect, and offers a binary. Here are three, in the form you can paste.

Three things make those work. They name a paragraph, so there is no argument about scope. They say what is wrong rather than how it feels. And each ends with permission to cut — often the right answer, and one writers will not take unless you offer it. For the version you hand the writer directly, our guide to writing content that is not slop is the companion document, and the measurable signs of AI writing gives them the thresholds in advance.

7Why It Matters That Nothing Is Uploaded

This is the part of the rubric a hosted competitor cannot copy, and the reason it belongs in a procurement conversation rather than a features list.

Most content-QA tools are a text box that posts your draft to a server. For a post about garden furniture, fine. For copy under embargo until Tuesday, a legal filing in review, an earnings note or client material under NDA, that is a disclosure event. The vendor's privacy policy does not help, because the question counsel asks is not whether the vendor promises to behave — it is whether the text left the building. Here it does not: the scoring modules contain no network calls, and the analysis runs in the browser tab.

Which changes how you deploy the rubric: it goes in the standard review step for every draft, sensitive ones included, instead of needing a carve-out. A process with an exception for confidential material gets skipped exactly when the stakes are highest.

8What This Rubric Does Not Do

It does not detect authorship. Not partially, not probabilistically, not as a hint. A high score means padded, templated, unsourced text — which humans produce unassisted, in volume, on deadline. A low score means specific and varied text, which a careful person using a model also produces. The rubric measures the draft, not its origin, and that distinction is the whole premise of how we define slop.

It is not evidence in a dispute. Never open a conversation with a writer by citing a score as proof they used AI. You will sometimes be wrong, the accusation is unfalsifiable, and the damage is asymmetric: the writer cannot prove a negative. People wrongly accused of using AI lose grades, contracts and jobs over tools that cannot explain themselves, which is also why authorship detectors keep failing. The correct use of a high score is to go look at the draft and name the concrete defect you find. If you cannot name one, there is no finding.

It does not check facts. A score of 8 means well-formed prose, not true prose. Specific, confident, well-sourced-looking and wrong is the most dangerous output in your queue, and given the fabrication rates above, verification stays a separate human step. The rubric tells you whether a draft is worth verifying; it is not the verification.

It does not replace the sign-off. The responsibility question is already settled in publishing ethics: COPE's position statement holds that AI tools cannot be listed as an author, and that authors are fully responsible for the content of their manuscript, even the parts produced by an AI tool. Swap "author" for "editor" and the rule holds for your queue. The rubric is an instrument you are accountable for using, not a party you can blame.

Frequently Asked Questions