Harvard just admitted something a lot of people have been dancing around.
Gary King and Steven Worthington run rapidPeer out of Harvard’s Institute for Quantitative Social Science. Paid human reviewers. Double-blind. Constructive. The kind of reviews journal editors have called among the best they’ve seen. Authors rate the service a 10 out of 10. These are not the lazy, “I found a fatal flaw on page three” reviews academia is famous for. These are humans at their best.
They compared 134 of those human reviews against AI reviews written by Anthropic’s Claude Opus, across 124 scholarly papers. Then they had another AI (OpenAI’s GPT-5.5) judge the pairs, and three former editors of major journals judge a sample. The AI review won every pair when the judge was another AI. It won 87 percent when the judges were humans. Higher mean rating on all seven criteria they measured: sources, weaknesses, methods, theory, framing, communication, figures and tables. Fewer fabricated sources. About a third the rate of incorrect claims.
King said they went in expecting to prove the value of human reviewing. “That’s not what we found.” They’re going to have to pivot what the organization does. He also floats the “realistic” policy: some journals skip routine human review. Author submits the paper, an official AI review, and a response to the AI. Editor may add a private AI pass. Human review only if needed.
Surprise. Actually, for me. Not so much. We have to be honest. This was always bound to be the outcome. I mean, it even won in the opinion of people. But, it still misses the point a bit, doesn’t it? Here is part of the reason why.
The paper itself warns that AI tends toward the conventional and the majoritarian, and that a world of AI reviews could push science toward homogenous arguments. King asks out loud whether OpenAI and Anthropic should be deciding what’s good science. Fair questions. They don’t answer whether AI reviews actually make the revised papers better. They measured the reviews, not the outcome.
Still. The scoreboard is the scoreboard.
Which brings me to the little world.
Circle is being built for writers who want to get better at the work. Members request assignments. Reviews are structured and prompted, submitted on the site, guiding reviewers through various lenses and encouraging real feelings. Writers can revise and resubmit. Earned credit can be spent on your own work or on someone else’s. It’s not a busy marketplace yet. It’s early. We’re looking for founding members who want to get to work, who’ll have real say in how this grows, and who’ll help build toward the vision. The operational bones are in place. We need builders, but we are also ready for those that just want a place to learn, practice, and improve. We have over 20 writing activities alread operational.
On AI, Circle doesn’t try to make the law, and it doesn’t draw up complex formulas for what’s allowed. Members are asked once, before their first submission, to be honest about if and how they use AI. It’s part of the membership agreement. Honesty. That’s what we ask. Your word and your reputation. How you feel about AI writing or production is yours to decide, as it should be.
There is one place AI is not allowed: reviewing someone else’s work. We are not interested in how AI feels about it. You can go get AI opinion from an AI chatbot. What we want from a review is honest and informed human feedback.
And no shit — that line just got a lot sharper. If AI can outscore paid experts on every checklist a journal cares about, the temptation to hand reviews to the machine will be enormous. Faster. Cheaper. Less awkward. That’s fine for science policy debates. Not fine here. Circle is not optimizing for the best review by every measurable criterion. It’s optimizing for a person who read your work, felt something, and told you the truth.
If that’s you, join us. Earn your satchel and your office.
Registration is open at gapoz.com. No fees. Membership to the Circle is limited and subject to approval. It always will be. We don’t want to build a city where nobody knows you. We want to build a community, where everybody does. Satchels are earned, not given away. Come meet people, make some friends, and grow your craft together.
Your name. Your word. Your work.
Subscribe and receive a weekly summary of news and events at Gapoz.
Gapoz Press (gapoz.com) has opened up some great new writer features and a cool page for finding out all about them Explore Gapoz Features
Also on the beat
- Mexican publisher Almadía cleared David Silva Rojas’s prize novel Donde el agua trabaja after Pangram flagged it ~99% AI; they used human review of his process materials and asked El País to apologize — El Español
- Publishers Weekly profiles Proudly Human’s $60 “AI-free” book certification (four detectors, under 10% pass / over 30% fail) — Publishers Weekly
- Philippe Jaenada’s Goncourt AI admission keeps rolling in English-language coverage — The Guardian
- YouTube is testing a 30-day locked “fix-it” period before spam terminations, including AI mass-production channels — Tubefilter
- A clear KDP walkthrough: disclose AI-generated text/images/translations; AI-assisted brainstorming and editing does not require disclosure — Publisher Success
How this was made
AI assisted research and a first draft of this Circle View. Richard Crossley told the AI what to look for and then reviewed it, edited (that’s a turn of the screw aint it), and takes responsibility for the judgment and the final take.
