THE MARGINALIA / № 04

What peer review actually filters.

2026-06-25  //  FROM THE DESK OF Anders L.  //  PHD — RESEARCH DESK

Students are told to use peer-reviewed sources and are almost never told what the phrase means. It is not a certificate of truth. It is a record that two or three people in the field read a manuscript and thought it worth publishing.

01What the reviewers were actually asked.

Roughly this: is the question worth asking, is the method capable of answering it, do the conclusions follow from the results, and is the writing clear enough to assess. That is a genuinely useful filter and it removes a great deal of poor work.

Notice what it does not include. Reviewers do not, in the ordinary case, see the raw data. They do not rerun the analysis. They do not visit the site, contact the participants, or check that the study happened as described. They are reading an account and judging whether the account is coherent and competent.

So the honest translation of peer reviewed is something like: plausible to specialists, on the evidence presented, at the time of publication. That is worth a great deal and it is not the same as verified, and the gap between those two readings is where students get into trouble.

02What the label does not tell you.

Five things worth holding in mind whenever you attach the phrase to a source in a paper.

03Reading the label like somebody in the field does.

Researchers do not treat peer reviewed as a binary. They read it alongside three other things, and you can do the same in about ninety seconds per source.

How many participants, and drawn from where. This single fact tells you most of what you need about how far a finding travels, and it sits in the abstract. Then the design: whether anything was manipulated or merely observed, which decides what the paper is entitled to claim rather than what it does claim. Then the date, held against how fast the field moves, because a decade in some areas is nothing and in others is a different discipline.

None of that requires expertise. It requires reading the methods section rather than the conclusion, which is the habit most undergraduate reading skips and which is covered practically in the issue on reading a paper fast.

04Why your marker cares about this.

Because the commonest substantive fault in student writing is not using a bad source. It is treating a good source as though it settled something it did not settle, and the peer-reviewed label is what licenses that overconfidence.

A sentence saying a study found something is safe. A sentence saying research shows something, resting on one study of forty people, is the sentence that gets marked, and it gets marked precisely because the writer clearly believed the label had done the work of qualification for them.

So the practical move is to attribute rather than announce. Naming who found it, in what kind of study, on how many people, costs about eight words and converts an overreach into a defensible claim. It also demonstrates exactly the reading a marker is trying to assess, which is why the eight words usually earn more than they cost.

05Where non-reviewed sources are the right answer.

Some things simply are not published in journals. Government statistics, professional standards, clinical guidelines, regulatory documents, industry data, dissertations. A student told only to use peer-reviewed sources will sometimes cite a weak paper describing a guideline rather than citing the guideline, which is worse research by any measure.

The rule almost every marker actually applies is that a source should be appropriate to the claim: authoritative for the thing it is being asked to support. Where your assignment genuinely restricts you to journal articles, follow it, and where it does not, the wider category is worth knowing about, which is grey literature.

Where you are unsure which applies, that is a two-minute question to an instructor rather than a guess, and asking it early is the cheapest thing on this page.

Questions at this desk.

So is peer review unreliable?

No, and that is the wrong conclusion to draw. It removes a large amount of poor work and it is the best filter the system has. The point is narrower: it filters plausibility to specialists at one moment in time, not truth, and a reader who knows which of those they are holding writes more carefully.

How do I tell whether a journal is any good?

Check whether your library indexes it, whether the publisher is one you can identify, and whether the editorial board lists real people at real institutions. Where something arrived through a web search rather than a database, that is worth two minutes of scrutiny, and a librarian will settle it in one.

Does a preprint count?

It has not been reviewed, and it should be described as a preprint wherever you cite it. Some fields rely on them heavily and some marking sheets forbid them outright, so check your own. What is never acceptable is citing one in a way that implies it went through review.

Is there a cut-off for how old a citation can be?

It depends entirely on the field and on what the source is being used for. A foundational paper from decades ago is often the right citation for a definition, while a claim about current practice usually needs something recent. The question to ask is what this particular source is doing in your sentence.

Anders L.
CH 001.42
Anders L.
PhD — research desk
THE MARGINALIA // ONE LETTER A WEEK

Notes in the margin, from the people at the desks.

READ THE LETTER
ELSEWHERE ON THE SHELF
Retractions and your citationWhat grey literature is forReading a paper fastThe research paper desk

Ask for a desk. A specialist replies within two hours.

REQUEST A DESK