Work through four questions before a published claim earns any influence over what you do: what exactly was compared, who was included and who was left out, how long anyone was followed, and who funded the work. You can answer all four from the methods section and the disclosure statement, which is why those come first and the abstract comes last.
The abstract is written to be quotable. That is where overstatement lives.
What does the design tell you?
Design determines the kind of question a paper is capable of answering, and nothing more.
A description of a handful of cases can show you that something is possible. It cannot tell you how reliably it happens. Work that looks backward through existing records can raise a question worth asking, while leaving open how the cases were selected in the first place. Work that follows patients forward answers more, and work that assigns patients to groups at random answers more still.
A summary that pools other people’s work inherits every weakness of what it pooled.
None of this is a ranking you apply mechanically. Design is where you start reading, not where you stop.
What should you check in the methods?
Five things, in this order.
- Who was included and excluded. Narrow criteria are common, and they are the reason a finding may not transfer to the patients actually sitting in your chair. Read the exclusion list and ask how many of your own patients it would have removed.
- How many patients. Small groups produce unstable numbers. Ask whether the authors said in advance how many they needed and why.
- How long, and how many dropped out. A claim measured over months tells you little about something you expect to last years. Heavy dropout can bend a result in either direction.
- What was actually measured. Substitute measures are everywhere. Something visible on an image is not the same as the thing lasting, and neither is the same as how the patient functions.
- Who did the measuring. Independent, blinded assessment is a different proposition from an operator grading their own work.
How much should funding matter?
It matters, and it does not disqualify anything on its own. Somebody has to pay for this work. Read sponsored work with exactly the same scrutiny you apply elsewhere, and pay closer attention when a convenient result rests on an odd comparison.
Watch for the comparison group in particular. A product compared against something dated, or against nothing at all when a current alternative exists, tells you more than any disclosure statement will.
How do you read the numbers without a statistics background?
Ask how big the difference is and how uncertain it is, rather than whether it cleared a significance threshold. Then ask the only question that matters at the chair: would a difference that size change what you do for a patient?
Watch for relative and absolute framing. The same result can be described two ways, both technically true, one of them far more flattering. When only the flattering version appears, go find the other one.
Also check whether the result being promoted is the comparison the authors set out to make. Findings that appear in the results but not in the stated aims are worth treating as questions for someone to test later, not as answers.
What does this look like on a Tuesday?
When someone hands you a paper, take it. Then run the four questions and the comparison-group check. If it survives, it deserves an hour of your attention.
Read a small number of papers carefully rather than skimming many. One paper you genuinely understand can change how you practice. A stack of abstracts changes nothing except your confidence.
FAQ
Where should I find full text I can actually read?
Start with your national and state association memberships, which often include library access, and with any academic affiliation you hold. If you cannot reach the full text, write to the corresponding author, who is usually willing to send it. Do not change practice on an abstract alone.
How long should I wait before adopting a new material or technique?
Long enough that independent work with meaningful follow-up exists and early adopters have described the failure modes. Then scale the wait to how reversible the intervention is. The less easily undone it is, the longer you should be willing to wait before going first.
Is a pooled summary always stronger than a single trial?
No. A well-organized summary of weak work is still weak. Check what was included, how the authors judged quality, and whether the underlying work was similar enough that combining it made sense at all. If that last answer is unclear, treat the conclusion cautiously.