How to Be a Savvy Internet Skeptic
Investigating Internet Information for the Savvy SkepticMarch 13 2021
For example, there are many people that write about pet food-- clinical veterinary nutritionists, industry PhD nutritionists, long-time pet owners, dog food enthusiasts, and retired dental surgeons. Vaccine articles may come from physicians, pediatricians, doctors stripped of their license to practice, public health officials, parents, and general members of the public that had a certain experience, good or bad.
Being well-qualified does not inherently make the information being provided correct, nor does lacking explicit qualification make the information inaccurate. Cause to suspect bias does not invalidate the position provided, and lack of known bias on part of the author does not mean the content is fair and unbiased. The purpose of this portion of the evaluation is simply to orient yourself and provide context for the remainder of assessing the source.
Are they citing research? (And if so, did they vaguely allude to 'research shows' or did they provide a hyperlink, the authors names, the publishing journal, or some other way to access the research?)
Is the main source the words of other people? (Because then it's time to revisit step one...)
Ideally, anything written will have academic sources to support it, and those sources will be made accessible to the reader either through a hyperlink or through identifying information such as the journal of publication and author names. If the paper is open access, double check the discussion and conclusions to make sure the secondary reporting is accurately representing the paper. If you have doubts about the validity of the paper, consider the repute of the publisher... does the journal require peer-review? Look into the funding and Conflict of Interest statement. A COI does not inherently invalidate research data, but it may prompt a closer look at the methodology and conclusions, particularly in assessing whether the authors extrapolate beyond what the data actually showed. It may also be helpful to see if other secondary sources have reported on the paper-- do they say the same thing? Do they raise any criticisms? If the paper has findings that are wildly different from previous studies on the issue, is there a good explanation for where the disconnect arose? Are there any meta-analyses available? Consider where on the hierarchy of evidence the paper falls.
If cited papers are not open access, and you don't have a journal subscription, it can complicate your fact-checking. Still read the abstract to identify any immediate discrepancies between the secondary reporting (the article) and the primary source (the paper). Make sure the journal is reputable and that peer-review is standard. In these cases, it can be particularly useful to see how the findings compare to previous findings (if applicable) and if the research has been discussed in any additional reporting.
Sometimes, there is little to no research available on a given subject, and these items should be approached with caution. When an author speaks from personal experience, or quotes someone else speaking to personal experience, it's particularly important to consider the next section.
To find an expert consensus on a topic, identify a major organized body that will have position statements and FAQs, such as the American Veterinary Medical Association (Or American Medical Association, American Psychological Association, etc), World Small Animal Veterinary Association (World Health Organization), American College of Veterinary Nutrition (American College of Nutrition). When identifying these organizations, double check when they were founded and what their membership is-- there are some organizations that use similar names but are actually political thinktanks with relatively small membership numbers that don't actually represent the profession in question.
The Messiah presents themselves, or members of their organization, as the one real 'truth,' to provide life-saving information that just cannot be obtained elsewhere. The information presented will often fly in the face of commonly held positions within the field, and the author will insist that it is because they alone have the answers-- or they alone are willing to share what others won't.
Broad disparagement of colleagues
If a professional establishes themselves as an authority by denigrating the rest of their profession, that is a major indicator to go look for information elsewhere. This looks like "most doctors don't want their patients to know..." or "your veterinarian probably won't tell you, but..."
Not only is this unprofessional, it often signals that there is a stake in making the reader believe that any contrary information they're exposed to is nefarious, unreliable, or untrustworthy. If what someone has to say is true and accurate, they won't need to fear the opinions of the majority of their colleagues; the truth doesn't need to hide behind lies.
Truth behind a pay wall
It's an unfortunate reality right now that many scientific publications are in subscription-only journals. An entire discussion could be dedicated to the issue of research behind a paywall. That isn't what this red flag is about, though. If an individual or an organization claims to have the real truth, the true facts that others won't share with you, the secrets to health and wellness ... and they require you to pay for access to that content, go look for the content elsewhere. The secret ultimate truth behind a pay wall can be described with one word: scam.
Selling a product
Someone selling a product is not always a bad thing; it just warrants approaching with caution. There are valid instances where professionals see a gap in the market and invest in making a valuable and reliable product. There is nothing wrong with that. However, if selling a product is accompanied by belittling other similar products on the market (talking mostly about what other products aren't, rather than what their product is) or the use of misleading marketing to elevate the product (claims without evidence, half-truths to make the product seem more valuable than it is).
Failure (or refusal) to cite sources
If an article (or podcast, webinar, etc) makes extensive claims about what research has found or supports but will not provide further information on what those studies are, where they're published, or who they are authored by, consider looking for information on the subject elsewhere. It is not the job of the reader to do the legwork for the author. Failure to provide sources is a red flag that those sources may not exist at all.
Misrepresentation of qualifications
An author should be transparent and honest about their qualifications and credentials. If they present themselves as a self-professed expert but lack the degree or certifications to claim that title, that should raise a red flag. This also applies to 'certifications' that are not widely recognized within a field. If you have questions or concerns, you can also find information on various certifications on the website of the issuing entity.