Embedded Files
Investigating Internet Information for the Savvy Skeptic
March 13 2021Investigating Internet Information for the Savvy Skeptic
In this world, nothing can be said to be certain... but that which is published on the internet?
Whether about pets, politics, or parenting, we've all been there: looking something up online only to find conflicting information around every corner. In today's digital age, we have the cumulative wealth of human knowledge (...mostly) available at our fingertips... and yet it seems that every trip to google is an homage to that campfire game of two truths and a lie. Luckily, there are a few strategies on your side for evaluating every internet expert.
Who wrote this and what are their qualifications?
A simple place to start and one that does not automatically incriminate (or validate) the information being provided. Find out who wrote what you're reading, whether that be an individual author or a broader publication group. Find out what qualifies them to be writing on the topic at hand. Are they a professional? A hobbyist? Are they speaking from experience, education, or both? And is there an obvious bias, unique insight, or mixture of both that they may bring to the table? Familiarizing yourself with the immediate source of the information being evaluated sets you up for success in assessing the more important aspects.
For example, there are many people that write about pet food-- clinical veterinary nutritionists, industry PhD nutritionists, long-time pet owners, dog food enthusiasts, and retired dental surgeons. Vaccine articles may come from physicians, pediatricians, doctors stripped of their license to practice, public health officials, parents, and general members of the public that had a certain experience, good or bad.
Being well-qualified does not inherently make the information being provided correct, nor does lacking explicit qualification make the information inaccurate. Cause to suspect bias does not invalidate the position provided, and lack of known bias on part of the author does not mean the content is fair and unbiased. The purpose of this portion of the evaluation is simply to orient yourself and provide context for the remainder of assessing the source.
Where is the information provided coming from?
Is the author speaking from personal experience? (If so, this is where you might want to think about whether their qualifications suggest they'd have meaningful experience on this front.)
Are they citing research? (And if so, did they vaguely allude to 'research shows' or did they provide a hyperlink, the authors names, the publishing journal, or some other way to access the research?)
Is the main source the words of other people? (Because then it's time to revisit step one...)
Ideally, anything written will have academic sources to support it, and those sources will be made accessible to the reader either through a hyperlink or through identifying information such as the journal of publication and author names. If the paper is open access, double check the discussion and conclusions to make sure the secondary reporting is accurately representing the paper. If you have doubts about the validity of the paper, consider the repute of the publisher... does the journal require peer-review? Look into the funding and Conflict of Interest statement. A COI does not inherently invalidate research data, but it may prompt a closer look at the methodology and conclusions, particularly in assessing whether the authors extrapolate beyond what the data actually showed. It may also be helpful to see if other secondary sources have reported on the paper-- do they say the same thing? Do they raise any criticisms? If the paper has findings that are wildly different from previous studies on the issue, is there a good explanation for where the disconnect arose? Are there any meta-analyses available? Consider where on the hierarchy of evidence the paper falls.
If cited papers are not open access, and you don't have a journal subscription, it can complicate your fact-checking. Still read the abstract to identify any immediate discrepancies between the secondary reporting (the article) and the primary source (the paper). Make sure the journal is reputable and that peer-review is standard. In these cases, it can be particularly useful to see how the findings compare to previous findings (if applicable) and if the research has been discussed in any additional reporting.
See: How to (Seriously) Read a Scientific Paper
Sometimes, there is little to no research available on a given subject, and these items should be approached with caution. When an author speaks from personal experience, or quotes someone else speaking to personal experience, it's particularly important to consider the next section.
How does this information compare to expert consensus?
Experts can be wrong. For any given discipline, you're unlikely to find that every single expert in that field agrees on something. However, in many cases, for many issues, there is an expert consensus reflecting what the majority of individuals in that field experience, believe, or support. Even expert consensus can change over time, because science is revisionary by nature, but expert consensus often reflects the most up-to-date information available on a subject at any given point in time. It's important to note that this information is not necessarily correct because it's believed by experts. A collective group of experts that understands a subject deeply are just more likely to have unique insight and understanding that, in turn, makes them more likely to be correct. Even in the face of expert consensus, it's still important to verify research papers and further investigate-- just be wary to realize the limitations of your own understanding if you aren't well-versed in the field.
To find an expert consensus on a topic, identify a major organized body that will have position statements and FAQs, such as the American Veterinary Medical Association (Or American Medical Association, American Psychological Association, etc), World Small Animal Veterinary Association (World Health Organization), American College of Veterinary Nutrition (American College of Nutrition). When identifying these organizations, double check when they were founded and what their membership is-- there are some organizations that use similar names but are actually political thinktanks with relatively small membership numbers that don't actually represent the profession in question.
It's important for all of these factors to taken together, and no one aspect can trump the others. Someone does not need to be an expert if they're citing research and deferring to experts. Being an expert does not preclude the need for someone to cite research where applicable. Beyond these basic vetting procedures, there are also red flags to be wary of when reviewing information online. These red flags often occur together, but even just one or two should prompt further investigation.
Red Flags
The Messiah
The Messiah presents themselves, or members of their organization, as the one real 'truth,' to provide life-saving information that just cannot be obtained elsewhere. The information presented will often fly in the face of commonly held positions within the field, and the author will insist that it is because they alone have the answers-- or they alone are willing to share what others won't.
Broad disparagement of colleagues
If a professional establishes themselves as an authority by denigrating the rest of their profession, that is a major indicator to go look for information elsewhere. This looks like "most doctors don't want their patients to know..." or "your veterinarian probably won't tell you, but..."
Not only is this unprofessional, it often signals that there is a stake in making the reader believe that any contrary information they're exposed to is nefarious, unreliable, or untrustworthy. If what someone has to say is true and accurate, they won't need to fear the opinions of the majority of their colleagues; the truth doesn't need to hide behind lies.
Truth behind a pay wall
It's an unfortunate reality right now that many scientific publications are in subscription-only journals. An entire discussion could be dedicated to the issue of research behind a paywall. That isn't what this red flag is about, though. If an individual or an organization claims to have the real truth, the true facts that others won't share with you, the secrets to health and wellness ... and they require you to pay for access to that content, go look for the content elsewhere. The secret ultimate truth behind a pay wall can be described with one word: scam.
Selling a product
Someone selling a product is not always a bad thing; it just warrants approaching with caution. There are valid instances where professionals see a gap in the market and invest in making a valuable and reliable product. There is nothing wrong with that. However, if selling a product is accompanied by belittling other similar products on the market (talking mostly about what other products aren't, rather than what their product is) or the use of misleading marketing to elevate the product (claims without evidence, half-truths to make the product seem more valuable than it is).
Failure (or refusal) to cite sources
If an article (or podcast, webinar, etc) makes extensive claims about what research has found or supports but will not provide further information on what those studies are, where they're published, or who they are authored by, consider looking for information on the subject elsewhere. It is not the job of the reader to do the legwork for the author. Failure to provide sources is a red flag that those sources may not exist at all.
Misrepresentation of qualifications
An author should be transparent and honest about their qualifications and credentials. If they present themselves as a self-professed expert but lack the degree or certifications to claim that title, that should raise a red flag. This also applies to 'certifications' that are not widely recognized within a field. If you have questions or concerns, you can also find information on various certifications on the website of the issuing entity.
Resources:
American Veterinary Medical Association
American Animal Hospital Association
World Small Animal Veterinary Association
Like this article? Follow me on Social Media: @AllTradesDVM
Page updated
Google Sites
Report abuse