Fake news has been in the news a lot lately. People say that it is hard to tell the real stuff from the fake stuff. A basic rule of thumb — what you read while in the checkout aisle in the supermarket is pretty much guaranteed to be fake. Elvis is still dead. But in other cases, it can be difficult to differentiate, particularly when there are numbers involved.
Numbers seem so factual. Take “10” for example, a good solid number that we accept for what it is. When someone says that they have 10 apples, you have a clear idea of what they are talking about and can easily verify if that is in fact the case. But much of what we deal with in managing organics is not that easy to count, quantify and verify. Even when numbers are provided, they are not as easy to count and interpret as those 10 apples. Despite this, we are often asked to make decisions based on those same numbers. So here’s a basic guide to help you understand what constitutes a real number — worth its weight in fingers and toes.
Let’s start with numbers relating to the safety of our products. I am currently working with data on concentrations of pharmaceuticals and personal care products in composts — something that people are really worried about, particularly if there are biosolids or manures in the feedstocks. The data set was generated by a very fancy piece of equipment and people tend to be reluctant to question a number when the machine that provides it costs over $100,000 and gives you four significant digits. While those machines can do amazing things, they are sometimes pretty stupid. You have to make sure to include enough checks and balances in your samples to check the machine’s work.

Crazy Numbers

We can use caffeine as an example. And you will need lots of caffeine to look at a full data set. The machine tells me one compost sample has 15.6 ng g of caffeine. Another type of compost has 41,000 ng g of caffeine. Units are the first thing to understand. Nano grams per gram or ng g is the same as parts per billion. A billion is a lot. But 41,000 is a very small fraction of a billion. In fact it is the equivalent of $0.41 to $10,000. Even though that 41,000 isn’t very large from a financial perspective, it needs to be put into the perspective of typical caffeine concentrations in compost. One of those composts had to be produced solely from coffee grinds or one of those numbers is wrong. That is Rule #1: Don’t take those numbers, however many digits they have and however expensive it was to have the analysis run, as absolutes. If you get a crazy number, it does not mean that the sky is falling. It most likely means that the crazy number is wrong.
One way to check is to see what other people have reported. Not everybody can be making compost from pure coffee grounds. That is Rule #2: Make sure that there are sufficient checks when you have your samples run or when you look at data. These help make sure that the machine didn’t have a quadruple espresso shot before running your samples. All analyses should include a range of safety checks to make sure that the machines and the sample preparation have been done properly. First thing to check for is blanks. That is your way to know if the analysis understands the concept of zero. If something that is supposed to measure 0 comes in at 1,400 you know you have a problem with the analysis. Say the blank for caffeine in compost came in at 10 — that tells me that that 15.6 ng g is likely a lot lower in reality.
Running duplicate samples is another way to check an analysis. When you run two of the same samples you almost never get identical results, but if you get similar numbers, that is a really good sign that your numbers are in the ballpark. Also make sure that known standards are included in the analysis. The National Institute of Standards and Technology (NIST) sells great standards — even soil, biosolids and plant standards. The numbers they report are worth trusting. If your lab gets results that are similar to the reported NIST values, you are in great shape.
So if you are in charge of running analyses or sending samples out for analysis or if you have to look at sample results, don’t take the numbers as cold, hard and fast. Instead, make sure that you or the lab you use are following appropriate quality assurance, quality control (QA/QC) steps.

Common Sense

What about people who don’t deal with data, but instead just read reports, news articles or other sources of information that contain numbers. In many cases, the reports’ authors present numbers like they are the whole truth and nothing but the truth. Unfortunately that is not always the case. The way numbers are interpreted can often be misleading. Which leads us to Rule #3: Common sense and logic are your best tools to see if what the numbers say and what the authors say are really the same thing. Going back to the caffeine example, say that Compost A had 15 ng g of caffeine and compost B had 60 ng g of caffeine. The author of a report on those composts could correctly say that Compost B had highly elevated caffeine in comparison to Compost A, and argue (technically) that Compost B had 400 percent more caffeine than Compost A.
That sounds terrifying. It is your job to realize that both of those numbers are parts per billion — and that even with a good pair of reading glasses you couldn’t distinguish between the two and neither could any environmental receptors. It is your job to realize that exposure to 60 ng g of caffeine couldn’t keep much of anything up for any amount of time. Reading literature to put numbers into context is the way to reality check the numbers and determine what they are really telling you.
Numbers — when correct and put into the proper context — are your friends. They are real news. They can assure safety and efficacy of your products and programs. They can give you a solid leg to stand on. It is up to you to use your suspicious mind to figure out if those numbers are nothing but a hound dog or the devil in disguise.
Sally Brown is a Research Associate Professor at the University of Washington in Seattle and a member of BioCycle’s Editorial Board.