I like facts. I especially like facts that are backed up by measurable and reproducible numbers. The way people talk about numbers sometimes annoys me in three areas: accuracy, precision and presentation.
Accuracy is the degree of closeness of measurements of a quantity to that quantity’s actual value.
Precision is defined as the degree to which repeated measurements under unchanged conditions show the same results.
Presentation is how the numbers are presented for a specific purpose.
A given measurement can be accurate but not precise, precise but not accurate, neither, or both. Consider your car’s odometer. It will be very precise, probably able to reproduce the same distance measurement between two points along the same path within a few feet. But it may not be very accurate, due to tire pressure, which lane you were driving in, or other factors.
These two terms are often used interchangeably in normal conversation or even in scientific papers. Often the concept is expressed either in terms of significant digits or a range. For example, the sun has a diameter of about 865,374 miles (more than three times the distance from Earth to the moon). It is not exactly 865,374 miles across – that is just the best estimate. But for most people, knowing that that sun is about 900,000 miles across is good enough for daily conversation. That is a one significant digit answer. 870,000 miles would be a two-significant digit answer.
Most people make an unconscious correlation between the number of significant digits and accuracy. If I told you there were 217 people at our meeting, you would believe that I actually counted them. If I told you there were about 200 people at our meeting, you would believe I did not actually count them. You might even think that I was deliberately overestimating for some reason, such as to show we had a lot of support for some position or action. In fact what may have happened is that I estimated 200 people in the meeting, and Joe asked if I had counted the 17 people in the balcony. When I said “no” Joe added them together and published the result.
When combining numbers in any way, the significance of the answer can be no higher than the least significance of any of the individual numbers. Taking a one significant digit number (200) and adding to it a two significant digit number (17) should be a one significant digit number. The reported number should have been 200, not 217.
A classic example of “precision enhancement” is women’s gymnastics at the Olympics and other significant events. The score is formed by a number of judges. Each judge provides two numbers: the degree of difficulty (D) and execution (E). D starts at 0.0 and increases based on the skills successfully completed. E starts at 10.0 and decreases based on errors in performance. Long gone are the perfect tens of Nadia Comaneci. World-class performers are typically in the 15.5 to 15.9 range. Anything above 16.0 is an exceptional score. Each judge provides a pair of numbers with three significant digits. None of these values is very accurate or precise. They are not real measurements; just subjective judgments based on a complex set of tables of difficulty and execution values. Different judges will give different scores for the same performance, and the same judge will give different scores for virtually identical performances at different times. When the performances were over, U.S. gymnast Alexandra Raisman and Russian Aliya Mustafina had the same score: 59.566. The judges went to a tiebreaker based on individual events, which awarded the bronze medal to the Russian. My problem is not with the tiebreaker, but with basing anything on a number with five significant digits based on a series of inaccurate numbers with only three significant digits at most.
Ever watch those CSI-like shows where they zoom in on a license plate or face from a grainy ATM camera. That is a similar kind of “precision enhancement.” If you start with a low resolution picture, you may be able to clean it up a little, but those extra pixels with the details just do not exist.
When it comes to presentation, most people cannot make much sense out of a huge table of numbers. To make the hidden significance clearer, most people use statistics and resulting graphs and charts. Ideally, these results actually give important clues to what the numbers are really saying. However, “statistics” can be something very different. Mark Twain is usually credited with originating the phrase “liars, damn liars and statisticians.”
The actual Twain quote is “There are three kinds of lies: lies, damned lies, and statistics” from his 1906 “Chapters from My Autobiography published in the North American Review. Twain himself attributed it to British Prime Minister Benjamin Disraeli. However, the phrase was never used in any of Disraeli’s surviving writings and the earliest known use of the phrase was years after Disraeli’s death. There are a few uses of the phrase or something very close to it in the period 1885-1891 in both England and the United States. Samuel Clemens (Mark Twain’s real name) was born in 1835 and had starting writing for newspapers by the mid 1860s, and published his first book in 1869, The Innocents Abroad, or The New Pilgrims’ Progress. He could certainly have heard the phrase, or even originated it. After all, he had been in England in 1872.
As a group, politicians are very good at taking facts and turning them into “statistics” that support their particular viewpoint. I quoted “statistics” because statistics is a science: the study of the collection, organization, analysis, interpretation and presentation of data. When done correctly, statistics are extremely valuable at summarizing a lot of numbers into something that is easy to grasp and understand. They can show how two or more sets of facts are related, and can indicate the degree of correlation between two sets of data. This is often not the “statistics” used.
Statistics were a prime driver in the war against cigarettes after they showed a clear correlation between smoking and lung cancer. Note that the statistics did not indicate whether or not there was any cause and effect; or, if there was which was the cause and which was the effect. In the case of cigarettes and cancer, it was fairly obvious that getting cancer did not cause one to smoke. But there could have been an unstudied third factor that caused both. There probably is not a third factor in this particular case, but at least think about the possibility when some politician or advertisement uses “statistics” to “prove” cause and effect.
Another favorite trick is to hide exactly what is being measured. A pharmacy company can measure their drug against doing nothing, the placebo, and claim that it is 85% effective. Their competitor’s drug may also be 85% effective, but that was not what they were measuring. Or they may deliberately select and compare different classes of people in the study to skew the results in their favor. In general, if others cannot get hold of the raw data and perform an independent statistical analysis, then beware. Governments almost always hide the raw data.
Even if you never let facts get in the way of a good story, it is a good idea to know what the numbers really say. If nothing else, it makes it harder to be blindsided by someone who really knows.
In reality, most people and organizations try to use statistics responsibly and usually succeed. But when you get two entirely different sets of “statistics” about the same question, then at least one of them is cooking the numbers to suit their own purpose. My advice: know who is putting out the information and determine what they gain by what they are showing. Beware of surveys of “public opinion” unless they are by reputable pollsters and have provided their selection and measurement details. Lots of these “public opinion” polls are based on self-selected respondents. I.e., they are the people who called into a specific radio show or were customers at a specific chain of stores. Not necessarily an unbiased group.
All valid survey results should include a “plus or minus x percent” statement indicating the calculated error range for the result. When the plus or minus value is larger than the difference between the compared values, just ignore the poll. “52% of the people surveyed prefer my brand, so you should also. Survey plus or minus 5%.” Five percent is bigger than the 4% difference between those who prefer my brand and the 48% who do not.
The last word:
Samuel Clemens took his most famous pen name from his work on Mississippi Riverboats. Since the river constantly changed it was important to know how deep the water was right here right now. Depth ranging by sonar was a little in the future, so the method was to tie a heavy weight to the end of a rope and throw the weight overboard while holding on to the other end and taking up any slack. Simply measuring the length of the rope in the water gave you a good approximation of the water depth. Mississippi River sailors would usually tie a knot in the rope every width of their outstretched arms, about six feet. This unit was a “fathom” and riverboats needed twelve feet of water to safely navigate. When they threw the sounding line in they would announce the depth of the water by the number of knots, or marks, in the water. Even sailors read the Bible, and it often uses “twain” for the number two. So a sailor would shout out “by the mark twain” meaning it was safe, at the moment.
The weight at the end of the rope was often made of lead and was always called a “lead” even if it was just a big rock. Potentially, the origin of the phrase “get the lead out?”
Keep your sense of humor.