Perhaps the greatest professional failure of the past ten years came when pollsters almost unanimously assured America that Hillary Clinton would win the presidential election by a landslide. The profession suffered a terrible blow to its credibility, but that doesn’t mean news outlets won’t inundate us with more numbers and more statistics. It does highlight the fact that Christians must be discerning in accepting specific pieces fo information.
The media often portrays polls and stats to as “facts,” but they can be “hard facts” or “soft facts.” Saying that 45 Senators voted for a bill while 55 voted against it is a hard fact because it is objectively verifiable. However, saying that 6 out of 10 people take a particular stance is a “soft fact” because the same poll could yield different results if the pollsters asked the question differently or asked different people. How do we make sense of all the numbers? Here are four questions to ask.
- Who conducted the study?
Knowing what group commissioned or conducted the poll may give you a good idea of what they are trying to prove. If a group associated with a particular political ideology wants to make a point they can conduct a poll and design it to show what they wish. Not all polls and statistics are like this, but the source is a driving force that can shape everything else.
- How were questions framed?
When polling individuals, the most powerful weapon that a pollster has is the question. How they frame a question can dramatically alter the outcome of the poll. For example, people may answer the question, “do you oppose same-sex marriage” differently than if the question were asked more broadly, “do you support equality for all people.” Even if the options mean the same thing, people’s responses can differ.
- Who did they ask?
Most nationwide polls have a general set of standards to try and get a random sampling. However, the move away from landline phones has shifted the broad cross-section upon which many polls rely. As an example of what can go wrong, Alfred Kinsey (cite) famously skewed his study on human sexuality by including the prison population in his statistics. Who you ask matters.
- What does this study prove?
If a headline is promoting the results of a study, ask if the study supports the headline’s assertion. Many studies don’t prove much. Don’t just take the statistics at face value, ask what the statistics show. Then ask what it is being claimed to demonstrate. Statistics and polls are pieces of data that need interpretation, and we must ask whether that interpretation is valid.
Christians recognize that anytime someone presents a piece of data they are also presenting a particular interpretation. We must take steps to ask whether that data is reliable or not. Additionally, the Scripture tells us that polls, studies, and numbers cannot be the final source of truth for the Christian life. They may provide insight into our culture and the world around us, but our ultimate source of truth and morality is found solely in the Word of God.