(CNN) Question wording is a crucial element of reading and dissecting a poll. The wording of each question and the order they're asked in are extremely important factors in how respondents consider questions, and seriously impact the results of a poll. One poll released earlier this week is a great example of such a case.
In a poll conducted by Suffolk University and sponsored by USA Today, half of registered voters appeared to agree that the Mueller investigation is a witch hunt. But there is way more to this result than meets the eye.
The exact question was: "President Trump has called the Special Counsel's investigation a 'witch hunt' and said he's been subjected to more investigations than previous presidents because of politics. Do you agree?"
The results showed 50% of voters said yes, while 47% of respondents said no and 3% were undecided.
The result is an outlier among other polls that asked similar questions and didn't find nearly as many respondents agreeing that Mueller's investigation is a "witch hunt." It was a result that Trump immediately seized on -- he tweeted the poll on Monday morning, touting its results as good for him and his case against the investigation.
"Wow! A Suffolk/USA Today Poll, just out, states, '50% of Americans AGREE that Robert Mueller's investigation is a Witch Hunt.' @MSNBC Very few think it is legit! We will soon find out?" Trump said.
The question presents one side of the argument (the President's side), but not the other. It's important that poll questions are balanced to reflect both sides of the issue or are as neutral as possible.
Here's a few things to keep in mind when reading a poll:
The actual conversation about the medium (phone, online, etc.) in which you should conduct a poll is extremely long (way more than we can get into), but it's important to pay attention in understanding their relevance and importance.
This particular Suffolk/USA Today poll was conducted well, surveying 1,000 registered voters over five days with live-person interviews over the phone (using both cell phone and landlines).
But presenting both sides of the argument stops people from seeing just what the pollster is presenting and more accurately gauges their opinions.
For example, CNN's recent polling doesn't find the same result as the Suffolk/USA Today poll, but instead shows an increasing approval of Mueller. According to a CNN poll conducted by SSRS and released on Wednesday, his approval rating is now at 48%, with 37% disapproving of the job the former FBI director is doing. That's a shift from near-even divides on the same question in February and December.
That same CNN poll found 56% of Americans said the investigation into Russian efforts to influence the 2016 presidential election is "a serious matter that should be fully investigated" while 38% said "it's mainly an effort to discredit Donald Trump's presidency."
Other polls show similar approval of the job Mueller is doing.
A Quinnipiac poll in early March asked a similar question, "As you may know, Special Counsel Robert Mueller was appointed to oversee the criminal investigation into any links or coordination between President Trump's campaign and the Russian government. Do you think that he is conducting a fair investigation into this matter, or not?"
Over half (54%) of registered voters said they think Mueller is conducting a fair investigation and only 27% thought he was not, almost a complete reversal from what Suffolk found with a differently phrased question.
A CBS News poll in January asked whether the investigation into dealings between Trump associates and Russia was "justified, or is the investigation politically motivated?" This one was slightly more split, but half of Americans said it was justified and 45% said it was politically motivated. Politically motivated is different from "witch hunt." Since Trump himself uses the phrase "witch hunt" in an attempt to discredit the investigation, it has become politically motivated rhetoric, something that when people hear it in a poll question, they associate it with the President.
To people steeped in polling methodology, the fatal flaw in the Suffolk/USA Today poll came in question wording, where other polls showed hugely different results.
Ken Winneg, managing director of survey research at Annenberg Public Policy Center of the University of Pennsylvania, also took issue with the Suffolk/USA Today poll, especially when it came to the response options.
"It was biased toward agreeing with whatever statement the respondent heard. Survey research questions using scales should be balanced always," Winneg said in an email. "In the USA Today/Suffolk case respondents were asked if they 'agree' with the statement. They weren't provided the same stated option to 'disagree.' Double-barrelled wording issues aside, respondents should have been asked: 'Do you agree or disagree?' Without the balance the bias is toward agreeing with the statement."
David Paleologos, the director of the Suffolk University Political Research Center, stood by the way the question was phrased. He told CNN in an email that Suffolk has seen a decrease in trust in Mueller through its polling, and an increase in those who believe Trump.
"As for the specific poll question ... we thought it important to use the President's name, his own words and stated viewpoint on the Mueller investigation, and ask if respondents agreed or disagreed," Paleologos told CNN. "(L)ike other questions in the poll related to the Mueller investigation, I believe the answers do indicate a shift in public opinion."
Like any large-sample poll, conducted over the phone with a live person asking questions, the Suffolk/USA Today effort was both costly and informative. But it just might not show exactly what the President hopes it does.
Since we're entering the 2020 season in earnest, this is something to keep in mind. As pollsters do their work and pundits dissect it, news consumers can look a little more closely to see how the work was done and if it really means what people say it does.