Comey and McCabe audits: How likely is it a coincidence?

New York times Reported That the Internal Revenue Service gave one of the strictest types of audits to James B. Comey, former director of the FBI, and former deputy director Andrew G.

This has raised a lot of perfectly reasonable questions, most of which are variable: What are the possibilities? As the article pointed out, the chances of two high-ranking political enemies of President Donald J. Trump reviewing by chance are very slim.

But minuscule is not zero.

If we are to believe that this was a coincidence, how highly improbable can we say it is? Here, we try to estimate this possibility as seriously as possible.

First, the facts: Both men were selected for audits under the National Research Program (NRP), a small subset of all audits that the IRS conducts each year. These audits examine a sample of returns to collect data on tax compliance.

According to the IRS, there were about 5,000 such audits in 2017, 4,000 in 2018, and 8,000 in 2019 — chosen from about 154 million individual tax returns each year. Mr. Comey’s audit of his 2017 tax return was; McCabe was back for 2019.

Many aspects of the NRP complicate our calculations, including the sampling methodology of the IRS auditors and the different years of the audits themselves. We will return to these issues later. For now, we’ll assume that all taxpayers have an equal chance of being audited and that both men were audited in 2017.

If this problem appeared in a textbook on probability, you might read as follows:

If there are 154 million marbles (the approximate number of tax returns filed each year) in a giant urn, and a few are red (the ones representing Mr. Comey and Mr. McCabe among them), what are your chances? Will you draw two or more red balls if you randomly draw a few thousand from the jar (the number of revisions in that year)?

It may sound complicated, but it is A relatively well thought out problem, which is something many math or statistics majors may encounter in college coursework. People have already derived equations to estimate these probabilities, with names like hypergeometric distributionwhich has applications such as election auditing and card counting.

We can simply enter our estimates for the number of total balls, the number of red balls and the number of draws, and we’ll get a probability. If we think that there are only two red balls – that is, if we limit the exercise to Just Mr. McCabe and Mr. Comey – This equation yields a probability of 1 in 950 million.

These are much greater odds than your chances of winning a Powerball game. It’s also an almost meaningless result. At best, this is the correct answer to the wrong question.

To understand why requires acknowledging the inherent absurdity of our exercise: to best estimate the probability of an improbable event, we must put aside the fact that we already know it did. (The probability of this happening is 100%).

Jordan Ellenbergthe professor at the University of Wisconsin who has written books on mathematics and thinking, described it this way: “In an unreal world, what is the probability of this thing happening, which has already happened in our universe?”

It may sound strange, but the same problems appear even in basic probability exercises like flipping a coin.

If you flip a coin 20 times in a row, the specific sequence of heads and tails is unusually rare, about one in a million, but it did happen. And the some The sequence of fluctuations will always occur. It’s only a surprising coincidence if this is the sequence you selected before flipping.

In the same way, it would be a mistake to narrow the search to only Mr. Comey and Mr. McCabe, because it is likely that we would examine these possibilities if we knew that two else Notable political enemies of the administration were scrutinized instead of these two men.

A better question is: What is the probability that there are two or more people Such as Mr. Comey and Mr. McCabe, will they be subject to scrutiny during this period?

Should this group of people include any two senior FBI officials? Which two senior officials in the Ministry of Justice? It is this framing—a subjective decision rather than a factual one—that often drives any probability estimate, more than any choice of statistical distribution or sampling weights.

Below is a graph of the probability that our equation will yield different choices for the number of red balls, ranging from two (Mr. Comey, Mr. McCabe and no one else) to 400 (a conservative estimate of the number of Americans Mr. Trump insulted by name on twitter since the beginning of his candidacy for the presidency).

The probability increases exponentially with the choice of who should be considered a red marble alongside Mr. Comey and Mr. McCabe.

The point is not to decide on a number but to realize that our choice of group size is what drives us into the answer. Although some guesses are certainly better than others, many of the options are defensible.

detail processing

Now let’s try to narrow it down to something a bit more realistic, and go back to some of the things we’ve overlooked in our simple explanation of this issue.

First, the two men were not audited for the same year. By expanding our scope to cover the three-year period from 2017 to 2019, our resulting odds increase exponentially. This is straightforward: if a person has a certain opportunity to revise in a particular year, then more years means more opportunities to revise.

Second, we are only interested in the possibility of choosing at least two people. We will not consider the probability that the same person will be selected twice; It seems unlikely given that the audits could stretch more than a year, according to Mr Comey’s account. Note that we’re looking at the probability of choosing at least two people, not specifically two, because it would also matter if three or more people were chosen from a group.

Finally, the IRS doesn’t really pick people at random. Instead, the agency tends to pick some types of taxpayers, including high income people, more than others. For tax year 2001, an NRP . form is included yields of people around the 90th percentile of income at about 1.7 times the rate one would expect, as returns were chosen independently of earnings. This rate rose through the highest-earning ranks, with people with incomes in the top 0.5 percent likely to be in the sample ten times more than those closest to the median income.

We can probably assume that any group of Mr. Trump’s enemies would gain more than a random sample of Americans. But we cannot realistically estimate the full income of everyone in our group each year. We also know that the IRS has considered other factors in sampling, such as Payout type This taxpayer file, and that sampling methods can change from year to year. This leaves us with little guidance on how to match the IRS’s methods. As such, we’ll leave our estimates unweighted by income. As an exercise in the back of the envelope, if you are concerned about how income will affect these outcomes, you can multiply the resulting probability if you think the group members have very high earnings, and multiply it by 10 if you think they are extraordinarily wealthy.

Put them all together

Incorporating these options, the table below provides some estimated possibilities depending on the size of the group being considered.

Alternatively, if our options are not satisfactory, we have created a simple calculator for you to create your own odds:

So what is the “correct” estimate?

Most realistic output of this equation can be accurately described as “very rare” or even “extremely rare”, yet there is no evidence of error.

“It’s a bit like an irresistible force and an immovable thing,” he said. Andrew Gilman, a professor of statistics and political science at Columbia University, when told in the abstract about this exercise. “On the one hand, you say that it is completely random. On the other hand, you suspect that it is not.”

Mr. Gilman, like any other statistician who spoke with the Times on this issue, said the biggest hurdle was not any of the details but defining the question itself.

When we try to calculate the probability of a certain event Because We suspect it might not be random, we end up in a complicated situation of trying to imagine how we would have predicted the probability of the event Before He said what happened David Spiegelhalter. He leads the Winton Center for Communication on Risk and Evidence at the University of Cambridge, an organization dedicated to improving the way quantitative evidence is used in society.

The math is easy, he said, but the question wording is difficult, close to “nonsense,” due in large part to the difficulty of defining which group we are interested in.

“What is the chance of that happening?” It is an easy statement to make.” “It is a familiar statement. But, in fact, it is very difficult to answer this question.”

Mathematics has its limits. The point of trying to estimate a probability like this, Gilman said, is not to put a lot of stock in the numbers, but to let the result push you to learn more.

In this case, the best question is not the answer that you can look for in a textbook of statistics.

Instead, Mr. Gilman said, the question to ask is: “What’s going on?”

Matthew Collin Contribute to the preparation of reports.