Earlier this year, the NFL set a record for most accepted penalties through week 3. There has been speculation that this increase in penalties is due to fewer full-contact practices as mandated in the CBA, and further speculation that the refs are idiots that can’t manage a game clock and have no idea what a catch is.

I know that I’ve done my fair share of bitching about officiating this season, but I also recall complaining about calls every other season. This led me to wonder, is this year really any different? I know, I can use the power of STATISTICS!!

First, a round of applause (or applesauce if you prefer) and acknowledgment to the fine folk at NFL Savant for compiling play by play data and providing it free of charge to anyone with the skills to use a mouse.

This post will compare penalties from weeks 1-8 of the 2014 season to penalties from weeks 1-8 of this season. Rulings as to what constitutes a catch, or the refs attempts to travel through time using the game clock as a Tardis are not covered. This is ALL ABOUT THE FLAGS BABY!

Now, ONWARDS! TO THE DATA!

At first things don’t look so hot. With fewer games and fewer plays than last year, there are already 46 more penalties. People frequently look at the number of penalties per game, but imo that is a dumb thing to do. Penalties are assessed on a per-play basis. During (or after) any given play, a player either does, or does not get called for a penalty. Therefore it makes more sense to look at penalties per play, or the percentage chance of a penalty being called on (or after) any particular play. Since games do not all have the same number of plays, looking at penalties per game is not an apples to apples comparison.

Looking at penalties per play evens things out a bit. There are more plays per game this year. This could be due to more teams using a hurry-up offense, but it could also be due to more teams repeating downs due to penalties. Whatever the cause of the increase, more plays gives more opportunities for penalties to be called. (I should note that a play is listed twice in the data if two penalties are called on the same play. This could bias the data, but given the infrequent nature of multi-penalty plays I’m ignoring it for now.) Looking at the penalties per play number, those look pretty close. Given the large number of plays, is there a significant difference between the two numbers? For the craps playing degenerates here, the chance of a penalty being called on a play is about the same as making a 6 or 8 hardway.

The standard tool for comparing summary statistics is called a t-test. Technically, it’s called the Student’s t-test, named after a guy who wrote under the pseudonym “A Student.” It’s a way to test simple hypothesis about the data. Here the hypothesis that we are testing is that the difference between the 2014 value of penalties per play and the 2015 value is 0.

Running a Welch Two Sample t-test on the penalty data for each year gives a p-value of 0.3899, which basically means we can’t reject the idea that the probability of a penalty is the same in both years. That there is weaselly statistics talk for “Same old shit, this year and last.” In general, people look for a 95% confidence level when making comparisons like this. That p-value gives the confidence level, though backwards from what you might think, we would want to see a p-value of <0.05 before considering the difference to be statistically significant.

“But Zymm! It’s not just about the overall number of penalties, what about penalty yardage? What about the types of penalties?”

Excellent points Other Zymm! Let’s take a deeper look, shall we?

Previously, I was only looking at whether a penalty was called, without considering whether or not it was accepted. When looking at yardage, we limit ourselves to looking only at accepted penalties, as no penalty yards are assessed if the penalty is declined. It turns out this doesn’t really matter, as penalties were declined at basically the same rate both years. At this point last year, 13522 penalty yards were assessed for an average of 8.034462 yards per penalty. So far this year there have been 14254 penalty yards assessed for an average of 8.229792 yards per penalty.

Due to the way penalty yards are assigned, it’s a little more difficult to compare yards/penalty year over year. Instead, I looked at the portion of penalties over 5 yards, over 10 yards, and over 15 yards (so basically pass interference calls only). There was no significant difference between any of these groups. This kinda makes me think that there won’t be a huge difference in the types of calls either. Have I been going “WTF IS UP WITH ALL THESE OFFENSIVE PI CALLS!?” unnecessarily all year?

There are some data issues here, the main one being the “Personal Foul” category in 2014. We can probably assume these are all unnecessary roughness calls. There are also a fair number of penalties that are only called a handful of times, which we can’t really do much with, so while it’s antecdotally interesting that there have been almost 4x as many “Player out of bounds on the punt” calls this year, there’s not really much we can say about that.

Let’s look at some of the more common calls. It’ll probably surprise no one that the most common call is offensive holding. Just eyeballing it, it appears there are quite a few more offensive holding calls this year, oddly enough, counterbalanced by fewer defensive holding calls. Surprisingly enough, the difference in offensive holding isn’t significant, but the defensive holding difference is! So, our first significant result is that the refs are calling defensive holding less frequently than they did last year. Why? Who the hell knows, I don’t have a theory on that one.

There’s no significant difference in OPI or PI calls, which is probably good news, since these particular calls usually have a pretty large impact on a drive.

There are really only two other penalties that show a significant difference from last year. Defensive offside calls have significantly increased this year, so maybe there’s something to all that “Aaron Rodgers is a genius with his hard count, blah blah blah” stuff, though I’m too lazy to actually go through and look at offside by team. The other one is the decrease in Illegal Use of Hands calls. Feel free to speculate on the reason for that one.

There’s no significant change in Roughing the Passer penalties, so despite all those “OMG, he touched the QB’s helmet, throw ALL THE FLAGS” calls, they were doing that last year too.

Last but not least, a quick break down of when penalties are called, by quarter and down.

The main thing I find interesting here is the data for the second quarter. There are significantly more penalties called in the second quarter than any other quarter. My interpretation here, the second quarter is frequently the most competitive part of the game. It’s rare that a team is totally out of it by the half, but the urge to keep the score close going into half time might lead to more bending of the rules. The same patter doesn’t emerge in the 4th quarter due to garbage time.

I’m not really going to go into the down data. The large number for 1st down is a bit misleading, as there are more 1st downs than 2nd, 3rd and 4th.

So this seems long enough already. To conclude, officiating is pretty much as annoying this year as last. If you want to feel smarter than your friends, complain loudly next time there’s an offside call.