Tuesday, March 31, 2009

Speculations on play-calling on first, second, and third and ten

From Advanced NFL Stats:

All the numbers that follow are from all 10-yards-to-go scrimmage plays in the first 3 quarters of regular season games from 2000-2007. The only other limitation was that the game score was within 10 points. I wanted to exclude situations when teams exercised an abundance of either risk or caution.

Note the percentage of play types called on 1st, 2nd, and 3rd downs (with 10 yards to go). There is a fairly even split between run and pass calls on 1st and 2nd downs. On 3rd and 10, the a pass is far more expected.

% of Play Types by Down, 10 Yds To Go

Type - 1st - 2nd - 3rd - Total
Pass - 47.2 - 52.7 - 91.1 - 49.6
Run - 52.8 - 47.3 - 8.9 - 50.4

Although 91.1% isn't 100%, it's close to where the anchor point on the lower right side of the game theory graph--almost the pure pass vs. pass defense strategy combination. Now let's look at the average outcomes for these situations.

Yds Per Attempt by Down, 10 Yds To Go

Type - 1st - 2nd - 3rd - Total
Pass - 7.0 - 6.3 - 6.5 - 6.9
Run - 4.2 - 4.4 - 6.9 - 4.3
Total - 5.5 - 5.4 - 6.5 - 5.6


When passing is most predictable, it yields half a yard less than on first down, when it is less expected. Conversely, running is most successful when it is least expected.

At this point, I should point out that passing on 3rd and 10 yields slightly more yards than on 2nd and 10, which isn't completely what we'd expect. This is almost certainly because defenses will allow short complete passes on 3rd down in exchange for being relatively assured to be able to stop the gain short of 10 yards. This is part of the problem posed by the fact that yards does not equal utility. We'll have to dig a little deeper. The next table lists interception rate by down.

Interception % by Down, 10 Yds To Go

Int Rate
1st - 2nd - 3rd - Total
2.6 - 2.9 - 3.5 - 2.7

Now we see more what we'd expect--a slight increase from 1st to 2nd down, then a large jump on 3rd down, in accordance with the associated increases in passing predictability. The next table lists adjusted yards per attempt, which is YPA with a -45 yd adjustment for every interception thrown. Adj YPA, however, still exhibits the same problem as plain YPA. It underestimates the drop off from 1st to 3rd down in passing effectiveness because defenses will allow gains, as long as they're not more than 9 yards.

Adj Yds Per Attempt by Down, 10 Yds To Go

Type - 1st - 2nd - 3rd - Total
Pass - 5.9 - 5.0 - 4.9 - 5.6
Run - 4.2 - 4.4 - 6.9 - 4.3


So what we can say is, the reduction in passing effectiveness due to predictability is likely at least 1 full adjusted yard per attempt. The drop from 1st down to 2nd was 0.9 yards, so the true reduction in effectiveness from 1st to 3rd down may be far larger.

Except that there's a problem with this analysis. There's a bias in the data. Which teams are more likely to face a lot of 2nd and 10s and 3rd and 10s? The ones that stink at passing. So the 2nd and 3rd down numbers are lower than would be representative of the league as a whole. In other words, poor passing teams 'get more votes' in the analysis.


All this is intended to tee up a game-theory analysis for finding some kind of ballpark run/pass equilibrium. Do read the whole thing.

But a few brief thoughts:

  • The adjusted final numbers intrigue me, particularly second down as compared to first. (As Brian notes, third down is tougher to break down since it's really a binary question of conversion versus failure.) But I'm struck that on second down the yards per pass attempt drops by nearly a full yard while the yards per run goes up only .2: why does the defense get so much better on second down? Is the data skewed to losers? Is play-calling worse on second down?

  • In that vein, I wonder if the old conventional wisdom about "getting back half on second and ten" works against the offense. On first down the passing plays are likely to involve play-action as well as quick or intermediate passes -- coaches can use their full asrsenal; maybe on second coaches are too concerned with screens and quicks -- trying to just get half -- that they give up too much in the way of expected points?

  • But on the other hand, what if they get this 5.0 yards per pass attempt on second and ten with more certainty and less variance than the 5.9 on first down. If so, then possibly the offense is in better position to convert third down than they would be even with a greater expected play value that carried more variance. Could cut either way; football is complicated.

Hopefully Brian can shed some light as his series develops. I look forward to it.

6 comments:

  1. It has always been my philosophy, that ,if I was faced with 2nd and 10, that after the 2nd down play I wanted to face at worst 3rd and 10. I want to get some kind of positive yards, but I absolutely want to make sure that I don't lose yards on the play. I'm three step dropping on a pass with usually 1 read or a fade. If it's not there , let's throw it away and try 3rd and 10. Most times we call a run.

    To tell you the truth I hate 2nd and 10. Play calling that down always brings stress.

    Enjoy the site. Good job.

    ReplyDelete
  2. "But I'm struck that on second down the yards per pass attempt drops by nearly a full yard while the yards per run goes up only .2: why does the defense get so much better on second down? Is the data skewed to losers? Is play-calling worse on second down"

    I think the skewed for bad teams is almost certainly a factor. Bad teams are the ones that end up in second and 10 the most.

    ReplyDelete
  3. So could you do a sample where you take ten-fifteen different examples of the X-10 situations from each team, and run that same information back?

    ReplyDelete
  4. I think you are right that the average is part of the issue of the gain on 2d and ten. And a sample which overweights bad teams. You also have to factor that defensive personnel packages and playcalling is a lot easier on 2d and 10.

    Averages are really deceptive in football. We'd win a lot more games with an offense that made 4 yards every play than one which averaged 8 per play.

    stan

    ReplyDelete
  5. Stan, I disagree with that last point. Averages can be misleading but they also dominate. I think it's be incredibly difficult to find a risk profile where an 8 yard per play was inferior to a 4 yard per play, even where the 4 was guaranteed. Now, maybe -- just maybe -- you could do it for 4 to 4.5 or maybe 5, but not 8.

    ReplyDelete
  6. I have to agree with stan in theory. Guaranteed 4 yards simply means you always score, you never fail to convert to a 1st down and your defense gets plenty of rest!

    Of course it is quite simply impossible, but I think stan is exaggerating to elaborate.

    PAUL.

    ReplyDelete