Saturday, July 19, 2008

Football and IQ: The Wonderlic Test

The closer you are to the ball, the higher your score.




The Wonderlic test is one of the most unique features of the NFL Combine. As I indicated in my last post (and as Michael Lewis suggested in The Blind Side), offensive tackles score the highest of any position on the field. Indeed, IQ tests are certainly not unique to football, but are an increasingly common feature of job applications for every sort of job, and the Wonderlic test itself is used in many professions. But its use in football strikes many as odd, if for no other reason than when it is given:
In another interesting twist, the test is also administered to players the day of the NFL combine—which means they first spend the day running, jumping, benching, interviewing, and lots of other -ings, before they sit down and take an intelligence test. It’s a bit like a medical student running a half marathon before taking the boards.

Hat tip Ben Fry. The questions themselves increase in difficulty over the course of the test:

The first questions on the test are easy, but they get harder and harder.
An easy question: In the following set of words, which word is different from the others? 1) copper, 2) nickel, 3) aluminum, 4) wood, 5) bronze.

A tougher one: A rectangular bin, completely filled, holds 640 cubic feet of grain. If the bin is 8 feet wide and 10 feet long, how deep is it?

If you'd like to try your hand at some sample questions, do so here.

Yet the question remains whether such IQ test results have any relevance to football. The popularity of the test suggests that it does, and much of this site is dedicated to the idea that, as they move up the ranks, players must synthesize and process lots of concepts and data in very short periods of time. But is an IQ test the best measure? Malcolm Gladwell suggests otherwise:
Picking a subject from his upcoming book, Malcolm Gladwell talked about the difficulty in hiring people in the increasingly complex thought-based contemporary workplace. Specifically that we're using a collection of antiquated tools to evaluate potential employees, creating what he calls "mismatch problems" in the workplace, when the critera for evaluating job candidates is out of step with the demands of the job.

To illustrate his point, Gladwell talked about sports combines, events that professional sports leagues hold for scouts to evaluate potential draftees based on a battery of physical, psychological, and intelligence tests. What he found, a result that echoes what Michael Lewis talks about in Moneyball, is that sports combines are a poor way to determine how well an athlete will eventually perform as a member of their eventual team. One striking example he gave is the intelligence test they give to NFL quarterbacks. Two of the test's all-time worst performers were Dan Marino and Terry Bradshaw, Hall of Famers both.

Yet two anomalies do not disprove the notion. Charlie Wonderlic believes that "What the score does is help match training methods with a player's ability, [like the ability to understand] a playbook. [O]n the field, the higher the IQ, the greater the ability to understand and handle contingencies and make sound decisions on the fly."

I don't have a firm answer either way. I think there's nothing wrong with giving the test, and different teams appear to put varying degrees of emphasis on the test results. Some only care if a player scores extremely low or extremely high, while others take the test quite seriously. The image above clearly indicates it matters more or less depending on the position. Advanced NFL Stats has previously discussed studies that attempted to chart out QB performance as a function of their Wonderlic results. My guess is that the Wonderlic is a weak predictor in the same sense as the 40-yard dash, shuttle run, and the bench press: If you chart out performance with those combine statistics, although you will see a positive trend, it will be full of noise, will not give you predictions with high certainty, and counterexamples - like Dan Marino - will be abundant.

5 comments:

  1. Chris: Anomalies explained - Bradshaw may not actually be dumber than dirt, but he and Marino are probably the least-disciplined modern-era players in the HOF. Good old-fashioned laziness goes a long way toward explaining results like theirs...

    ReplyDelete
  2. So is your argument that they both were too lazy to put in five minutes of effort with their draft status and financial future in mind? While I'm sure some get lazy and undisciplined on tests like this, the Wonderlic is only a five minute test and I do not think it can explain more than a point or two on Marino's and Bradshaw's tests.

    ReplyDelete
  3. Chris,

    I think you point remains but, the Wonderlic test is 12 minutes.

    ReplyDelete
  4. "A tougher one: A rectangular bin, completely filled, holds 640 cubic feet of grain. If the bin is 8 feet wide and 10 feet long, how deep is it?"


    Just another example of how Nebraska players get an unfair advantage in subject matter.
    (/Boomer Sooner)

    What does it take to keep a good team down in the Big 12 these days?

    ReplyDelete
  5. "Joe C said...
    Chris,

    I think you point remains but, the Wonderlic test is 12 minutes."


    Why not a BIG TEN minutes?
    (/Ohio)
    Or even a PAC TEN minutes?
    (/Carrol?)

    In all honesty, the person most likely to use these tests are agents. It might determine how marketable players are in some respects.

    ReplyDelete