Let's say you figured out a way to predict a coin toss. It's not a sure thing, but you think you can call the toss with 82% accuracy. Naturally you're excited about this, so you tell everybody.
The neighborhood bully gets wind of this, and confronts you. "That's stupid," he says. "Nobody can call a coin toss. It's fifty-fifty for everybody."
So you offer the bully a bet: you'll try to call one coin toss. If you call it right, you get $1000, if you're wrong, the money goes to him.
Wait, what? How does that decide anything?
Nate Silver is a brilliant statistician, and I think his call of the election is on the money. But his bet with Joe Scarborough solves nothing and confuses his point. Silver can be wrong and still win the bet with 50% probability, which Scarborough would be sure to point out. Worse, Silver can be completely right and still lose the bet with 18% probability. That's around one in six -- or the odds of rolling a one on a fair die. Not something I'd bet my reputation on.
If Silver wants to bet, surely he can craft a bet that he is sure to win if he is right, and sure to lose if he is not. This should be a case study in decision theory.
The neighborhood bully gets wind of this, and confronts you. "That's stupid," he says. "Nobody can call a coin toss. It's fifty-fifty for everybody."
So you offer the bully a bet: you'll try to call one coin toss. If you call it right, you get $1000, if you're wrong, the money goes to him.
Wait, what? How does that decide anything?
Nate Silver is a brilliant statistician, and I think his call of the election is on the money. But his bet with Joe Scarborough solves nothing and confuses his point. Silver can be wrong and still win the bet with 50% probability, which Scarborough would be sure to point out. Worse, Silver can be completely right and still lose the bet with 18% probability. That's around one in six -- or the odds of rolling a one on a fair die. Not something I'd bet my reputation on.
If Silver wants to bet, surely he can craft a bet that he is sure to win if he is right, and sure to lose if he is not. This should be a case study in decision theory.
4 comments:
The probabilities matter a lot more when you have repeated trials. One trial every four years isn't much. For the same reason, if it turns that Romney wins, it doesn't mean Silver was wrong.
Exactly. Which is why Silver should have found a different way to bet. He's got reams of polling predictions on which he's calling the race; surely he could craft a bet on those predictions that he'd win with probability close to 1 if he's right. (It's not the kind of bet you could describe in a Twitter post, though.)
It occurred tome on re-reading that I was repeating your thoughts. So what you are saying is that Silver should construct a significance test to decide whether to accept or reject the hypothesis either his methodology or at least the prediction based on his methodology that Obama has rughly an 80% chance of victory. Presumably a statistic based on state by state results generates a better test than a test based on the aggregate, which is what Silver's bet represents. Despite being semi-obsessed about the election, I still don't know Silver's exact method. Is it documented at 538? (I realize this reveals my laziness since i could go to 538 and check.) Still, in the spirit of idle speculation, what I wonder about is whether in his simulation is whether each state is a bernoulli trial, independent of other states, or whehter he has some interstate correlation.
That's right ... I don't know enough about Silver's methodology, but I would hope he takes inter-state correlations into account. Anyway, we'll find out today.
Post a Comment