I’ve stumbled today upon this old tweet of Madeka: https://twitter.com/DhruvMadeka/status/1266188053510533122

“You want to be comparing two tradeable instruments – if there’s a divergence/arbitrage the market corrects for that. Comparing a tradeable one to a non-tradeable one then saying the market is arbitrageable isnt clear to me”

That’s funny, because their argument seems to be based on arbitrage-free pricing of election forecasts as derivatives of a hypothetical, non-tradable underlying instrument.

]]>„I’m no quite sure what is the definition of a black swan event, but I don’t think that “old guy gets sick” is one!“

This aged quite well

]]>Since 2020 economic conditions present an “unusual difficulty” is it possible to produce results based on low, middle, and high values for treating the covid recession?

The unrest in Kenosha since the August 23 shooting of Jacob Blake has dimmed Biden’s campaign according to the betting odds. According to the RealClearPolitics betting odds index, Biden’s had a 12 point advantage over Trump on August 23. It is now only 2 points. https://www.realclearpolitics.com/elections/betting_odds/2020_president/

Can the model capture the anger and anxiety of voters over unrest that many blame Democratic Party rhetoric for kindling? Your colleague Mr Morris is highly skeptical that the unrest in Kenosha will benefit Trump (below). I highly disagree.

“I don’t think we can look at trends in polls this summer, which included numerous news cycles about BLM protests & abt violence/looting/unrest, and conclude that THIS TIME it’ll be different and Trump will get a bounce. Our prior should be that it won’t matter to the horse race.

10:52 PM · Aug 30, 2020”

Kai:

It’s possible to play around with the model can get just about anything. But for both theoretical and empirical reasons, I think that bad economic conditions are bad for the incumbent, and I think it would be a mistake to remove this information from the prediction. Just as an example, recall that the bad economy in 1980 was in part due to an oil price hike following the Iranian revolution.

]]>“The 2020 election presents an unusual difficulty, because the recession caused by the coronavirus is both far graver than any other post-war downturn, and also more likely to turn into a rapid recovery once lockdowns lift. History provides no guide as to how voters will respond to these extreme and potentially fast-reversing economic conditions. As a result, we have adjusted this economic index to pull values that are unprecedentedly high or low partway towards the limits of the data on which our model was trained. As of June 2020, this means that we are treating the recession caused by the coronavirus pandemic as roughly 40% worse than the Great Recession of 2008-09, rather than two to three times worse.”

Choosing “40% worse than the Great Recession” sounds like guesswork. Without any history as guide, would it be more sound to exclude this variable from the fundamentals model?

]]>John,

Joseph and I tried to resolve this through email unsuccessfully. We’re planning a post-pandemic lunch to discuss. You’re welcome to join! Feel free to email me through my website!

Dhruv

]]>It’s now down to $710!

]]>The ‘rare’ events which we used to think of as ‘rare’ (or merely Manchurian Candidate movie-fare) are no longer rare at all. What was the effect of the then FBI director’s communiques in 2016? What will the analogous thing be this time? I presume there will something of that order; and because I am not a writer of 1st-rate screen plays it is almost beyond my comprehension — whatever it is to be! Something of that order though will come to pass. Mark my words.

]]>Daniel:

Fair point! We don’t really have a large enough sample size at the moment to properly evaluate this, but for what it’s worth, it seems that the polling for the 2018 midterms was also quite accurate (https://www.cnn.com/2018/11/19/politics/2018-midterm-elections-good-year-polls/index.html).

]]>Andrew:

Sorry, that’s my mistake. That sentence should read “the [models] Silver/Gelman are using.” I’m not trying to imply that you’re using the same models. My goal was to point out that the critique laid out by Taleb/Madeka is really better understood as advancing an alternative model, not a proof that volatile forecasts violate the rules of probability theory.

]]>Interesting. I certainly don’t have any expertise in polling. But it’s interesting that 538’s analysis says that proper high probability density intervals for real polls are about 10-15 percentage points wide, and ALWAYS HAVE BEEN.

He then goes on to say that averaging polls helps a lot. And this is where my concern lies: what if there are consistent biases in polling that have gotten worse over time as response rates have gotten worse.

We only have a couple of election cycles to evaluate that kind of bias. I don’t think 538’s analysis addresses this concern. In fact he even mentions that averaging polls doesn’t help when they all go the wrong direction together… but dismisses the issue immediately and goes on to say that averaging is a good thing. Maybe that’s true, and maybe he has the info to prove it, but I didn’t see it skimming that article.

I just don’t think we know a lot about these potential consistent biases because we don’t have a lot of elections to look at in the period post conversion to mostly-cell-phones.

Googling graphs of cell phone adoption and smart phone adoption. I think 2015 or so is an important year: https://www.statista.com/chart/2072/landline-phones-in-the-united-states/

So what polling is like today vs what it was like in 2012 might be very different. And yet there has only been ONE presidential election in that time frame.

]]>John:

Please. There is no “the model Silver/Gelman are using.” I have no idea what Nate is doing.

]]>Joseph:

I think I’m in agreement with everything you’ve stated. And I believe your interpretation of his second set of graphs is correct. If there is some justification for why future volatility is always going to infinity, then his argument is valid. Though again, I would repeat, at this point, Taleb is clearly just stipulating that we should be forecasting with a different model of the world than the model Silver/Gelman are using. It’s not as if Silver also agrees that future unrealized volatility is going to infinity, but nevertheless produces a forecast that is highly variable over time.

As far as I can tell, Taleb 2018 does not make these kinds of arguments regarding bounds. To define the bounded process, he applies the error function to the underlying Brownian motion (see Sec. III). In fact, he explicitly claims that it doesn’t matter if you price the option based on the unbounded process or the bounded transformation of that process (see the last paragraph of Sec. III). Again though, doing something like this supports an argument of the form “I have a different, better model of the world.”

]]>Radford:

As I predicted, the odds are shifting. At Betfair, a $1000 bet on Biden now returns $750, down a bit from the $840 when I wrote the above post a few days ago.

]]>Thanks John, once again that’s fantastically helpful.

He really does seem to be arguing in bad faith. In his response to Clayton, he snarks: “There is no mention of FiveThirtyEight in Taleb (2018),and Clayton must be confusing scientiﬁc papers with Twitter debates. The paper is an attempt at addressing elections in a rigorous manner, not journalistic discussion, and only mentions the 2016 election in one illustrative sentence.”

Yet Taleb (2018) includes a chart with a line labeled “538” that came directly from a Taleb Twitter posting and goes entirely undescribed in Taleb (2018). It’s the chart that Madeka likes and sent to Andew — lefthand chart in this graphic: https://gelmanstatdev.wpengine.com/wp-content/uploads/2020/06/image-1.png

Anyway, all I can think in that response to Clayton is that he’s trying to twist his own argument by positing a _literally_ infinite sigma for future volatility (and not simply a very large sigma). He notes in his response to Clayton: “The σ here is never past realized, only future unrealized volatility.” and “At inﬁnite σ, it becomes all noise, and such a level of noise drowns all signals.”

I can’t claim to fully understand the first comment, but the second I get — if sigma is “truly” infinite, then the potential future movement is such that neither outcome is more probable than the other. And I think that informs his prior comment, “the σ here is never past realized…”, because the flip side of that is that the magnitude of prior actual movement is irrelevant if the potential for future movement is infinited.

But you are correct — for any real value of sigma, it is possible (likely?) that actual, realized prior experience will drive the process far enough from equilibrium to cause a significant shift in probability.

But if I’m reading him wrong, I think the core of the thing is the first sigma sentence I quoted. If there is some justification for the assumption that past, realized volatility is always negligible with respect to future volatility then yes, the argument makes sense. If Taleb can justify that then his argument works; otherwise it would appear that Schmaus has identified correctly and Taleb simply made a coding error (not increasing the volatility of the actual process) and got a result that comported with his intuition, and dumped it out on Twitter.

Also, can you tell what Taleb is doing here? My guess is that he’s showing what the “rigorous updating” model would produce for different values of sigma, with the blue line being low sigma and consistent with what he termed the fivethirtyeight projection and the others corresponding to larger values of sigma. https://twitter.com/nntaleb/status/762655751143886848/photo/1

But again, these all make the error increasing the sigma used as a future expectation for pricing purposes but not increasing the sigma on the underlying process. If he had updated the underlying process, they would all look similar to the blue line (but perhaps not as jagged – I’m not sure).

So the one way I could get to Taleb’s result is if the process were bounded somehow, such that volatility were asymmetric based on prior realized results. For example, take your coin flip model and say that the accumulated dollar value must remain inside the corridor -$100 to +$100. As you increase the volatility, the corridor has greater effect and for very large values of volatility you would approach a 50% probability estimate even if prior realized results have planted you firmly at the extreme of the corridor. It doesn’t feel wrong to assume that such a thing might exist in politics, that a candidate’s situation can only be improved so much regardless of how much “good news” occurs for them, leaving any volatility to act to their detriment (and the reverse for a candidate who has gotten a lot of bad news). But that might just be me coming up with justification for a gambler’s fallacy belief that aligns with my intuition about elections.

But more importantly, I don’t see anything in Taleb’s Twitter or articles that would support that as being an element of his model. I admit though that I haven’t tried to fully understand Taleb 2018, maybe it’s in there (or implied in there) somewhere.

Curious:

Another more abstract way of intuitively grasping what might be problematic about Taleb’s statement is that it involves conditioning on the random quantity whose properties he is interested in modifying.

]]>Curious:

In the lengthier follow up post to Patrick above, I (tried to) explain why that limit is somewhat nonsensical. Taleb is attempting to argue that if the variance of the underlying security is high, then the option price should concentrate at 0.5. The limit he sets up proves a subtly different conditional statement. By pausing at some point t and sending the variance of the process to infinity, he assumes “future volatility of the security” goes to infinity. This is *not* the same as assuming that the volatility of the underlying security is going to infinity. The former setup might be relevant if we were studying time-dependent variance in voter preferences (i.e., we think there’s greater variance in voter preferences in October than June), but that’s not what Taleb is talking about.

Again, the coin flip example makes this concrete. Let’s assume we’re playing that coin flip game I described in the original comment. Heads means I get a $1, tails means I lose a $1, and some option pays out $1 if I have a positive dollar value at the end of the game (clearly its value at any point is equal to the probability that I will have a positive balance at the end of the game). Let’s say the game lasts 10 coin flips. Now, if we pause before coin flip 2,…,10 and send the pay out from a subsequent coin flip to $1000 or -$1000, the price of the option will obviously become very close to $0.5. But pausing at coin flip 5 and assuming that the rules of the game suddenly changed from flips 6 through 10 isn’t helping us answer the question of how the option will be priced when the payouts from the coin flips get larger *everywhere*.

Increasing the variance of the underlying security everywhere has *two* effects when we study the option price at time t (for any t). 1) it pushes the value of the option towards $0.5 because, in plain English, more crazy things could happen in the future, 2) it increases the volatility of the option price at time t because more crazy things could have happened in the past. In fact, for the coin flip example, these two effects precisely cancel out. The win probability doesn’t depend on the variance of the random walk at all, but solely on the number of coin flips I win.

]]>I was referring to:

“Taleb rejects Clayton’s setup because he says that when you pause at any point, the option price should go to 0.5 if the variance of the asset in the remaining time period goes to infinity.”

Why would this not be the correct assumption?

]]>Curious:

Sorry if something wasn’t clear! What do you mean by the price being constrained on the positive side?

]]>John:

I’m not sure I understand your argument. Why would a price be constrained on the positive side at any given point in time?

]]>It’s not much of a response to the substantive criticisms laid out by Clayton or Schmaus (in fact, Schmaus’ criticism goes entirely unaddressed). I struggle to deal with Taleb because, to be perfectly honest, I don’t think he’s responding in good faith. If you read through that response post, it’s filled with personal attacks and smokescreen math that has nothing to do with the criticisms levied by Clayton. A particularly important claim re: arbitrage is accompanied by a footnote that it was “proved in the class notes” of some class at NYU. Broadly, I’m not sure whether it’s worth engaging with someone who replies to criticism like this.

But since it’s posted, I just want to note that in Taleb’s response he makes the same mistake he made in the original analysis. For a fixed value of the underlying asset, it is true that if you send the variance of the asset to infinity for the remaining time period, the option value will go to 0.5. Taleb rejects Clayton’s setup because he says that when you pause at any point, the option price should go to 0.5 if the variance of the asset in the remaining time period goes to infinity. But does that make any sense? Let’s take the coin flip example again to illustrate this concretely. We can apply Taleb’s set up as follows: pause part way through the sequence of coin flips, and ask yourself, “OK, how should we reprice this option if we change the game such that each coin flip now changes the value of the tracked asset by +1000 or -1000 instead of +1 or -1?”

If we think that the volatility of voter preferences (i.e. the underlying asset tracked by political forecasters) is high, it should be high presumably both before and after we observe the current voter preference. By fixing the asset price at some time t and sending sigma to infinity for the remaining period, he is creating a world in which the future volatility in voter preferences is for some reason always going to infinity no matter what the past observed volatility is. Frankly, even if this was a sensible model of voter preferences (or any asset price for that matter), this decision is simply a modeling choice, not some kind of mathematical error.

The remainder of the reply is filled with ridiculous statements (and the aforementioned footnote to class notes I can’t find). He claims the following: “a forecaster…can be arbitraged by following a strategy that sells (proportionately) when the forecast is too high and buys (proportionately) when the forecast is too low.” What does this even mean? Am I arbitraging an S&P 500 ETF right now by short selling it today and then buying it later if I think the market is overvalued now and likely to be undervalued later? No! I just have a different model of the world than the average market participant, and I think I can profit from my superior model. There is no way to look at the 538 forecast at any point and know with certainty whether it is too high or low. If there is some secret method for doing so (that is not equivalent to apply my superior model for election forecasting), Taleb and Madeka should tell us about it.

]]>I think they responded here – https://twitter.com/nntaleb/status/1157399736980295680

]]>The explanation contains the following paragraph, but it’s unclear if it has any relation to what is shown in the chart:

“For every day that remains until the election, the MCMC process allows state polling averages to drift randomly by a small amount in each of its 20,000 simulations. Each step of this “random walk” can either favour Democrats or Republicans, but is more likely to be in the direction that the “prior” prediction would indicate than in the opposite one. These steps are correlated, so that a shift towards one candidate in a given state is likely to be mirrored by similar shifts in similar states. As the election draws near, there are fewer days left for this random drift to accumulate, reducing both the range of uncertainty surrounding the current polling average and the influence of the prior on the final forecast. In states that are heavily polled late in the race, the model will pay little attention to its prior forecast; conversely, it will emphasise the prior more early in the race or in thinly-polled states (particularly ones for which it cannot make reliable assumptions based on polls of similar states).”

]]>My previous comment disappeared I think, but essentially the rigorous updating curve is incorrectly computed. Both graphs correspond to different models for forecasting that Taleb came up with for a toy model. If you’re interested in delving into this more deeply, check out this Quora answer: https://www.quora.com/Whats-Nassim-Talebs-best-argument-that-Nate-Silver-is-not-very-good-at-what-he-does/answer/Markus-Schmaus

]]>I agree. It looks to me like the probability of a particular polling share… but it seems weird that the interval stays basically constant through time. Perhaps its the probability of not an aggregate polling share, but any given poll reporting a given share. Which then the width being constant says essentially that the polls have variability that is pretty much constant.

So, it’s kind of the difference between stddev of population and standard error of estimate kind of thing.

]]>While it is true that response rates have gone down, it is not true that polling has gotten noticeably less reliable (https://fivethirtyeight.com/features/the-polls-are-all-right/).

]]>So…that graph is not explained in the paper in which it is included as Figure 2, but is presented in a Twitter thread here with some accompanying Mathematica code: https://twitter.com/nntaleb/status/762048739334885380

The 538 curve is not a real forecast time series from Silver but his assessment of how 538 would compute the option price for an underlying security whose value follows a Wiener process (Brownian motion). The rigorous updating curve then captures how he would compute that price. However, as explained in the Quora answer here, the curve he describes as “rigorous updating” is computed incorrectly.

https://www.quora.com/Whats-Nassim-Talebs-best-argument-that-Nate-Silver-is-not-very-good-at-what-he-does/answer/Markus-Schmaus

The error he makes while constructing the “rigorous updating” curve illustrates the central problem with Taleb’s argument. Essentially, he assumes that the variance of the underlying process is high when making his forecast, but forgets to actually increase the variance of the underlying process. We can use the above example re: coin flips to make this error more concrete. Taleb assumes in his forecast that each coin flip changes the value being tracked by 14, but forgets to increase the variance of the underlying coin flip process to 14. In his plot, each coin flip is only changing the tracked value of the security by 1. In that case, of course you’ll never issue a forecast much higher than 0.5 — you are constantly living in fear of a coin flip that will decrease the value of your security by 14 that will never come!

Anyways, the Quora respondent shows that when you fix this mistake (i.e. increase the variance of the underlying process to match the assumption made when pricing the option), the resulting forecast curve looks just like the 538 curve. As in the coin flip example, the variance of the process doesn’t really affect the volatility of the optimal forecast over time.

Hope that makes sense!

]]>The article ends with: “And if our stated probabilities do wind up diverging from the results, we will welcome the opportunity to learn from our mistakes and do better next time.” What does it mean for the result and your stated probabilities to diverge? Does a Trump victory at the election count as a divergence given that the model claims that the probability is 12%?

]]>Thanks John, that’s very helpful.

Do you understand the graph that Makeda provided (the one with arrows labeled “538”, “rigorous updating” and “election day”)?

Is the one labeled “538” meant to be 538’s predictions over time of Clinton’s chances of winning 2016? Because those numbers don’t look right.

Is “rigorous updating” Taleb’s corrected version of 538’s predictions, or something similar? It seems to follow the same shape as the “538” line, but with much smaller amplitude. It would seem to comport with Makeda’s note that volatility means the farther out you are the closer the prediction will be to 0.5.

If so, it doesn’t seem reasonable — Makeda’s description would have me expect it to creep up over time, but it never breaks 0.55 until the final day when it jumps to (what appears to be) 100%.

Huh? You’re saying it’s predictable for old dudes to get COVID? :) Yeah that wasn’t a great example. Almost as bad as “trump gets indicted!”.

]]>Jim:

I’m no quite sure what is the definition of a black swan event, but I don’t think that “old guy gets sick” is one!

]]>What’s funny about the “black swan” discussion is that no one knows if an event is one or not until the effect on the result is observed.

“Misfire!” Gene Hackman says in the ultimate scene in Unforgiven. It’s the perfect break – the black swan!! Except Clint handles it perfectly and kills Hackman anyway.

A black swan “misfire” in the Presidential race: Biden gets COVID and becomes too ill to continue. OMG!! Trump has his chance!! Kamala Harris steps up and she wins by the expected margin.

]]>Yes, it’s a good thing you pointed out how imprecise my previous statement was too ;-)

]]>Sure. It’s a good think that the model makes no effort to predict p(Biden Wins | an election happens) then.

]]>Also, you don’t *predict* a probability. A probability isn’t an observable thing. Like Jaynes said, you can’t measure a boys love for his dog by doing experiments with the dog. A probability is like that, it’s a thing you assert about your understanding of the world based on all the things you’re taking into account. In this case they provide probabilities over vote differentials, meaning they think certain vote differentials are more likely and others are less likely.

]]>It looks like what they’re actually modeling is the vote differential. So there’s a probability distribution over the vote differential and from that you can derive a p(biden wins | …). That’s what I meant by the prediction “not being a [single] number” ([single] added for clarification)

]]>You said a couple of comments back that “The model makes no effort to predict p(Biden Wins | x) it’s p(Biden Wins | y)”. In that case, the prediction *is* a number.

]]>“You’re thinking that “the prediction” is a number. I mean, you can extract a number like the mean, but “the prediction” is a probability distribution”

+1

]]>Harrison:

But it’s not true that if you remove the 1980 data point that the line would be lower!

Why am I so sure? Because the line already goes through the 1980 data point. So removing that point will not change the least-squares line.

]]>You’re thinking that “the prediction” is a number. I mean, you can extract a number like the mean, but “the prediction” is a probability distribution, and the existence of weird possibilities as you say widens that probability distribution. It should widen more and more as time goes on because there are more possibilities for strange things to happen.

Furthermore, as they widen, the probability you assign to “biden over trump” decreases towards whatever your baseline is. That is, evidence from today that doesn’t tell you much about the future also won’t cause your prior to shift much.

Imagine for example that you have some prior idea how long my pencil is… Then you look through a super blurry telescope from 10 miles away and try to measure the length of my pencil… The errors in your measurement are on the order of +- 20cm. So you get 20 +- 20. Now your prior for my pencil length is 10cm +- 5.

If you take normal(10,5) truncated to the positive axis, and multiply by normal(20,20)… and re-normalize. the posterior isn’t very different from normal(10,5). (you can plot the two curves and see for yourself).

]]>He’s saying, in the graphic you posted (https://gelmanstatdev.wpengine.com/wp-content/uploads/2020/06/Screen-Shot-2020-06-17-at-8.38.59-PM.png), on the bottom half, red line, if you remove the 1980 data point, and refit the linear regression line, it would have a much sharper slope, and if you then used that new line to predict Carter’s vote share, it’d be something in the mid 30s.

His point is, that single data point is very influential, and the 2020 data point is far outside of the range of previous outcomes. Extrapolating in modeling is a risky thing, and you’re liable to be very wrong, especially when the regression is already heavily influenced by an outlier point (1980) in a limited data set.

]]>From a mathematical standpoint I see your point. But is there really any more (or less) uncertainty in 2020 than there was in any previous election? How can you know that? And if it’s just the spread or volatility in this election vs. past elections wouldn’t that already be in your model?

]]>The problem is that through time polling by telephone has gotten FAR less reliable. Response rates have gone from way up above 40% down to like 9%:

https://www.pewresearch.org/methods/2017/05/15/what-low-response-rates-mean-for-telephone-surveys/

Furthermore vast swaths of america don’t have landlines anymore at all.

All this means, no-one really knows what population they’re targeting by telephone polling. Andrew has done some work on adjusting these polls based on demographics, but I’m pretty convinced that telephone polling has lost its edge. It’s a little like testing for a disease that has 2% prevalence with a test that has 3% false positives.

Suppose that telephone polling has overall an 8 percent bias which through adjustments can be brought down to 4%… But of course we don’t know that. So we’re making predictions based on highly adjusted polls which still give consistent 4% errors that we’re not really aware of… Well this makes the polls useless, because in the modern world elections are won by ~2% differences.

This is why I asked Andrew if he’s got in his model an unknown measurement error bias. If you take his estimates and acknowledge a normal(0,3) percentage point residual measurement bias in all of the polls… I don’t think you’d be saying 85% chance of winning. It’d probably drop to like 59% which is only slightly better than flipping a coin and calling the winner.

There isn’t that much information in polls with 9% response rate with complicated response biases. Whatever info there is, I expect Andrew knows how to extract it… but if there’s not much to begin with… it’s rough

]]>Well explained John. +1

]]>I believe the random walk model here is intended to illustrate Clayton’s point that option prices can vary considerably even when the volatility of the underlying asset increases. I don’t think that random walk is intended to define a good model for election forecasting. I’ll try to summarize the arguments on both sides here, but I think the right conclusion here is that the derivatives pricing stuff is just unnecessary. Examining the argument made more carefully reveals that all of this options pricing stuff conceals the actual argument Madeka and Taleb are making: “I disagree with your model. I think there is higher chance than you do of Trump gaining ground in the polls.”

Taleb makes essentially two arguments:

1) as the volatility of the underlying quantity of interest (e.g. voter preferences) increases, the price of the option that provides $1 if Biden wins should approach 0.5 (no matter what the current value of the voter preferences are).

2) You’re telling me today (6/20) that the probability of Biden winning today is 88%. But conditioning on today’s information, your model’s expected forecast on August 1st for the probability of Biden winning is 65%. Then, I can make money off of you.

On their face, these arguments seem correct, but they both make bad assumptions to set up their claims! Starting with argument 1, it is true that if the volatility of the polling is high, then I shouldn’t be able to say anything useful about who will win in November today. But this argument essentially sets up the problem incorrectly. To understand why, Clayton provides the example of the random walk. In layman’s terms, let’s pretend that there’s some number we can track to determine who wins the election: if this number is positive on November 3, Biden wins. Each day between now and then, we flip a coin. If the coin ends up as heads, this number increases by 0.01, and vice versa if the coin is tails. Halfway between now and Election Day, we notice that Biden is up by 0.32, and conclude he has better than 50% odds of winning the election. OK, so far so good, Taleb probably agrees. But now, Taleb would argue that as we increase the variance of this random walk, our forecast probability for Biden should concentrate at 0.5. But is that true for the process I just mentioned? Well let’s increase the variance of this process by changing the shift at each coin flip to +/- 100. The forecast we will make halfway between now and Election Day hasn’t changed! It hasn’t concentrated around 0.5 because what mattered here was the *number of coin flips* that went Biden’s way (not their value).

Where did Taleb go wrong? It’s true that as the volatility of the underlying asset increases, the forecast probability goes to 0.5 for a fixed value of the current asset. However, the asset price (e.g. the polling margin, etc.) was also subject to that volatility in the past. The only way Taleb’s argument guarantees that the forecast probability stays around 0.5 until the very end is if you assume that future volatility goes to infinity but past volatility is fixed. Maybe it’s true that your model (and Nate Silver’s model) understates the volatility of polling later in the election cycle, but there’s nothing mathematically wrong here. An argument about modeling assumptions is being disguised as an assertion of mathematical fact.

I tried to partially address the second argument in my brief comment above regarding arbitrage. Above, Madeka claims that he can do “arbitrage” on your prediction because he’ll simply sell Biden now and buy Biden later when your forecast inevitably drops. That isn’t inevitable though. Unless you’re purposefully issuing forecasts that you anticipate will be revised downwards later, your forecast is a martingale with respect to past observations and there’s no guarantee that Madeka will make money with this strategy. This isn’t an example of arbitrage. Madeka simply thinks he knows more about election modeling and believes that your model underestimates the likelihood of future events that will harm Biden’s prospects. This assertion is perhaps true but there’s no mathematical error here. He just thinks your model is bad because he disagrees that it’s a good model of the world.

There are some details I’ve glossed over here; it is true that as your forecasts approach 100% or 0%, the volatility of your forecast should decrease. To see why, if your forecast of Biden winning is 100%, that necessarily means that you do not envision a world in which adverse events between now and the election decrease Biden’s probability of winning. If your forecast dropped from 99.9% in June to 63% in August, you would probably want to question whether your model was correct to begin with. This is a valid criticism of Nate Silver’s 2016 forecasts, but to provide a counterpoint, we’ve all seen sports games where the win probability over the course of the game goes from 99% to 50% in a flash. Rare events happen and using the 2016 election to say that political forecasting must be mathematically broken because there was lots of volatility is jumping the gun. The simpler argument is that models validated on prior election cycles did not anticipate the events of the 2016 election cycle very well (i.e. you made bad modeling assumptions).

Why did I write this long screed? Well, for one, I hope it will shed some light on the quantitative finance terminology used by Taleb to make his point. I found it confusing for a long time until I read Clayton’s articles. But also I think it’s really important that we distinguish between verifiable mistakes (i.e. incorrectly computing a confidence interval) and disagreements over modeling. Papers/authors that make the former mistake need to retract and revise their work; the latter is part of healthy scientific discourse.

]]>Andrew, since Clayton is responding to Taleb, it is Taleb who injected the ‘random-walk’ approach to forecasting a candidates vote share, e.g. here: https://arxiv.org/pdf/1703.06351.pdf

]]>Steven, good question. The more uncertainty you have in what amounts to a binary win probability, the more the probability moves towards 0.5 – this is the maximum entropy result if you will.

]]>Jim:

Sure, but suppose the forecast is based on conditions in June, so we’re comparing June 2020 with June in previous election years. Now further suppose that there’s more uncertainty in 2020 then in previous years. So we’d want our forecast to incorporate this uncertainty in the economic and political conditions in the next few months, beyond the uncertainty that was happening in earlier years.

]]>