Predictions #4

July 30, 2023 at 4:08 PM

Another exciting week of forecasting in the books! Two questions resolved since my last post (I picked up a few points on both) and a bunch of new forecasts have opened. Forecasting websites have been buzzing this week after the release of a paper claiming the discovery of a room temperature, ambient pressure superconductor - so this week’s deep dive focuses on that topic. Other topics include India’s Chandrayaan-3 lunar landing attempt, an extremely rare and deadly virus, and Argentina’s upcoming presidential elections…

Current Metaculus Stats
Total Points	204 (+76)
Overall Rank	4197 (-1511)
QC Resolved Qs	5 (+2)
QC Open Qs	15 (+6)
QC Rank	28 / 273 (no change)
QC Take	1.1% (+0.1%)

New Predictions

Deep dive of the week - Will an attempted replication of LK-99 superconductivity be published before August 4, 2023?

Resolution Criteria:

This question will resolve as Yes if, before August 4, 2023, credible sources report that a preprint or paper describing a complete replication attempt of the LK-99 results has been published.

Context:

Last week, a group of Korean researchers made waves (on Twitter at least) by releasing a set of papers claiming that they created a room temperature, ambient temperature superconductor called LK-99, made from readily available materials and able to be produced in just a few days. [paper 1, paper 2]

Superconductors are materials which, below a certain critical temperature exhibit zero resistance to electrical current. This, combined with the various magnetic properties that come along with it, make superconductors extremely useful in a wide range of applications, including MRI machines, particle accelerators, and a variety of ultra-sensitive instrumentation.

Superconductors would be useful in a host of more common applications like energy transmission and storage, transportation, and power generation - however, conventional superconductors require very low temperatures and/or very high pressures to activate their superconducting properties. The high energy requirements and complexity of the machinery required to sustain these conditions greatly limit their widespread use.

A room temperature, ambient pressure superconductor would be a monumental achievement with genuinely world-changing implications if it could be mass produced and used at scale. Trends in superconductor advances gave no indication that we were particularly close to producing one. If all the claims in the paper were reproducible and held up to scrutiny, this would be a once-in-a-generation event in science.

There are many reasons that the scientific community is skeptical of the paper’s claims:

• Once-in-a-generation discoveries are very rare. Claims of once-in-a-generation discoveries are less rare
• Many scientific papers claim results that are subsequently not reproducible
• There has been a recent incident of scientific fraud specifically concerning room temperature superconductors, by a different research group
• There isn’t a consensus that room temperature, ambient pressure superconductors are physically possible
• The paper itself is poorly written and the authors seem to have a poor understanding of superconductivity in general
• Most (maybe all) the reported properties of the material could be explained by diamagnetism, which is still cool, but not really world-changing

On the other hand, the evidence so far seems to suggest that the researchers genuinely think they’ve discovered something incredible, and probably haven’t committed outright fraud:

• The nature of LK-99 is such that other labs should be able to attempt to replicate their work in a matter of weeks. It would be strange to commit fraud in such a way that you’ll immediately be found out.
• The researchers have been supportive of attempts to replicate their work
• They published a version of the paper with 3 authors, the maximum number of recipients for a Nobel prize, which might mean they think they are going to win the Nobel prize (although there could be other explanations for this as well)
• Discontinuous advances in science, though rare, are not unprecedented

Forecasting websites have been quick to pick up this topic, with a variety of questions now active regarding the likelihood of replication, retraction, fraud, etc. This question is specifically asking about the likelihood that a paper or preprint describing an attempted replication of the original work will be published by August 4th.

Reasoning:

This question is tough because you're combining the challenges inherent to the very short timeline, with the "it only takes one" nature of the question. A thorough, high quality attempt at replication would probably be hard to pull off on such a short timeline. But only one group needs to publish something and it doesn't need to be high quality work for the question to resolve Yes.

Qualitative/intuitive approach:

This story has gone more viral, more quickly than almost any research ever does, so lots of people who could potentially attempt a replication are certainly aware of it. Whoever publishes the first replication attempt will probably get a fair bit of publicity (at least relative to their typical level of publicity), which may encourage some labs to try and go faster. Plus, there's just a lot of general excitement in a way there usually isn't around this kind of research, so I imagine there may be some groups putting in extra time / long hours to work on this, just for the fun of it.

The procedures involved are (relatively) simple, but the timeline here is very tight. And researchers are not operating on the timeline of a random question on a forecasting website!

To publish by August 4th, you probably need to already have all the materials and equipment on hand and to have started working on replication right away. There are lots of things you can mess up in terms of procedure, especially considering there are lots of missing details / ambiguous procedures in the original work. Even in a perfect world, producing LK-99 takes a few days, so you don't have time for many attempts to make this time window. Just writing up a paper will take a day or two, assuming you've got results you want to publish.

The consensus is that the study will not replicate and this probably reduces the chances of Yes here. If you aren't immediately able to reproduce the results, you will probably try more things (e.g. different preparations of the material) before concluding that the procedure doesn't work and publishing that result. If you actually just followed all the steps and immediately reproduced the results from the first paper, you may be quicker to publicize that, but this seems quite unlikely given the poor level of detail in the original papers and the prevailing view that the results are unlikely to be replicated at all.

Also, you don't necessarily need to publish anything to get the publicity for being the first replication attempt. I think it's quite likely that some groups will publicize their initial findings on social media in the next week, but actually publishing a paper is a different matter.

Finally, many of the groups working on this probably don't care about being first or going particularly fast. Being super-online & plugged-in to this topic probably has inflated my sense of how quickly news of this has spread. Realistically, many of the labs that could / will eventually try to replicate this won't actually have heard about it right away.

Currently, I'm giving a lot of weight to the "any single attempt resolves this to yes" argument, so my quick gut-feeling prediction is that I think it's more likely than not, maybe 55%.

Quantitative approach:

Let’s also try some quick napkin math:

Criteria	Estimate	Groups	Notes
Labs equipped to quickly replicate this work	10,000	10,000	4,000 universities in US, avg of 1 properly equipped lab per university, double this for rest-of-world, plus a variety of non-university labs
% that heard about the study in time	20%	2,000	probably most haven’t but many have
% that will try to quickly replicate this work	1%	20
% of those labs that care about being first / early	10%	2	if you’re already attempting replication, you’re disproportionately likely to care about going fast

Anecdotally, there seem to be more than 2 groups who have already given indications on social media that they’re working on this, so let’s say this could be off by an order of magnitude. So between 2 and 20 eligible groups.

Now what’s the chance that each one of the eligible groups completes a replication attempt and quickly publishes their results. I estimate maybe 2 - 10% chance?

# of Groups	Chance of publishing a paper	Chance of Yes
2	0.98	1 - (0.98^2) = 4%
2	0.90	1 - (0.9^2) = 19%
20	0.98	1 - (0.98^20) = 33%
20	0.90	1 - (0.9^2) = 88%

I really don’t trust this math very much (lots of uncertainty for every individual step), but I think it at least gives some useful ballpark numbers and a framework for thinking about the problem. There are probably a single or low double-digit number of labs working to very quickly replicate this work. Each one of these groups is itself pretty unlikely to publish a replication this week, but the chance of any one group pulling it off is much higher, increasing exponentially with the number of individual attempt.

This also makes the assumption that each attempt is independent, which is certainly not true - if certain aspects of the procedure slow down one group, this increases the chance that other groups are slowed down as well.

Forecast:

To come up with a final prediction here, I’ll average the 20 / 0.98 scenario (which seems the most likely to me), with my intuition-based prediction of 55%. I'm also reducing the 33% chance from the quantitative method to 20% to account for the non-independence of the trials.

This puts me at 37.5% chance of Yes

Will India's Chandrayaan-3 mission successfully land a rover on the moon?

The Chandrayaan-3, pronounced chan (as in can) - dry - Ann, mission is India’s second attempt to achieve a soft landing on the moon. Here ‘soft landing’ means the lander touches down at very low speed, hopefully with everything intact. India’s previous attempt, Chandrayaan-2, made a soft landing attempt in 2019. The lander reached lunar orbit and performed okay during the first part of the descent, but lost control during the final slow descent stage and crashed, destroying the vehicle.

The launch for the Chandrayaan-3 mission took place on July 14th without any issues, and the spacecraft (which carries the lander which carries the rover) is currently in orbit around Earth. Starting August 1 it will enter a transfer trajectory which takes it to the moon and subsequently enter lunar orbit, at which point the lander will deploy and descend to the lunar surface.

for a great overview of the mission, I recommend this Scott Manley video

Soft landings like this are very difficult, and the track record for recent attempts is mixed. While China has rapidly established itself as a major player in space with 3 successful soft landings, each of the 4 other attempts since 2013 has been unsuccessful. The last attempts pre-2013 were in the mid 1970s when the US and Soviet Union were each making several attempts per year. I’m ignoring the pre-2013 attempts here since there’s such a time gap, and since there’s no overlap in terms of the countries involved. This 4/7 success rate gives me a starting point of around 43%.

Mission	Year	Country	Result
Emirates Lunar Mission	2022	UAE	Lander Failure
Hakuto-R	2022	Japan	Lander Failure
Chang’e 5	2020	China	Success
Chandrayaan-2	2019	India	Lander Failure
Beresheet	2019	Israel	Lander Failure
Chang’e 4	2018	China	Success
Chang’e 3	2013	China	Success

Having made a previous attempt should increase India’s chance of success this time around, and we know that last time they made it pretty far before things went wrong. Still, it’s a difficult task that nobody aside from China has pulled of in decades. I give the landing attempt a 60% chance of success, and then reduce this slightly to factor in the small chance of an issue with the spacecraft, deployment of the rover, or communication issues after landing.

Prediction: 55% chance of success

What percentage of the vote will presidential candidate Javier Milei receive in Argentina's upcoming Open Primaries Elections?

Javier Milei is a far-right / libertarian candidate in Argentina’s upcoming presidential election, running unopposed in the “Liberty Advances” party. Early in the year he was polling unexpectedly well, even leading in a small number of polls over the favored “Union for the Homeland” and “Together for Change” parties.

His polling numbers have declined a bit in recent weeks and while he remains comfortably in 3rd, he’s now well behind the two leading parties. Primaries for the election are coming up on August 13th and should give a strong indication as to the relative strength of each party.

To get an estimate here, I’m just doing a very simple polling average, averaging together all the recent polls, and distributing all the undecideds proportionally to each party. To get 25th and 75th percentiles, I’m honestly just picking numbers that look about right. I’d like to read more about good strategies for averaging polls and getting reasonable margins of error, but for now I’m just going for something in the right ballpark.

Prediction:

• Lower quartile: 13.4%
• Median: 18.4%
• Upper quartile: 23.4%

Will a US warship enter the Black Sea before September 25, 2023?

This one isn’t terribly interesting. I’m at 1%, as is the community average. The impetus for the question is that the Black Sea Grain initiative has expired and Russia is taking an aggressive posture in the Black Sea, threatening to attack any ships travelling to Ukraine. There have been a few scattered editorials suggesting that NATO could provide protection to Ukrainian grain shipments now that the deal is expired.

This seems very unlikely. Turkey closed the Bosphorus (the only entrance to the Black sea) to all warships at the start of the war in Ukraine, and hasn’t given any indication it plans to open it soon. The US sending warships to the Black Sea would needlessly create an extremely tense situation in which if Russia was true to its word, would directly attack a US warship.

In the unlikely event that NATO did decide to protect Ukrainian grain shipments, Turkish ships would be the most likely ones to do it, given Turkey’s somewhat less antagonistic relationship with Russia.

Will there be a confirmed case of Marburg virus in Algeria by July 28th?

Marburg virus disease is a rare but extremely deadly disease that has had infrequent small outbreaks, mainly in central Africa, dating back to the 1980s. There were also two outbreaks of MVD in the Soviet Union in the late 1980s as a result of lab incidents. Disease symptoms are similar to Ebola, and the case fatality rate is well over 50%.

The question was prompted by a very sketchy report of potential MVD cases in Algeria, coupled with the fact that there have been two separate outbreaks already this year, one in Equatorial Guinea and one in Tanzania.

Algeria is not a place you’d expect to see MVD pop up. Pretty much all prior outbreaks have originated in forested regions of central Africa, while Algeria sits on the north side of the Sahara desert. It’s extremely unlikely that an outbreak would originate in Algeria.

Additionally, there are no other ongoing outbreaks (the prior two outbreaks this year have been over for several months), so it’s unlikely anyone would have brought the disease to Algeria from somewhere else.

Finally, the reports suggesting there could be a cluster of cases in Algeria seemed… not trustworthy. The only actual evidence I could track down was a photograph of a document that mentioned the disease, which both looked fake and didn’t actually seem to say anything about people having MVD. The hospital in question had made a statement saying there were no MVD cases and the people rumored to have it had died due to an ongoing heatwave.

My prediction opened at 2%, which I lowered to 0.5% and then 0.1% after a few more days. The question has now resolved to No. I ended up slightly better than the community, though only a bit since everyone went very low very quickly on this one.

Will a nationwide UPS strike be underway in the United States on August 4, 2023?

UPS has been negotiating a new contract with the Teamsters Union, which represents roughly 340,000 UPS workers, more than half of its workforce. This question was raised after negotiations broke down between the two sides on July 5th. Key issues in the contract negotiations include:

• Installation of AC in more trucks
• Various wage-related issues, especially concerning experienced part-time workers, who often earn less per-hour than new full-time hires

Strikes of this size are very uncommon. This would be the 6th largest strike in US history, and the largest since 1959, so a strike here would be rather unprecedented in recent decades. Strikes are costly to both sides involved, especially so when they involve such a large number of employees. UPS temporarily losing half its workforce would of course be a huge problem for business. At the same time, as another commenter pointed out…

A strike is actually a terrible thing from a union’s standpoint. Why? There are many reasons, but one of them is simply because you have to pay strike pay to the picketers. In the case of the Teamsters, the commitment is 5x union dues. With cash and investments on its balance sheet of about $450 million or so (annual report here) and with 340k UPS Teamsters, this will start to bite pretty fast. And then afterwards you might have to raise union dues. The Teamster’s dues are only about 1.5% of full-time wages, making it one of the most inexpensive unions in the country. So the Teamsters doesn’t want to lose that low-dues advantage.

Talks between UPS and Teamsters resumed shortly before the question opened. This, together with the rarity of large strikes like this, the high cost of a strike to both sides, and the seemingly reasonable nature of the major union demands, gave me pretty high confidence that a strike would be avoided.

I opened at 7% chance of a strike. The two sides reached an agreement on July 25th, so this question will eventually resolve to No.

Updates

Will Ohio Issue 1 Pass?

Increase from 14% to 22%

Two new polls have come out since this question opened, one showing a major lead for the ‘no’ side, and one showing a dead heat. Also, campaign finance information was released on July 27th and showed that both sides had raised very similar amounts.

My priors here say that this is unlikely to pass with the most likely outcome being a decisive win for ‘no’. In total, the polling and finance data points towards a fairly close race, so moved upwards a bit to 22%.

Resolved Predictions

Will the Dow Jones close at or above 35,000 before August 1, 2023?

This resolved Yes at the end of the day on July 19th since the Dow closed above 35,000 that day. My basic historical-data-based did fairly well - I basically broke even on points because I had accumulated nearly equal weight on both the Yes and No sides as my estimate moved around day-to-day.

Some commenters suggested that the community should have been more bullish on Yes because of the fact lots of major companies were reporting earnings during the question period. The argument basically boiled down to “earnings season is more volatile”, and volatility increases the chance of at least one day closing particularly high.

I’m not totally convinced though, and I don’t think anyone actually gave hard data showing that earnings season is more volatile (though it does make sense intuitively). I think my approach was totally fine on this one.