January 17, 2026 at 1:15 PM
Classics
Moby Dick - Herman Melville: Reading this book I kept thinking to myself, “how is it possible that a human mind managed to put all of these words together? How does that sentence come into a person’s head? I could spend a year and not produce one paragraph like this, and he’s come up with 500 pages!” The opening chapter of Moby Dick is one of my all-time favorite bits of prose. This is by no means a ‘can’t-put-it-down’ book in terms of the plot, and is probably one where I’ll revisit specific chapters, rather than reread it from start to finish. Not that I didn't enjoy the plot, and the symbolism of the white whale and Ahab and the voyage will stick with me, but this is a book you read first and foremost for the words.
Yes, there is death in this business of whaling—a speechlessly quick and chaotic bundling of a man into Eternity. But what then? Methinks we have hugely mistaken this matter of Life and Death. Methinks that what they call my shadow here on earth is my true substance. Methinks that in looking at things spiritual, we are too much like oysters observing the sun through the water, and thinking that thick water the thinnest of air. Methinks my body is but the lees of my better being. In fact take my body who will, take it I say, it is not me.
…
there is a Catskill eagle in some souls that can alike dive down into the blackest gorges, and soar out of them again and become invisible in the sunny spaces. And even if he for ever flies within the gorge, that gorge is in the mountains; so that even in his lowest swoop the mountain eagle is still higher than other birds upon the plain, even though they soar.
Meditations - Marcus Aurelius: I enjoyed reading this - I read most of it at the gym during the breaks between sets. It’s a collection of the Roman emperor Marcus Aurelius’s personal journals and is composed of many short entries (a few sentences to a few paragraphs), without any narrative or overarching theme, so it lends itself to being read in little bits. There are some great nuggets of wisdom throughout, but beyond that I just found it fascinating to see how the most powerful person in the world 2,000 years ago was thinking about essentially the same existential questions we all grapple with today.
For Whom the Bell Tolls - Ernest Hemingway: This was the first selection of my new ‘classics’ book club, which I intend to continue throughout this year. It was the second Hemingway novel I’ve read, the first being The Old Man and the Sea which I listened to as an audiobook all in one go. I loved this book. It takes a little while to get going, but then builds and builds all the way to the end. The Roberto-Maria storyline felt rushed and unnecessary, and Maria in particular wasn’t really given any agency, but the rest of the characters and relationships were compelling. El Sordo on the hill was an unforgettable scene.
Other
The Familiar - Leigh Bardugo: A contemporary historical fantasy novel. Wasn’t for me.
The Ministry of Time - Kaliane Bradley: A contemporary sci-fi novel. Also wasn’t for me.
The City and the City - China Miéville: A 2009 sci-fi novel winner of the Hugo prize. I liked this one significantly more than the other contemporary fiction I read last year. The premise was unique and thought-provoking, and while I didn’t love the ending, the bulk of the book was good enough that I’d still recommend it.
The Deep Places - Ross Douthat: This book captures, better than anything else I’ve read, the experience of living with chronic illness/pain. Douthat recounts how he went from being an ordinary, thriving adult, to having his life up-ended by [what he believes was] chronic lyme disease, and the extreme lengths he went to in an effort to make the pain stop. I had a similar experience after college (things were never as bad for me as what Douthat describes) where chronic pain dominated my life for a few years, and it was cathartic to see many of my own thought patterns mirrored by Douthat.
In the same way that “we filter for people who are like us intellectually and politically,” he [Scott Alexander] wrote, “we also filter for misery,” so that the suffering around us passes unheard and unseen.
To get sick and fail to get better is to realize the harsh truth of this insight. Human beings have a great capacity for kindness, empathy, and help, but we are more likely to rise to the occasion when it is clearly an occasion—a moment of crisis, a time-bound period of stress. In the aftermath of a hurricane, society doesn’t usually fragment; it comes together in solidarity and support. Likewise with families and individuals facing suffering in the moment that it descends, or when a terrible arc finally bottoms out: Not always, but very often, people behave well, with great generosity, in the face of a mortal diagnosis, a mental collapse, an addict’s nadir. Not least because in those circumstances there are things you can clearly do, from the prosaic—making frozen dinners for a suffering family—to the more dramatic and extreme, like flying across the country to help drag a friend into rehab.
But when the crisis simply continues without resolution, when the illness grinds on and on—well, then a curtain tends to fall, because there isn’t an obvious way to integrate that kind of struggle into the realm of everyday life. It’s not clear what the healthy person is supposed to give to a friend or family member who isn’t dying, who doesn’t have some need that you can fill with a discrete act of generosity, but who just has the same problems—terrible but also, let’s be frank, a little boring—day after depressing day.
“Pain is always new to the sufferer, but loses its originality for those around him,” the nineteenth-century French writer Alphonse Daudet wrote” … “Everyone will get used to it except me.”
I’m dubious of his theories about what was really going on with his body, and extremely dubious of some of the treatments he pursued (though I absolutely relate to the desperation that led him to pursue those treatments), but those aren’t the reasons to read this book. The reason to read the book is to understand what it’s like for a person to be in pain all of the time and not know how to make it stop.
The Way Out - Alan Gordon: A very different book about chronic pain, this one by a doctor who has helped many patients recover from chronic back pain by (in my opinion correctly) treating it as dysfunction of the nervous system rather than solely, or even primarily, as a structural problem. I had not read this book during my earlier run-in with chronic pain, but the ideas Gordon explains are consistent with the things that eventually worked for me. If you’ve been dealing with chronic pain (pain that’s continued long beyond an initial injury or has no clear cause, and occupies a lot of space in your mind), I’d recommend this book.
Feel Good Productivity - Ali Abdaal: I bought this one because I’ve watched many of Ali’s YouTube videos on productivity, motivation, and business and think he usually has good advice. The book was fine, it’s got plenty of good ideas and experiments to try, but it does feel like it’s written by a YouTuber. I don’t think there was anything in it that you wouldn’t find on his YouTube channel and I think he’s more effective in that medium. You could read this one in a day or two if you wanted though, so if you’re just looking for a quick infusion of ideas to get out of a rut or something, it might be helpful for you.
January 11, 2026 at 2:30 PM
At the end of 2024 I started an in-person history book club in order to motivate myself to read a bunch of history books. We went through 7 books over the course of the year (one was a DNF due to the book being terrible), covering a diverse set of topics.
The book club has been a success - I had no idea if anyone would sign up and now we’re up to about 140 members (of which maybe 20 or so are active), and it did indeed get me to read lots of books that I otherwise would not have read. All the meetings had at least 5 attendees and there seems to be a core of regulars. I’ll write another post sometime with more reflections on the book club, but here I’ll run through the books we read and my brief thoughts on each one.
Everything Under the Heavens - Howard W. French: Not bad and not too long, but not the best choice for your first book on Chinese history, which it was in my case. This book surveyed China’s relationship with most of its neighboring countries, in each case going back in history and then working up to the present to show how events of the past shape the current relationship. It did a fine job of what it was trying to do, but was more focused on specific, modern issues and how they came to be rather than primarily being a history book, which made it an imperfect choice for my book club.
The Three Emperors - Miranda Carter: Covered the lives of King George V (plus a lot of Victoria and Edward VII), Kaiser Wilhelm II, and Tsar Nicholas II, all of whom were of similar age and reigned as monarchs in their countries during the onset of WWI. The most interesting theme was the way that Britain, Germany, and Russia differed in how much ‘real power’ their monarch had at the beginning of the 20th century, and the problems this caused, particularly in Russia where there was a very weak parliament and the tsar had mostly unchecked power. The book went into extensive detail on the minutiae of each monarch’s life which I found tedious. I would also recommend reading some other primer on the buildup to WWI before reading this book. This book gives an interesting viewpoint for analyzing WWI, but focuses mainly on the lives and relationships between the monarchs and other government officials and I found it hard to connect these to the larger context.
Inventing Japan - Ian Buruma: This was a non-book club selection that I chose because I was looking for an introduction to Meiji-era Japan. It's a short book and does indeed give an introduction to Meiji-era Japan, but it wasn’t exactly what I was looking for. I think it assumed more background than I had, so a lot of it went over my head. The writing style was a bit odd as well. I don’t anti-recommend it, but you may be better off trying something else if you’re interested in the subject.
SPQR - Mary Beard: A broad survey of Roman history and an excellent book. It covered a large timespan (which was what I wanted, being new to Roman history) and was both accessible and thorough. She did an excellent job of differentiating between “what we know definitely happened,” “what we think might have happened or have only a fuzzy picture of,” and “what is pure speculation / legend.” In addition to being a great book on Roman history, it was also a great book for learning how we learn about antiquity at all, the information sources we rely on for this era (sometimes we’re trying to infer a great deal from a fragment of an inscription on a tombstone) and the pitfalls ancient histories can have (written to legitimize the current ruler and/or denigrate prior rulers).
The Anarchy - William Dalrymple: A history of the British East India Company and how it gradually amassed control over all of India. Interesting throughout, it spent more words than necessary on a number of different battles, but the broad story was well-told. One interesting idea was that, indirectly, the American Revolution had a major impact on Indian history by prompting the British to change tactics in India to avoid a similar outcome. I was also surprised at how independently the East India Company acted. I had the impression that the colonization and conquest of India was a project of the British government, and that the EIC was just a vehicle for accomplishing it, but it seems that really was not the case in the beginning. The EIC raised armies, built cities, established local governance, often while actively hiding its actions from the British government. If anything, the government was holding the EIC back for the first hundred years or so.
The Silk Roads - Peter Frankopan: A very broad history of much of the world, with a loose focus on the Middle East and the way the dominant trade routes (“silk roads”) of different eras shifted the balance of world power. This is a great choice if you’re new to reading history, given its breadth, though that also means it doesn’t go especially deep on any one topic. The 'Silk Roads' framing of the book was forced and unnecessary, but didn't really detract anything. One thing that stood out to me was the way that (1) demand for prized goods from India and China led to sprawling trade routes between Europe and Asia, (2) control of those trade routes concentrated wealth in the most powerful cities along the routes, and (3) desire to circumvent those ‘middlemen’ and gain direct access to eastern goods spurred innovation which eventually irrevocably upended the world order. Countries like Spain and Portugal began the 'Age of Exploration' because sea routes were the only way for them to gain unrestricted access to eastern goods. And then England followed suit and leapfrogged them, likewise forced by their powerful neighbors to innovate further. The accidental discovery of the Americas and the enabling technological innovations had the side effect of elevating those countries to power at a pivotal moment in history.
[DNF] Debt -David Graeber: Utterly incomprehensible. I think I disagree with most of the author’s ideas but it’s difficult to say for sure because I don’t think he is himself clear on what his ideas are and the book doesn’t have any kind of logical structure. I voted for this book in our book club poll thinking it would be some kind of history on the evolution of debt (I know, how foolish of me to think that of a book titled “Debt: the first 5,000 years”), but in reality it was a mashup of philosophy / political treatise / anthropology with only a bit of (dubious) history, and does none of those things well. I gave up about half way through. Everyone in my book club who had made significant progress on the book also thought it was terrible. Anti-recommend.
1776 - David McCullough: An excellent read to end the year, and the first American history we read for the book club. This was a notably different style of book from our other selections, focusing on a much narrower subject (the American Revolutionary War, specifically in the year 1776) and told mostly through first person accounts by prominent figures in the conflict. I was dubious for about 10 pages and then I was hooked, this was the most “can’t put it down” history book I read during the year. The book focused on the three major events of the war in 1776, the siege of Boston, the battles in and around New York, and Washington’s crossing of the Delaware. I had not realized how dire the situation was at the beginning of the war. It really does seem that if not for a handful of very specific decisions (timely retreats, the Delaware crossing, Howe’s conservatism), Britain would have simply wiped out the Continental Army, forcefully suppressed the rebellion, and … who knows what history would look like from there.
January 11, 2026 at 10:45 AM
Since the last update I:
- Took a break for a few weeks to visit family for the holidays (and be sick)
- Created a Robotics Prep LinkedIn page and started collecting followers
- Started posting educational and interview prep content on LinkedIn
- Did an introductory call with my first prospect for the mentor program, and booked a second call (no actual signups yet though)
- Started deep diving a specific robotics topic, which I will be producing a variety of educational content for over the next couple weeks
I unfortunately had a very unproductive couple of weeks over the holidays. I had intended to work maybe 50% of the time during my 2 week visit home, but illness derailed things at the beginning and then I never got on track. It was nice to have a little break though and now I’m feeling excited to get a bunch of work done.
I had my first ‘introductory call’ for the career mentorship program this week after, surprisingly, somebody booked a call a few days after my first LinkedIn post about Robotics Prep. The call went okay. It seems like the person isn’t going to sign up for the program, but I think that’s mostly because they’re more experienced than my ‘ideal client’ and sensed that I wasn’t really more experienced then them, not because I did a terrible job on the call. Although there are plenty of things I’ll do differently for the next call.
Speaking of which, I’ve got another call with a second prospect for the mentor program this week. Given the limited reach my advertising for the website has so far, it’s encouraging that I’m seeing even this small amount of preliminary interest. I’m curious to see if that continues or grows over the next couple weeks as I expand my network and create more educational content.
I spent some time this week creating posts for LinkedIn, one “explainer” post for a very narrow robotics topic, and one “interview prep” post that just included a handful of practice interview questions. The first was very slow to create, the second went much quicker. I’m sure I’ll get faster over time, most of the slowness came from ‘not knowing how to use Canva’ rather than ‘not knowing what to say’. I will be trying hard to stick to a one post per weekday pace for now, but if that seems unsustainable, I’ll err on the side of fewer, better posts vs. more, lower quality posts.
The broader marketing and content strategy I have is something like this: I’ll spend a block of two weeks focused on a specific topic. I’ll study that topic, take notes, think about how to explain the key ideas clearly, and do some kind of interesting project that demonstrates the ideas for that topic. Then, I’ll create:
- A YouTube video (or set of videos) where I explain the key ideas and build up to the software demo
- A longform written article that explains the key ideas
- A set of 10 or so LinkedIn posts related to the topics I’ve been reviewing
- A set of links to any resources I found useful in learning the topics (to add to the study guide)
I’ll then repeat that cycle for a new topic. I’ll post the content from the first cycle while building the content for the next one. Every video, post, etc. will include a brief call-to-action referring back to the website. I like this approach because I can focus on one topic at a time and create a bunch of resources all at once for that topic, rather than jumping around and constantly switching contexts.
Tentatively, I’d like these to be two week cycles. I think this pace will be very challenging at first (especially the video part), but I’ll try to keep to that pace as much as possible. Hopefully after a few cycles I will get faster at all the different steps. I will try my best to not let the perfect be the enemy of the good here, particularly when it comes to ‘production quality.’ For educational content, the quality of the explanation is the most important thing.
For now, most of my effort is focused on this marketing work, but I’ve also scheduled some time into my weekly schedule for both website upgrades and general networking / outreach. And any ‘client work’ takes precedence over everything.
I think I was feeling a bit burned out after all the effort that went into creating the guide and website, but this new phase has been fun so far. Learning about robotics topics, thinking through how to explain things clearly, building demo projects… all of that is fun for me and I think will be invigorating to work on. And working with real clients will be motivating. I also have metrics to track now (impressions, signups, website visitors, etc.) so there is real world feedback on the actions I’m taking, which I didn’t have before now.
| Metric |
Value (Last 7 Days) |
| New LinkedIn Posts |
2 |
| LinkedIn Post Impressions |
172 |
| LinkedIn Page Visitors |
10 |
| LinkedIn Followers (new) |
5 |
| LinkedIn Followers (total) |
18 |
| Website Visitors |
?? (Need to track this!) |
| Mentor Program Interest Calls |
1 |
| Mentor Program Signups |
0 |
| Interview Signups |
0 |
January 7, 2026 at 11:30 PM
In 2025 I started working through Henry Oliver’s Great English Literature syllabus. Shakespeare is Henry’s recommended starting point, given his profound influence on all the literature that followed. My goal was to read all 11 of his suggested sampling of plays over the course of the year. I finished 9 of the 11 (plus Richard II, which I wanted to read before Henry IV pt. 1), and will round out the last two in January or February.
This was my first time reading Shakespeare, aside from an in-class reading of Macbeth back in high school, which I have little memory of. For a while I was following each read with a viewing of a recorded performance of the play, though I’ve fallen behind on this. This is perhaps the most obvious thing a person could say about literature, but I can’t recommend Shakespeare enough… I’ve had a blast reading these and plan to knock out another 10 or so in 2026!
Hamlet: Properly rated. Maybe not the easiest starting point, but I think it’s better to start with one of the masterpieces, even if difficult, so you get a taste of how good Shakespeare can be. I enjoyed Adam Scott’s rendition, and his performance of the “To be, or not to be” soliloquy inspired me to memorize the speech. I love the Hecuba speech as well. Looking forward to reading this one again now that I have a year of practice reading Shakespeare’s writing.
The Tempest: I put this in Tier 2 out of 4 in my personal Shakespeare rankings (additional tiers may be added as I read more, all rankings subject to change). I found the character of Caliban difficult but overall the themes of this play were more interesting to me than those in the other, more conventional comedies.
Our revels now are ended. These our actors,
As I foretold you, were all spirits, and
Are melted into air, into thin air:
And, like the baseless fabric of this vision,
The cloud-capp'd towers, the gorgeous palaces,
The solemn temples, the great globe itself,
Yea, all which it inherit, shall dissolve,
And, like this insubstantial pageant faded,
Leave not a rack behind. We are such stuff
As dreams are made on; and our little life
Is rounded with a sleep.
A Midsummer Night's Dream: A good play. A classic Shakespearian comedy. I’m glad I read it. But it goes in Tier 3. The bar is high!
Measure for Measure: Very enjoyable. More complex than the other comedies I read (in fact it’s sometimes categorized as a ‘problem’ play, not fitting neatly in the comedy or tragedy category), and though the ending is a bit awkward, that’s a minor part of the play all things considered. I liked the Globe Theatre rendition, especially the Duke. I'm looking forward to the forthcoming Tyler Cowen - Henry Oliver discussion of Measure for Measure.
Romeo and Juliet: I’m glad that I finally know the actual plot of this one and that all the references make sense now! It goes in Tier 2. There’s lots of memorable poetry, a good plot, and it’s more ‘re-readable’ than most of the other plays. At the risk of giving deeply-uninformed literary analysis, this one felt a bit ‘messier’ or less ‘polished’ than, say, Hamlet or Richard II, almost as though the ideal version of it would be edited to 80 or 90% of the words. But of course we're nitpicking here, it's a wonderful play. I've not yet found a top-tier recording. The Globe Theatre taping had a phenomenal Juliet, but the Romeo was so bad it ruined everything else about the play and made it borderline unwatchable.
Much ado about nothing: I feel much the same way about this one as I do about Midsummer. Very enjoyable, good poetry and good plot, but not as sensational as some of the others. I think I am just less compelled by the comedies than the tragedies.
Macbeth: I loved the poetry (lots of highlighting in this one, which I do when I enjoy a line or section), but as a play it was merely good. I need to give this one another go, though, because I read most of this before bed, where my tired brain sometimes struggles to fully grasp what I read. This is also the point where I stopped keeping up with watching recordings of the plays, so consider the rest of these appraisals incomplete.
Winter's Tale: A bit meh. Mid-tier poetry and mid-tier plot. However, I read this after a long break from Shakespeare, so perhaps my brain needed some time to adjust to the writing. I’ll give it another shot sometime in the future.
Antony and Cleopatra: Also a bit meh. This one has noticeably more ‘plot development’ than the other plays I read, and I found this made it difficult to follow. The poetry was just okay (by Shakespeare standards), I didn’t highlight much and there aren’t any lines or phrases that have stuck in my brain. I do need to watch a performance of this one though, I’m sure the plot will be easier to follow with visuals and maybe that will change how I feel about it.
Richard II: Tier 1, easily. The plot probably isn’t a Tier 1 plot, but the poetry is so good that it doesn’t matter. The poetry was so good there didn’t even need to be a plot. I probably highlighted about half the lines in the play. I could open this one to any random page and I’m sure there’s something on that page that’s fantastic. Here, let’s try:
pg. 67
Methinks King Richard and myself should meet
With no less terror than the elements
Of fire and water when their thund’ring shock
At meeting tears the cloudy cheeks of heaven.
Be he fire, I’ll be the yielding water;
The rage be his, whilst on the earth I rain
My waters; on the earth, and not on him.
And
King Richard doth himself appear,
As doth the blushing discontented sun
From out the fiery portal of the east
When he perceives the envious clouds are bent
To dim his glory and to stain the track
Of his bright passage to the occident.
Marvelous! There are very few ‘filler’ lines in this play, even minor plot development is written beautifully. And they say Henry IV and V are the real standouts of the Henriad!
October 9, 2023 at 7:57 PM
I won the tournament! Would not have predicted that would happen… It ended up being a pretty strong victory, with about a 25% tournament take. This means that I win 25% of the $0 prize pool which, alas, only comes out to $0 if you do the math. Apparently I may get a Metaculus-branded hat though, and you really can’t put a price on that. And of course glory. You can never have enough of that.
I’m now attempting to defend my crown in the 2nd Quarterly Cup which is already underway. I’m not going to write about every question, but I may pick and choose a few to share thoughts on if I feel like it. Also, it’s not too late to join if you’re interested!
For now, here are 10 thoughts about forecasting, Metaculus, and the forecasting community that have been bouncing around in my head the last few weeks:
Forecasting is a great way to follow the news. when you’re reading the news to make a prediction on something, you read it in a very different way than a typical reader would. Instead of passively browsing, you actively seek out articles in order find specific information. You formulate specific questions - “How long do Israeli court decisions typically take?” “How long do we expect this one to take?”, and then try to extract that information from news articles, which may answer it explicitly, or may just offer hints.
I think this style of news-reading results in much better comprehension than a more passive, browsing style of reading. I’ve found that I’m much better informed about topics I’ve made predictions on compared to topics I’ve just read passively about in the news.
Writing about every forecast was difficult - but also is probably a big part of why I did so well. For the first 2/3 of the tournament, I tried to write something about my initial prediction on every question. I settled on a style where I’d spend a few paragraphs summarizing the question context so that a reader who wasn’t closely following the topic could understand the question, followed by a few paragraphs where I explained how I landed on my number.
There were lots of tournament questions, so this took a lot of time. I was usually just learning about each topic, and it took serious effort to summarize all my background reading into a few concise paragraphs. Also, it forced me to actually write down the specific reasons I landed on the number I did, which often were very vague, intuition-based reasons, rather than easy-to-justify baserates and statistics.
As painful as this process was, I think it was also really good for my predictions. After all, I think I’m the only tournament participant who did this and also the only tournament participant who won, and those things may be related! Concretely, it ensured that on every question, I knew the context of the question well-enough to explain it fairly concisely, and it forced me to understand what factors I was weighing and how much when I made the prediction. This is helpful in the moment, as it exposes things you were tricking yourself into thinking you understood, and is doubly useful later on, as you now have a historical database of how you approached different questions and what the result was.
Am I just a natural at this? If you decided to take up a random sport one day, you (without practicing) joined a tournament hosted by one of the leading institutions behind sport, and then you won that tournament, what would you conclude?
If it’s a very luck-based sport, you probably just got very lucky. But if not, I think you should conclude that both (a) you’re probably naturally pretty good at that sport, at least more than the average person, but also (b) you probably weren’t competing against that strong of a field. This is how I feel about my performance. While there was definitely luck involved, there were too many questions for it to really be about luck. But also, if I was really competing against a bunch of elite forecasters, I should not have won. For the first handful of questions, I didn’t really even know how the tournament scoring worked and pretty quickly plunged my score well into the negatives. I also completely mailed in a few questions, some of which resulted in very bad scores. So I should have been very beatable.
I think the best explanation here is, (a) forecasting is a very niche thing that not many people do, so the average field in any tournament will be pretty weak, (b) this specific tournament had no prize pool, so if you’re a very good forecaster, you may have just skipped it, and (c) I do have some attributes that make me especially good at forecasting - more on this below
I would also add that I joined a different forecasting site, Manifold Markets back in August, and in 3 months have turned the 500 starting ‘Mana’ you get when you sign up into 8500 mana, and have specifically made a point to not do any research and just buy/sell based on intuition. Again, not sure what to conclude here, but it seems very possible that these sites are just full of people who are terrible at predicting things, such that it’s easy to do quite well by just being half-decent.
I’m particularly well suited to do well in forecasting tournaments - I’ve been “very online” for many years and am quite good at quickly tracking down information about questions. I’ve also read, with genuine interest, about such a wide variety of topics (geopolitics, tech, sports, science, political polling, etc.), that while I usually don’t know much about the specifics of a question, I have lots of ‘reference points’ of knowledge that help me orient myself when reading about it. In other words, I’m a generalist, with surface-level knowledge a very many things, even if lacking in-depth knowledge in specific areas. The result of this is that when faced with a new question, I can usually understand it pretty well within an hour or two of reading.
I’m also the kind of person that enjoys tracking 15 different events as they unfold by meticulously doing filtered Google searches each day and keeping a library of bookmarks and Twitter accounts to check in on repeatedly. This is pretty important in a tournament like this one, where you can have questions about rapidly unfolding events, which require you to track down up-to-date information every day. I can completely understand how people would find this overwhelming, or just not enjoyable. But for whatever reason, I find it very fun. I was already sort of doing this just in a more passive way before I started forecasting, so it felt like a very natural transition from my prior online habits.
Different styles of questions, different styles of predictions - Different styles of questions lend themselves to very different prediction strategies, all of these strategies can do well for the right types of questions, and choosing the right strategy for each questions is really important.
For some questions - “will the Dow Jones cross such and such threshold by this date?” - you should clearly just use a model based on some historical data and keep intuition out of it. I can think of several examples of where the community was clearly letting vibes move them away from what the quantitative model would say, and where by just sticking to a historical average, I scored well on the question.
For other questions - will India’s rover successfully land on the moon? - there is a very clear baserate you can start with, and then adjust from there based on the specifics. In recent decades, landers have about a 40% success rate. India has failed once, but got close, and has a competent space program. So probably a predicting of a bit over 50% is a reasonable guess.
Some questions didn’t lend themselves well to quantitative analysis, and lacked a clear baserate to start from - will the Black Sea grain deal be revived before July 16th? - this style of question relies much more on analyzing different news sources and trying to gain an intuitive sense for what will happen. This is an example of where I tried to by overly quantitative in my approach early on and latched on to a poor baserate, and ended up doing poorly. I may have been better off just reading a bunch of articles and thinking “does it seem like Putin cares about this deal?” and going with the number that felt right.
To comment or not to comment… Tournament-style forecasting leads to strange commenting incentives, which probably reduce the quality of the community forecast. Forecasts are scored such that the important thing is doing well relative to the field, rather than doing well in an absolute sense. To get a really positive score on a question, you need to be significantly closer to the final resolution (e.g. 10% when the community is at 40% on a No-resolving question), for an extended period of time. This means if you’re trying to win, the incentive is to not share important information that might give up your edge over the community.
There were a few times in the tournament where I realized that due to some hard-to-track-down piece of information that the community probably wasn’t considering, the real probability should have been <1% for a question, while the community was sitting at 20 or 30%. With a simple comment, I could have easily shifted the whole community to my number, thereby improving the performance of the site. But this would have eliminated any points that I would gain in the tournament from being more right than the community, so I didn’t comment.
I think this is the biggest advantage true prediction markets like Manifold have over a site like Metaculus. In prediction markets, you could just put a huge amount of money on No to move the market and gain a big profit, or put as much as you’re willing to spend and then reveal your information so that the community follows. In other words, you can cash in on your special knowledge immediately, whereas on Metaculus you can only cash in over a long time period
Who is this for? Is any of this actually useful? Most of the forecasts on Metaculus seem pretty useless, including most of the questions I was forecasting. As in, I often will read a question and think, “what specific person would be looking at this market to help decide whether to take one action vs. another?” and I’m unable to think of almost anyone. Lots of the questions are interesting which I guess is a usefulness in itself, but I’m not sure they’re useful enough that people would, say, pay lots of money for the information.
For some questions, I can think of people that might find them useful, but they’re people who would have so much better “insider-knowledge” about the question, that the usefulness of the forecast is diminished. For example, I can think of people who might really care about what % chance there is that the Black Sea Grain Initiative will be renewed - leaders in affected countries, people in the grain industry, etc. But those parties are also likely to have much more direct knowledge about the state of the negotiations, to the point that their internal estimate should be better than my outside-observer estimate that’s based on news articles.
Unfortunately, one of the best cases I can think of - questions around the likelihood that LK-99 was a room temperature superconductor - was actually a mini-disaster in terms of prediction market credibility! I think you could reasonably argue that the wildly over-optimistic forecasts on these sites were the major driver behind the hysterical online coverage (I recall many prominent people on Twitter posting the Manifold estimate very early on as justification for why people should be freaking out more), which is to say they had the exact opposite effect that prediction-market proponents claim they are supposed to have!
This isn’t to say that I don’t think these sites can be useful, just that I think the vast majority of questions are phrased in a way that makes them useless, or are about topics where the relevant stakeholders who would care about the question are likely to have superior insider-knowledge.
I also would add that there is still some benefit to having lots of useless / unimportant questions, provided that many of them have short timelines - these questions are still great practice!
Metaculus scoring generally feels pretty fair. I strongly prefer it to the prediction market style system on Manifold. Metaculus rewards being right early more than being right late. If you change your prediction right at the last second, this has close to zero impact on your score. This reduces the need to constantly check for news related to every question - if breaking news means a question will resolve immediately, it’s too late for you to do anything about it. If it doesn’t cause an immediate resolution, you have a bit of time to adjust your forecast without a big penalty. On Manifold, you can be completely wrong for a year but check Twitter at the right time and end up wildly profitable on a question. Metaculus avoids this issue.
I also like that they weight the “middle probabilities” more than the extremes - guessing 40% when the community is at 50% earns more points than guessing 30% when the community is at 40%, which is better than 20% vs 30%, and so on. I think this correctly leads to higher weight being placed on hard-to-predict questions, and discourages people from “racing” to 0 or 99.99% to try and get an edge on very low or very high %-chance questions. A side effect is that many questions end up not mattering much for the tournament as they spend the whole time near 0 or 100%, where there aren’t many points to be gained, but that seems fine.
One quibble I have is that continuous questions seem to be rather arbitrarily weighted much more strongly than yes/no style questions. Maybe I just don’t understand how they’re being scored or something, but continuous questions were generally much more consequential in terms of tournament score than the Yes/No questions, and it’s not obvious to me why this is something we would want. It’s particularly annoying because Metaculus also has a very limited UI for inputting continuous distributions (guys, some things have probabilities that aren’t bell curves!). So continuous questions are both more important, but also impossible to input my “real” prediction on.
I’m pretty sure almost everyone is just following the community on almost every question. People seem really worried to deviate much from the community. With tournament scoring, I sort of understand this - you want to avoid a big negative score, so if unless you’re super confident that the community is wrong, it makes sense to defer to the crowd judgment. However, it’s important to realize that if lots of people do this, the community average gets worse, and loses the thing that made it good in the first place - a bunch of different perspectives considering different things which average out to a good guess.
There were a bunch of questions in the tournament where people were clearly very uncertain initially, but then once the community forecast was revealed, everyone moved towards the average. This creates an illusion where it looks like the community is highly confident in that number, even though the reality is that most of the predictors just have no idea.
Many of my best questions in the tournament were ones where I was pretty far off the community, thought about hedging towards the average, but ultimately decided that my reasoning was actually good and the community was probably just herding around a number without good reason. They key thing was being able to distinguish between “I’m very different from the community, and also pretty clueless on this question, so I should probably trust them more”, and “I’m very different from the community but I think I know what I’m doing”. Winning the tournament comes down to maximizing the latter without being wrong too many times.
They really should put a prize pool on these tournaments. These “breaking news” style tournaments which have lots of short-timeline questions covering a wide range of topics are very valuable to the forecasting community and should be prioritized more.
IMO one of, if not the single biggest problem in the forecasting community is that too many people seem to be spending too much time predicting on very long term questions (spanning many years to many decades), for which I’m not remotely convinced it’s even possible to make useful predictions on, and which it is certainly impossible to use to develop good forecasting calibration/skills. You can’t get calibrated if none of your predictions ever resolve! I’m very dubious of people who make these decade-scale predictions with no prior track record of making successful short term predictions (which describes about 95% of the AI doom / not doom blog posters).
Prize pools, even relatively small ones, will attract more predictors, improve the quality of the community prediction, and properly emphasize short term forecasting over forecasting on questions where forecasters may never know the resolution.
As an addendum to this, tournaments should include one-off prizes for particularly useful comments (or some other reward scheme for comments). Tournament prizes are good because you get more people engaged in the tournament which means more different perspectives on each question. On the other hand, prizes (and tournament scoring in general) also discourage useful commenting. Useful comments that meaningfully sway the forecast in what turns out to be the correct direction should be rewarded, perhaps comparably to winning the tournament. Ultimately the goal of the site is to get good predictions out to the world as far in advance as possible, and it’s hard to achieve this if the incentive is to hide the best information.
