The Monty Hall Problem: Could an LLM have convinced Paul Erdős?
Could a large language model have convinced one of the most prolific mathematicians—Paul Erdős—that he was wrong about his intuition about the Monty Hall Problem, as were so many others?
In WHY MACHINES LEARN, I begin the chapter on probability and statistics with a brief description of the Monty Hall Problem. But here’s how Claude 3.5 Sonnet answered when asked to describe it:
Prompt: Do you know the Monty Hall Dilemma?
Claude: Yes, I'm familiar with the Monty Hall Dilemma. It's a famous probability puzzle named after Monty Hall, the host of the television game show "Let's Make a Deal."
The basic scenario is:
There are three doors. Behind one door is a prize (usually a car), and behind the other two doors are goats (or some other non-prize).
The contestant chooses a door, but it isn't opened yet.
The host, who knows what's behind all the doors, opens one of the other two doors, always revealing a goat.
The host then offers the contestant the chance to switch their choice to the other unopened door.
The key question is: Should the contestant switch doors?
The counterintuitive correct answer is that the contestant should always switch doors, as it increases their chances of winning from 1/3 to 2/3.
You can read the chapter in WHY MACHINES LEARN for a thorough analysis of multiple ways of thinking about this problem, and why most of us get it wrong, because our reasoning often gets messed up in the presence of uncertainty. Erdős thought there’s no point in switching, because the probability of winning is 1/2 regardless. He was wrong.
In 1995, the Hungarian mathematician Andrew Vázsonyi used a computer program that ran 100,000 simulations of the game (where each trial involved making a choice about whether to switch or not), to show that you win 2/3rds of the time if you switch.
For WHY MACHINES LEARN, I too wrote some code to do the simulation and create an illustration (with due credit, for inspiration, to data scientist Paul van der Laken’s blog post on the same).
Could one use an LLM to do the same? Of course! This blog post is about the way you’d go about prompting an LLM.
But first, here’s the final output of my efforts at prompting Claude to generate an animation, depicting how the probability of winning converges to 2/3 if you switch, and to 1/3 if you don’t.
Animation of how probabilities converge to 1/3 and 2/3 over 10,000 trials of the Monty Hall Game
Here’s how you can use an LLM (I used Claude 3.5 Sonnet) to get it to simulate the game. You can tell that the LLM has most likely seen Paul van der Laken’s code during training, because the plot it generates is very much like his! I’m only showing the prompts and the outputs here, not the code.
Prompt: Do you know the Monty Hall Dilemma?
Claude: Yes, I'm familiar with the Monty Hall Dilemma. It's a famous probability puzzle named after Monty Hall, the host of the television game show "Let's Make a Deal."
The basic scenario is:
There are three doors. Behind one door is a prize (usually a car), and behind the other two doors are goats (or some other non-prize).
The contestant chooses a door, but it isn't opened yet.
The host, who knows what's behind all the doors, opens one of the other two doors, always revealing a goat.
The host then offers the contestant the chance to switch their choice to the other unopened door.
The key question is: Should the contestant switch doors?
The counterintuitive correct answer is that the contestant should always switch doors, as it increases their chances of winning from 1/3 to 2/3.
Prompt
Can you write python code that simulates the game? It's a Monte Carlo simulation. Calculate the probability that the contestant wins when they choose to switch and when they choose not to switch, by running 10,000 trials. Then plot the odds of winning for either case (Y-axis) against the number of trials (X-axis)
Output: The graph produced by running the generated code i shown here.
Prompt: Please describe the code you generated using pseudo-code
Claude: Certainly! I'll describe the code using pseudo-code to give you a high-level overview of the algorithm and structure.
Pseudo-code for Monty Hall Simulation
This pseudo-code outlines the main structure and logic of the Monty Hall simulation:
We define two main functions:
simulate_monty_hall_game: This simulates a single game of the Monty Hall problem.run_simulation: This runs multiple trials of the game and calculates probabilities.
In the
simulate_monty_hall_gamefunction:We randomly select the prize door and the contestant's initial choice.
We determine which door the host opens (it must not be the prize door or the contestant's choice).
Based on whether the contestant switches, we determine their final choice.
We return whether they won or not.
In the
run_simulationfunction:We run the specified number of trials, keeping track of wins for both switching and staying strategies.
For each trial, we calculate and store the current probability of winning for each strategy.
In the main part of the script:
We run the simulation for 10,000 trials.
We plot the results, showing how the probabilities change as more trials are conducted.
We print the final probabilities for both strategies.
This pseudo-code captures the essence of the simulation without going
At this stage, I wanted Claude to generate a video file or a GIF, to show the probabilities changing with the number of trials. Again, it’s worth looking at the prompts to see how you might get to clean, working code that does what you want. I’m only showing the series of prompts here, and not their outputs.
Prompt: Great. Can you change the plot, so that it appears as an animation. Show the probabilities changing over a span of about 5 seconds, for a total of 10,000 trials
Prompt: The animation takes way too long to complete. Finish it in 5 seconds
Prompt: At the end, please save the animation as video file
Prompt: Running the code generates the following error; please fix it:
Saving animation as video file... MovieWriter ffmpeg unavailable; using Pillow instead. Traceback (most recent call last): File "/Users/anilananth/anaconda3/lib/python3.11/site-packages/PIL/Image.py", line 2390, in save format = EXTENSION[ext] ~~~~~~~~~^^^^^ KeyError: '.mp4'
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/Volumes/mydisc/Dropbox/BACKUP/Why Machines Learn/ChaptersNew/python/code/ai-assist/prob/test3.py", line 67, in <module> ani.save('monty_hall_animation.mp4', writer='ffmpeg', fps=20) File "/Users/anilananth/anaconda3/lib/python3.11/site-packages/matplotlib/animation.py", line 1089, in save with writer.saving(self._fig, filename, dpi), \ File "/Users/anilananth/anaconda3/lib/python3.11/contextlib.py", line 144, in exit next(self.gen) File "/Users/anilananth/anaconda3/lib/python3.11/site-packages/matplotlib/animation.py", line 245, in saving self.finish() File "/Users/anilananth/anaconda3/lib/python3.11/site-packages/matplotlib/animation.py", line 515, in finish self._frames[0].save( File "/Users/anilananth/anaconda3/lib/python3.11/site-packages/PIL/Image.py", line 2393, in save raise ValueError(msg) from e ValueError: unknown file extension: .mp4
Claude: I apologize for the error. It seems that the ffmpeg writer is not available on your system, and Matplotlib is falling back to using Pillow, which doesn't support MP4 format. Let's modify the code to save the animation as a GIF instead, which should work with the Pillow writer. Here's the updated version:
Prompt: Please annotate the plot so that it shows the final probabilities after 10,000 trials
Prompt: Move the final probability labels to inside the bounding box of the plot
Prompt: Make the labels for the two probabilities BOLD font
The final GIF that appears above in this post is the final output of running the code generated by Claude after all the prompts.