> After running 10 million simulated draws, I got an answer of 0.0323, which matches the real answer very closely.
If you enjoyed reading about how you can approximate complex statistical values using basic programming, I highly recommend Jake Vanderplas "Statistics for Hackers" [0][1].
It converts the sometimes hard to grok statistics terms and formulas into some very simple loops and basic operations.
> okay, fine. I'll use arc4rand which uses a cryptographically-sound prng
> nooo! Random numbers are crap! you should use space-filling sequences that are guaranteed to work optimally for infinity dimensions
> okay
Billy, what are you rambling about? Wake up! We already have a result using RANDU that the verification process confirms is correct.
rubikscube09 5 hours ago [-]
the space filling sequences are only useful in large but moderate dimensions interestingly enough. E.g. d = 100 but not d = 10 ^ 6
wodenokoto 14 hours ago [-]
One of my favourite stats / data science talks ever.
A few years after watching it for the first time I rewatched it and paused after every problem statement and tried implementing a “simple loops and basic operations” solution and failed almost every single time.
So be warned: he does make it look easy :)
alexpotato 12 hours ago [-]
It helps if you think about it as:
"Before they had functions and lookup tables, what would a human have done in the 1940s?"
e.g. the statistical significance is just "if I were to randomly try this, what percentage of time would it be above 5%"
sambeau 10 hours ago [-]
I used Monte-Carlo simulations to work out the rules to a small kids’ game.
The idea was to start with a seemingly biased game—a race between a tortoise (rolling one die) and a hare (rolling 2 dice)—and to tweak the rules until both players had an equal chance of winning.
It was a fun experiment and the game actually works well!
I wrote Monte-Carlo simulations to calculate the thermodynamics of oxygen ordering in a high temperature superconductor way back in 1993.
Calculating thermodynamic fluctuations on a 256x256 lattice took all night. And that, even though it ran on an HP 9000/750 with a 66 MHz PA-7000 (PA-RISC 1.1) 32-bit CPU.
While waiting, I used to look at weather satellite images downloaded from CERN using NCSA Mosaic. It was impressive. All the information available at your fingertips by using arrow keys and pressing return on underlined text to navigate to the next page! Emmm, link.
pizzathyme 20 hours ago [-]
I wrote a Monte-Carlo simulation while working on a Pokemon-Go style game to simulate people walking all over the world. I used statistics for average walking speed and times played for similar games. To my amazement when the game launched, my predictions were within +/- 2%! A career highlight.
theOGognf 1 days ago [-]
Those interested in this should read about DSMC: https://en.wikipedia.org/wiki/Direct_simulation_Monte_Carlo. It’s the same idea for low density gas interactions without having to actually simulate all the particles involved while still getting accurate results. Cool stuff!
hingler36 1 days ago [-]
There's been a lot of interesting work in the last ~decade on improving monte-carlo methods for solving PDEs, the field is really fascinating!
As a statistician, I’d be remiss if I didn’t mention bootstrapping - theoretically distinct, but another demonstration of the power of randomization techniques.
ViscountPenguin 19 hours ago [-]
The power of bootstrapping is pretty insane really, I'm still impressed by the performance of random forests, and even bootstrapped confidence intervals are really cool.
xinu2020 7 hours ago [-]
Probabilistic programming languages that supports sampling are a good fit for these kinds of simulations.
You elided the reason it's a valid set, "all different or all the same for each attribute separately"
The four attributes:
- Count: 3 in each; "all the same"
- Color: red, purple, green; "all different"
- Shape: diamonds, ovals, squiggles; "all different"
- Shading: empty, solid, striped; "all different"
So for each attribute the conditions are met, they are either all the same or all different.
piotraleksander 7 hours ago [-]
Got it now, thanks for explaining it to me.
SukadarBukadar 11 hours ago [-]
How is it useful when the underlying distribution is unknown, specially if it likely isn't stationary? For example, when doing such retirement portfolio projections or testing stock market trading rules, how do we know that the simulations we took into account are representative of what could have happened? Maybe some paths were unreachable and some underrepresented...
shiandow 9 hours ago [-]
You go to a higher level of abstraction and define a distribution over the underlying distribution.
Though such predictions are infamously bad at extrapolating without some strong theories behind the kind of distributions you could expect.
SiempreViernes 10 hours ago [-]
Probabilistic Monte Carlo simulations are a way to check your guesses, if you claim you can't even guess at how things work you should consult a professional astrologer.
SukadarBukadar 9 hours ago [-]
No, I claim that I can't check it without making another, more difficult guess. If my guess is a trading rule, to check it I need to (independently?) model the whole distribution of the multivariate stochastic process that produces it, which is more complex. My question was: how can I claim that my Monte Carlo check is reliable if I need to _guess_ how to check instead of _knowing_ how to check?
jbs789 50 minutes ago [-]
It’s a more useful tool than no tool. We are talking about predicting the future after all…
Maro 7 hours ago [-]
The Unreasonable Effectiveness of Monte Carlo Simulations in A/B Testing
I (and Claude) have been working on an open source monte carlo based retirement tool for quite a while now. Its definitely geared towards people with more individual financial ownership and tech knowledge but I feel it gives good flexibility for me to model my situation. I had initially gravitated to this project because I think Monte Carlo and numerical methods are just so dang fun.
I was working on a retirement planning simulation this week with Opus and it added this simulation into my tool! Had never heard of Monte-Carlo before but was fairly impressed.
charlieyu1 24 hours ago [-]
If you are doing combinatorial sampling like estimating pi or doing probabilities with SET, stratification is a great idea
8 hours ago [-]
shoo 23 hours ago [-]
Here's a few related ideas:
1) rejection sampling - this is the name to the general technique used in the first example, to estimate pi by sampling from a uniform distribution over a unit square (easy) and then rejecting the samples that fall outside of the distribution you're actually interested in (unit disk).
Another simple example of rejection sampling is if you want to randomly pick between 5 ordered options and only have a d6. Roll the d6. If you roll a 6, reject the roll & re-roll. If you roll 1-5, that's your choice.
2) the Metropolis-Hastings algorithm -- this is a Markov chain Monte Carlo algorithm. It can be useful in some applications to sample from a distribution that is difficult to sample from.
Suppose you have some system described by a state vector x & you know that the probability of the system being in state x, p(x), is proportional to q(x), where q is some function you can compute. If you could integrate q over x, to give Z := integral_{x in X} q(x) dx then you'd have p(x) = Z^-1 q(x)
For a concrete example, x might be a state vector of a physical system -- say a bunch of molecules in a box, where you can compute the energy of any given state u(x) and know the probability of x, p(x), is proportional to exp(-u(x)).
Metropolis-Hastings is neat as it gives you a way to sample from p even if you cannot compute Z or sample from q --- you just need to have some way to sample a proposed a new candidate state x' from x (this proposal distribution doesn't need to have any relationship to the distributions p & q you're actually interested in). Can often think of the proposed candidate state x' as some kind of local mutation of x. Provided you can compute q(x') / q(x), you can use that ratio as a probability of accepting the move to replace x with candidate x'.
By the algorithm depending on the ratio of probabilities of two different states p(x') / p(x) = q(x') / q(x), it lets you sample from p even if you can't compute Z or have any direct way to sample from p or q. *
Metropolis-Hastings & Markov Chain Monte Carlo methods are definitely in the territory of clever 'what to do if you don't know what to do' methods --- if there's some other more direct & practical way of computing the thing you're trying to compute, use that instead! But if there's not, & you have some gnarly problem, maybe MCMC might be the best option available.
* caveat caveat caveat: provided you let the sampler run long enough for it to 'burn in', & perhaps restart it a few times from different initial states x_0 to reduce the chance of it getting trapped in some unrepresentative local optima
psunavy03 21 hours ago [-]
Ironically, there have been studies done in using Monte Carlo methods to predict software delivery timelines (Dan Vacanti and Prateek Singh's work specifically), that found out that the more mature and well-behaved your pipelines and test automation were, the more accurate a naive Monte Carlo simulation was.
The only teams where Markov Chain methods or messing with sampling dates would provide more accurate timelines were the teams whose delivery processes were the most borked and spiky.
stevenhubertron 1 days ago [-]
I use a Monte-Carlo for my retirement planning. This did a good job of telling me not to trust it but didn't really give me a better option. Anyone have one?
Jtsummers 1 days ago [-]
> This did a good job of telling me not to trust it
What do you mean? The retirement planning with Monte-Carlo gave a better result than the base assumption of 4% returns and 2% inflation. The information MC provides let's you, when taking into account your risk threshold, whether you're satisfied with your savings or not by giving you more realistic odds of how long the money will last.
lastofus 23 hours ago [-]
There's no sure fire option for making sure you don't run out of money, as nothing can 100% predict the future. The best we have is the assumption that the future will look kinda like the past over long periods of time.
If you acknowledge that assumption, then you have to choose what model of the historical market you want to use a predictor for the future.
You can use exact market data, test retiring out at every single possible year at every possible asset allocation and see what happens. This is the basis of the 4% rule from Bill Bingham.
You can pull random years out of history and string them together a bunch of times and see what happens with a given asset allocation. This is a naive Monte Carlo simulation.
You can pull blocks of years and string them together, to acknowledge that there are repeating trends in years (what happens in one years influences the next) and do a Monte Carlo simulation which is a bit less naive.
There are more esoteric models overly smart people have constructed, each of which will give you slightly different answers for a safe withdraw rate.
At the end of the day though we are all just crossing our fingers, and hoping for a black swan catastrophe to not happen.
sokoloff 22 hours ago [-]
I don’t think a pure Monte Carlo is as helpful as “replaying” past actual scenarios (or a hybrid of the two).
I don’t have reason to believe that 40 years in a row would all have randomly and independently chosen results from a distribution (pure Monte Carlo) as a better model than past actual results.
Spooky23 6 hours ago [-]
None of these things are oracles. The Monte Carlo helps you visualize the universe of outcomes. Simulating past events helps you specifically apply your knowledge of the past to your portfolio.
Think of it as a tool to understand and explore risk. I’ve found all of these tools more useful as a I get older and the day I need to use the cash is closer.
sokoloff 5 hours ago [-]
I agree that it’s a tool to explore risk. I just think it’s more useful if you pull the likely universe of outcomes from the sequences of the past (sliding your retirement across multiple starting dates) more than worry about what happens if we get multiple -43% years in a row. There’s no realistic portfolio that will survive that, but also no realistic underlying economic model where that happens (or if it does, where what you have saved really matters).
Spooky23 7 hours ago [-]
Why wouldn’t you trust it?
Usually critics of these estimates say stuff like “the simulation includes scenarios like the Great Depression”.
They give you a range of results that illustrate things like sequence of return risk, which most people have difficulty understanding and incorporating into plans. Many people have an underlying “stonks go up” assumption that may not serve them well.
fithisux 17 hours ago [-]
If you want to look further on this fascinating topic
the best book on the subject with a Coursera series of top quality
positron26 1 days ago [-]
> Can't differentiate the function? Fuck gradients. Here's the entire fucking posterior.
- Monte Carlo
coderenegade 20 hours ago [-]
It should also be said that "can't differentiate a function" is a pretty high bar for Monte Carlo methods, which will still give you a gradient for un-differentiable functions.
psunavy03 21 hours ago [-]
"We have no idea why the fuck these neutrons are doing what they're doing, because we haven't figured the physics out yet, but we realized we can still statistically simulate what they're going to do to a good enough level of probability."
That's literally why they invented Monte Carlo as part of the Manhattan Project.
fedecaccia 5 hours ago [-]
[dead]
Rendered at 22:28:01 GMT+0000 (Coordinated Universal Time) with Vercel.
If you enjoyed reading about how you can approximate complex statistical values using basic programming, I highly recommend Jake Vanderplas "Statistics for Hackers" [0][1].
It converts the sometimes hard to grok statistics terms and formulas into some very simple loops and basic operations.
Can't recommend it enough.
0 - https://speakerdeck.com/pycon2016/jake-vanderplas-statistics...
1 - https://www.youtube.com/watch?v=Iq9DzN6mvYA
> I know, I will use rand
> noo, do you even know a thing about randomness?
> okay, fine. I'll use arc4rand which uses a cryptographically-sound prng
> nooo! Random numbers are crap! you should use space-filling sequences that are guaranteed to work optimally for infinity dimensions
> okay
Billy, what are you rambling about? Wake up! We already have a result using RANDU that the verification process confirms is correct.
A few years after watching it for the first time I rewatched it and paused after every problem statement and tried implementing a “simple loops and basic operations” solution and failed almost every single time.
So be warned: he does make it look easy :)
"Before they had functions and lookup tables, what would a human have done in the 1940s?"
e.g. the statistical significance is just "if I were to randomly try this, what percentage of time would it be above 5%"
The idea was to start with a seemingly biased game—a race between a tortoise (rolling one die) and a hare (rolling 2 dice)—and to tweak the rules until both players had an equal chance of winning.
It was a fun experiment and the game actually works well!
https://github.com/sambeau/tortoise-vs-hare
I wrote Monte-Carlo simulations to calculate the thermodynamics of oxygen ordering in a high temperature superconductor way back in 1993.
Calculating thermodynamic fluctuations on a 256x256 lattice took all night. And that, even though it ran on an HP 9000/750 with a 66 MHz PA-7000 (PA-RISC 1.1) 32-bit CPU.
While waiting, I used to look at weather satellite images downloaded from CERN using NCSA Mosaic. It was impressive. All the information available at your fingertips by using arrow keys and pressing return on underlined text to navigate to the next page! Emmm, link.
https://github.com/rohan-sawhney/mcgp-resources
estimating pi: https://playground.probl.dev/#code=ZchBCoMwEEbhfU7x40pRyiS4k...
(I'm the author of probl)
The four attributes:
- Count: 3 in each; "all the same"
- Color: red, purple, green; "all different"
- Shape: diamonds, ovals, squiggles; "all different"
- Shading: empty, solid, striped; "all different"
So for each attribute the conditions are met, they are either all the same or all different.
Though such predictions are infamously bad at extrapolating without some strong theories behind the kind of distributions you could expect.
https://arxiv.org/abs/2411.06701
https://github.com/jgrazian/finplan
Try it live: https://finplan.rayknot.com
1) rejection sampling - this is the name to the general technique used in the first example, to estimate pi by sampling from a uniform distribution over a unit square (easy) and then rejecting the samples that fall outside of the distribution you're actually interested in (unit disk).
Another simple example of rejection sampling is if you want to randomly pick between 5 ordered options and only have a d6. Roll the d6. If you roll a 6, reject the roll & re-roll. If you roll 1-5, that's your choice.
2) the Metropolis-Hastings algorithm -- this is a Markov chain Monte Carlo algorithm. It can be useful in some applications to sample from a distribution that is difficult to sample from.
Suppose you have some system described by a state vector x & you know that the probability of the system being in state x, p(x), is proportional to q(x), where q is some function you can compute. If you could integrate q over x, to give Z := integral_{x in X} q(x) dx then you'd have p(x) = Z^-1 q(x)
For a concrete example, x might be a state vector of a physical system -- say a bunch of molecules in a box, where you can compute the energy of any given state u(x) and know the probability of x, p(x), is proportional to exp(-u(x)).
Metropolis-Hastings is neat as it gives you a way to sample from p even if you cannot compute Z or sample from q --- you just need to have some way to sample a proposed a new candidate state x' from x (this proposal distribution doesn't need to have any relationship to the distributions p & q you're actually interested in). Can often think of the proposed candidate state x' as some kind of local mutation of x. Provided you can compute q(x') / q(x), you can use that ratio as a probability of accepting the move to replace x with candidate x'.
By the algorithm depending on the ratio of probabilities of two different states p(x') / p(x) = q(x') / q(x), it lets you sample from p even if you can't compute Z or have any direct way to sample from p or q. *
Metropolis-Hastings & Markov Chain Monte Carlo methods are definitely in the territory of clever 'what to do if you don't know what to do' methods --- if there's some other more direct & practical way of computing the thing you're trying to compute, use that instead! But if there's not, & you have some gnarly problem, maybe MCMC might be the best option available.
* caveat caveat caveat: provided you let the sampler run long enough for it to 'burn in', & perhaps restart it a few times from different initial states x_0 to reduce the chance of it getting trapped in some unrepresentative local optima
The only teams where Markov Chain methods or messing with sampling dates would provide more accurate timelines were the teams whose delivery processes were the most borked and spiky.
What do you mean? The retirement planning with Monte-Carlo gave a better result than the base assumption of 4% returns and 2% inflation. The information MC provides let's you, when taking into account your risk threshold, whether you're satisfied with your savings or not by giving you more realistic odds of how long the money will last.
If you acknowledge that assumption, then you have to choose what model of the historical market you want to use a predictor for the future.
You can use exact market data, test retiring out at every single possible year at every possible asset allocation and see what happens. This is the basis of the 4% rule from Bill Bingham.
You can pull random years out of history and string them together a bunch of times and see what happens with a given asset allocation. This is a naive Monte Carlo simulation.
You can pull blocks of years and string them together, to acknowledge that there are repeating trends in years (what happens in one years influences the next) and do a Monte Carlo simulation which is a bit less naive.
There are more esoteric models overly smart people have constructed, each of which will give you slightly different answers for a safe withdraw rate.
At the end of the day though we are all just crossing our fingers, and hoping for a black swan catastrophe to not happen.
I don’t have reason to believe that 40 years in a row would all have randomly and independently chosen results from a distribution (pure Monte Carlo) as a better model than past actual results.
Think of it as a tool to understand and explore risk. I’ve found all of these tools more useful as a I get older and the day I need to use the cash is closer.
Usually critics of these estimates say stuff like “the simulation includes scenarios like the Great Depression”.
They give you a range of results that illustrate things like sequence of return risk, which most people have difficulty understanding and incorporating into plans. Many people have an underlying “stonks go up” assumption that may not serve them well.
[1]: https://austinrochford.com/posts/intro-prob-prog-pymc.html [2]: https://vincentk1991.github.io/Bayesian-regression-tutorial/
I can't recommend enough
[3]: https://academic.oup.com/book/54722
the best book on the subject with a Coursera series of top quality
- Monte Carlo
That's literally why they invented Monte Carlo as part of the Manhattan Project.