98% chance to do something!!

Learn to play D&D indeed or just watch any D&D stream, twitch and youtube are full of choices. Many games will have repeated double crits or double fails, this is such an expectation even though no D&D game even has the 400 rolls where it would be expected to occur just once, that prior versions of D&D even made it a mechanic to confirm the crits/fails by hitting it again.

The devs have stated the RNG has been checked and it meets expectant probabilities, although they leave open that there could always be display processing errors. You will need some other thesis than broken RNG because the devs are on record stating they will no longer entertain reports of broken RNG (Chris in Oct/Nov postmortem - responding to the last time the forum threads raged on this topic)

No player and no guild has even with heavily botting has the capability to test that expectations are being met, because the activity required would be an automatic ban. Only Portalarium can do this testing, and they have confirmed it meets probabilities of very large numbers.

Probablity is a science of extremely large numbers, what that means is that 1/8000 odds not mean one out of eight thousand trials. In fact one of the tests of randomness is what is the magnitude of trials to achieve expectations with confidence (usually 95%) - and that literally is millions of trials. Random sequences by definition include short term sequences that might seem unusual by the very reason that random systems have no memory so such sequences must exist. The existance of these sequences can be tested to meet expectations but again only by Portalarium.

The odds of a crit on a d20 is 1/20 every time, it has no relevance that you just rolled a crit.

If you really want to learn more, study this NIST report

http://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-22r1a.pdf

My favorite test is to plot the sequence as an image. It should look like grey TV snow (for 50/50 odds), but even in TV snow if you zoom in you will see the image is not just alternating 1/0 - it would not be random if it was. Instead y0u will see streaks of black and white that clearly defy the odds of 50/50 because that is the very nature of randomness.

Now combining this with reporting bias, the forum posts will only ever say it crit fails all the time. Of course the person crit succeeding all the time will never post for fear it is indeed a bug, and the person achieving the odds thinks nothing is wrong and will never post for fear of getting shouted down that there experience is irrelevant because YOU failed.

 

Sorry, I missed it somehow. Answering now.

OK. Millions? Why not billions? Or not thousands? It is a fine tradition in this group to skip everything (formulas, calculations, results), and just "I have a private opinion". ;-) I calculated (and explained how) 1/8000. If you think that the confidence interval requires millions (you wanted 95% confidence, if I am not mistaken?), please show me numbers and explain how you got them.

Arlin gave me three links to convince me that statistics is very complex. It is. Some people think that when estimating I take too much liberties. I do. I will be happy to land at an order of magnitude within the correct answer (that is an interval between 1/10th and 10x of a true value). If somebody can produce better number, I would love to see them, and how that someone arrived to them.

BTW, I will skim through the paper you recommended (thank you for that) --- I have actually read Arlin's links too. The last one is about calculating the longest run for a given probability and number of trials. I am more interested in "given N trials how many runs of M or more is expected with a reasonable confidence?". Yet, the guy does all the right things, and on page #201 arrives to a formula:

1 ~=~ n * q * p ^ Rn

Where n is a number of trials, p is a probability of desired binary outcome (heads/tails, success/failure, and so on), q is (1 - p) (the probability of an opposite outcome), and Rn is an expected length of the longest run. From this formula we can solve for Rn, or for n. I did both.

This are super-simple functions in JavaScript (you can try them right in the debugger of your browser):

Code:

var r = function(p, n) {
         return -Math.log(n * (1 - p)) / Math.log(p);
       };
       v = function(p, r) {
         return 1 / (1 - p) / Math.pow(p, r);
       };

// test
console.log(r(0.05, 8000));
console.log(r(0.20, 20));
console.log(v(0.70, 8));

r(p, n) takes a probability of a desired outcome from 0 to 1, n is a number of trials. It returns an approximate length of the longest run.
v(p, r) takes the same probability as p, a desired longest run as r, and returns an expected number of trials to achieve it.

So what is the longest run of failures (5% = 1/20) when we do just 8000 trials? In fact, 3 as I estimated before. The program printed: 2.982877877692756. Amazing coincidence!

I complained above crafting 20 items, none of them exceptional at >20% success rate, and I deemed it "unlikely". Let's see what the formula says: 1.7227062322935722 --- so I could expect 1-2 (closer to 2) exceptional items, I got none, hence "unlikely".

Oh, totally forgot: the paper actually includes calculations of an expectation, and variance, and prediction intervals, and all goodies people learned to ask --- they are all there, go and calculate away to your heart liking. If you don't like author's math, please contact him, and open his eyes on errors of his ways. I can recommend the good beginning: "Stats is complex. Your math is simple..."

How to use v() above? Say I have a spell, which works with probability 31%. So it fizzles with probability ~0.7. It failed for me 8 times in the row. How many times should I invoke it to expect 8 failures in the row? v(0.70, 8) gives an answer: 57.822175185810146. So every ~60 attempts I am bound to see 8 failures in the row. It seems very plausible to me. I use this opportunity to stress "an order of magnitude" thing I mentioned before: if the true result is 50, 70, or even 10, or 200 --- it is in the same ballpark. If it is <6, or >600 (an order of magnitude about 60) --- yep, it is not an estimate at all. But the estimate of 60 looks correct because it corresponds to my observations in the game.

PS: if you read the paper to the end, you'll find the most interesting part of it --- how people perceive probability. A quote: "so there is some empirical evidence that the rarity of events in the neighborhood 0f .05 begins to set people's teeth on edge."