Learn about Law of Large Numbers Video

Get it off your chest – be the first to comment on this video!

Learn about Law of Large Numbers

Learn about Law of Large Numbers Let’s learn a little bit about the Law of Large Numbers which is on many levels one of the most intuitive laws in mathematics and then probability theory but because it’s so applicable to so many things it’s often to misuse law or sometimes a slightly misunderstood. So that is just to be a little bit formal in our mathematics, let me just define it for you first and then we’ll talk a little bit about the intuition. So let’s say I have a random variable x and we know its expected value or its population mean. The law of large numbers just says that if we take a sample of n, observations of our random variable and if we would average all of those observations and let me define another variable it’s called that x of n with a line on top of it. This is the mean of n observations of our random variables. So it’s literally, this is my first observation so you could kind of say, I run the experiment once and I get this observation and I run it again I get that observation and I keep running it n times and then I divide by my number of observation so this is my sample mean. This is the mean of all the observations I’ve made. The Law of Large Numbers just tells us that my sample mean will approach my expected value of the random variable or I could also write as my sample mean will approach my population mean for n approaching infinity. And I will be a little informal with what is approach or what is convergens mean but I think you have the general 0145 that if I take a large enough sample here that I’m going to end up getting the expected value of the population as a whole. I think there are a lot of us that’s kind of intuitive that if I do enough trials that over large samples that the trials would kind of give me the numbers that I would expect to give in the expected value in the probability, all that. But I think it’s often a little misunderstood in terms why that happens and before I go into let me give you a particular example. So the Law of Large Numbers would just tell us that let’s say I have a random variable x is equal to the number of heads after 100 tosses of a fair coin, tosses or flips of a fair coin, the Law of Large Numbers just tells us—well, first of all we know what the expected value of this random variable is, it’s the number of tosses, the number of trials times probability of success of any trial so that’s equal to 50. So the Law of Large Numbers just says, if I were to take a sample or if I were to average the sample of a bunch of this trials so you know I get—I don’t now my first time I run this trial, I flip a 100 coins or of have a 100 coins in a shoe box and I shake this shoe box and I count the number of heads and I get 55, so that would be x1 then I shake the box again and I get 65 then I shake the box again and I get 45 and I do this n times and then I divide it by the number of times I did it. The Law of Large Numbers just tells us that this average, the average of all my observations is going to converge to 50 as n approaches infinity or for n approaching infinity. And I want to talk a little bit why this happens or intuitively why this is. A lot of people kind of feel that like, oh, this means that if a 100 trials that if I’m above the average that somehow the Laws of Probability are going to give me more heads or let a fewer heads to kind of make up the difference and that’s not quite what’s going to happen and that’s often called the gamblers policy. Let me differentiate it, I’ll use this example. So let’s say—let me make a graph and I’ll switch colors. Let’s say that this is—so let me make—this is n, my x axis is n, this is the number of trials I take and my y axis, let me make that the sample mean. And we know what the expected value is, we know the expected value of this random variable is it’s 50, let me draw that here, this is 50. So just going to the example I did, so when n is equal to—I just fly here. So my first trial I got 55 and so that was my average, I only have one data point. Then after two trials, let’s see then I have 65 and so my average is going to 65 plus 55 divided by 2 which is 60 so then my average went up a little bit then I had a 45 which will bring my average down a little bit. I won’t plot a 45 here, now I have to average all of this out, what’s 45 plus 65—let me actually just get the number to see you get the point. So it’s 55 plus 65 it’s a 120 plus 45 is 165 divided by 3 is—it’s 55 so the average go down back to 55 and we could keep doing this trials. So you might say that the law of large numbers tells us, okay, after we’ve done three trials, you’ve done three trials and our average is there. So a lot of people think that somehow, the gods of probability going to make it more likely that we get fewer heads in the future that somehow the next couple of trials are going to have to be down here in order to bring our average down and that’s not necessary the case. Going forward, the probabilities are always the same, the probabilities are always 50% that I’m going to get heads. It’s not like if I had a bunch of heads to start off with or more than I would have expected to start off with then all of sudden things would be made up and I’ll get more tails and that would be the gamblers fallacy that if you have a long streak of heads or you have a disproportionate number of heads that at some point you’re going to have a higher likely than of having a disproportionate number of tails and that’s not quite true. What the Law of Large Numbers tells us is that it doesn’t care let’s say after some finite number of trials your average actually—it’s a low probability of this happening but let’s say your average is actually up here, is actually at 70 like wow, we really diverge a good bit from the expected value. The Law of Large Numbers says, well, I don’t care how many trials this is, we have an infinite number of trials left and the expected value for that infinite number trials or especially in this type of situation is going to be this. So when you average a finite number that averages out to some high number and then an infinite number that’s going to converge to this, you’re going to overtime converge back to the expected value. And that was a very informal way of describing it but that’s what the large numbers tells you is that—and it’s an important thing. It’s not telling you that if you get a bunch of heads that somehow the probability of getting tails is going to increase to kind of make up for the heads. What is telling you is that over—no matter what happened over a finite number of trials no matter what the average is of our finite number trials, you have an infinite number of trials left. And if you do enough of them it’s going to converge back to your expected value and this is an important thing to think about. But this isn’t used in practice everyday with the lottery and with casinos because they know that if you a large enough samples and we could even calculate—if you do large enough samples what’s the probability that things are deviate significantly. But casinos and the lottery everyday operate on this principle that if you take enough people, sure in the short term or with a few samples a couple of people might beat the house but over the long term, the house is always going to win because of the parameters of the games that they’re making you play. Anyway, this is an important thing in probability and I think it’s fairly intuitive although sometimes when you see it formally explained like this with the random variables and that’s a little bit confusing. All that saying is, is that as you take more and more samples the average of that sample is going to approximate the true average or I should be a little bit particular. The mean of your sample is going to converge to the true mean of the population or to the expected value of the random variable. Anyway, see you in the next video.

Khan Academy

The Khan Academy is a not-for-profit organization with the mission of providing a high quality education to anyone, anywhere.

http://www.khanacademy.org/

Learn about Law of Large Numbers

Let’s learn a little bit about the Law of Large Numbers which is on many levels one of the most intuitive laws in mathematics and then probability theory but because it’s so applicable to so many things it’s often to misuse law or sometimes a slightly misunderstood. So that is just to be a little bit formal in our mathematics, let me just define it for you first and then we’ll talk a little bit about the intuition.

So let’s say I have a random variable x and we know its expected value or its population mean. The law of large numbers just says that if we take a sample of n, observations of our random variable and if we would average all of those observations and let me define another variable it’s called that x of n with a line on top of it. This is the mean of n observations of our random variables.

So it’s literally, this is my first observation so you could kind of say, I run the experiment once and I get this observation and I run it again I get that observation and I keep running it n times and then I divide by my number of observation so this is my sample mean. This is the mean of all the observations I’ve made. The Law of Large Numbers just tells us that my sample mean will approach my expected value of the random variable or I could also write as my sample mean will approach my population mean for n approaching infinity. And I will be a little informal with what is approach or what is convergens mean but I think you have the general 0145 that if I take a large enough sample here that I’m going to end up getting the expected value of the population as a whole.

I think there are a lot of us that’s kind of intuitive that if I do enough trials that over large samples that the trials would kind of give me the numbers that I would expect to give in the expected value in the probability, all that. But I think it’s often a little misunderstood in terms why that happens and before I go into let me give you a particular example. So the Law of Large Numbers would just tell us that let’s say I have a random variable x is equal to the number of heads after 100 tosses of a fair coin, tosses or flips of a fair coin, the Law of Large Numbers just tells us—well, first of all we know what the expected value of this random variable is, it’s the number of tosses, the number of trials times probability of success of any trial so that’s equal to 50.

So the Law of Large Numbers just says, if I were to take a sample or if I were to average the sample of a bunch of this trials so you know I get—I don’t now my first time I run this trial, I flip a 100 coins or of have a 100 coins in a shoe box and I shake this shoe box and I count the number of heads and I get 55, so that would be x1 then I shake the box again and I get 65 then I shake the box again and I get 45 and I do this n times and then I divide it by the number of times I did it. The Law of Large Numbers just tells us that this average, the average of all my observations is going to converge to 50 as n approaches infinity or for n approaching infinity.

And I want to talk a little bit why this happens or intuitively why this is. A lot of people kind of feel that like, oh, this means that if a 100 trials that if I’m above the average that somehow the Laws of Probability are going to give me more heads or let a fewer heads to kind of make up the difference and that’s not quite what’s going to happen and that’s often called the gamblers policy. Let me differentiate it, I’ll use this example. So let’s say—let me make a graph and I’ll switch colors. Let’s say that this is—so let me make—this is n, my x axis is n, this is the number of trials I take and my y axis, let me make that the sample mean. And we know what the expected value is, we know the expected value of this random variable is it’s 50, let me draw that here, this is 50. So just going to the example I did, so when n is equal to—I just fly here. So my first trial I got 55 and so that was my average, I only have one data point. Then after two trials, let’s see then I have 65 and so my average is going to 65 plus 55 divided by 2 which is 60 so then my average went up a little bit then I had a 45 which will bring my average down a little bit. I won’t plot a 45 here, now I have to average all of this out, what’s 45 plus 65—let me actually just get the number to see you get the point. So it’s 55 plus 65 it’s a 120 plus 45 is 165 divided by 3 is—it’s 55 so the average go down back to 55 and we could keep doing this trials.

So you might say that the law of large numbers tells us, okay, after we’ve done three trials, you’ve done three trials and our average is there. So a lot of people think that somehow, the gods of probability going to make it more likely that we get fewer heads in the future that somehow the next couple of trials are going to have to be down here in order to bring our average down and that’s not necessary the case. Going forward, the probabilities are always the same, the probabilities are always 50% that I’m going to get heads. It’s not like if I had a bunch of heads to start off with or more than I would have expected to start off with then all of sudden things would be made up and I’ll get more tails and that would be the gamblers fallacy that if you have a long streak of heads or you have a disproportionate number of heads that at some point you’re going to have a higher likely than of having a disproportionate number of tails and that’s not quite true.

What the Law of Large Numbers tells us is that it doesn’t care let’s say after some finite number of trials your average actually—it’s a low probability of this happening but let’s say your average is actually up here, is actually at 70 like wow, we really diverge a good bit from the expected value. The Law of Large Numbers says, well, I don’t care how many trials this is, we have an infinite number of trials left and the expected value for that infinite number trials or especially in this type of situation is going to be this. So when you average a finite number that averages out to some high number and then an infinite number that’s going to converge to this, you’re going to overtime converge back to the expected value. And that was a very informal way of describing it but that’s what the large numbers tells you is that—and it’s an important thing. It’s not telling you that if you get a bunch of heads that somehow the probability of getting tails is going to increase to kind of make up for the heads. What is telling you is that over—no matter what happened over a finite number of trials no matter what the average is of our finite number trials, you have an infinite number of trials left. And if you do enough of them it’s going to converge back to your expected value and this is an important thing to think about. But this isn’t used in practice everyday with the lottery and with casinos because they know that if you a large enough samples and we could even calculate—if you do large enough samples what’s the probability that things are deviate significantly. But casinos and the lottery everyday operate on this principle that if you take enough people, sure in the short term or with a few samples a couple of people might beat the house but over the long term, the house is always going to win because of the parameters of the games that they’re making you play.

Anyway, this is an important thing in probability and I think it’s fairly intuitive although sometimes when you see it formally explained like this with the random variables and that’s a little bit confusing. All that saying is, is that as you take more and more samples the average of that sample is going to approximate the true average or I should be a little bit particular. The mean of your sample is going to converge to the true mean of the population or to the expected value of the random variable. Anyway, see you in the next video.

Transcription by: Scribe4you Transcription Services