The Exponential Power Series
I am a big fan of the Poisson distribution, there’s something about its simplicity and elegance (I really like \(\lambda\)s) that makes it way easier to deal with than some monstrosity like the gamma or normal distribution.
Another thing I am a big fan of is the book I am currently reading: Surely You’re Joking, Mr Feynman. In one of the chapters, Dr. Feynman discusses his interest in the power series expansion of \(e^x\), and how efficient it is. Naturally I agreed with him, because this same power series expansion is the formula from which the Poisson distribution is derived. Let’s show this real quick:
Power Series and the Poisson
First let’s start with the power series:
\[\begin{aligned} e^{\lambda} = & \sum_{n = 0}^{\infty} \frac{\lambda ^ n}{n!} && \\ e^{\lambda}/e^{\lambda} = & \sum_{n = 0}^{\infty} \frac{\lambda ^ n}{n!} /e^{\lambda}&& \text{divide by }e^{\lambda}\\ 1 = & \sum_{n = 0}^{\infty} \frac{e^{-\lambda}\lambda ^ n}{n!}&& \text{simplify }\\ \end{aligned}\]
We can see that by simply dividing the power series representation by \(e^{\lambda}\) (aka multiplying by \(e^{-\lambda}\)) we get a series that sums to one, when we have a series (or integral) that comes out to one, that (usually) means we’re dealing with a distribution function. This idea of making a probability distribution out of basically any series that sums or integrates to one is a trick that will come in massively handy anytime you’re doing any sort of probability theory work. Simply find the constant and divide by it to switch back and forth between a series/ probability. In this case it’s the Poisson. To get the probability mass function (since we are dealing with the nicer discrete case) we simply take a single step of this function, aka \(\frac{e^{-\lambda}\lambda ^ n}{n!}\).
This means that any probability associated with a specific count for a poisson distributed variable is simply a scaled step in the power series expansion of \(e^{\lambda}\)! This is the kind of stuff that gets me excited.
Investigating the power series
Let’s go back to the power series and investigate its form:
\[\begin{aligned} e^{\lambda} = & \sum_{n = 0}^{\infty} \frac{\lambda ^ n}{n!} = 1/1 + \lambda/1 + \lambda^2 / 2 + ... \end{aligned}\]
We see the first term is always 1 You can use the fact that the first term in this power expansion is one to reason about why something raised to the zero power is one and not zero. and then after that we get a series of terms that change in size following a nice pattern. To get the \(i^{th}\) step we simply take the \((j - 1)^{th}\) step and multiply it by \(\lambda\) and divide it by \(j\) (\(s_{j+ 1} = s_j\lambda/j.\)) It’s quite beautiful in its simplicity. By carrying this process out an infinite number of times our sum will converge to the true value of \(e^{\lambda}\), and rather rapidly at that. Recognizing patterns like this one can result in massively sped up algorithms. For instance the naive way to calculate this series would be to perform n factorial calculations, where as if you use the iterative pattern described you only ever have to multiply and divide at each step.
In order to demonstrate this I have put together a little interactive that allows you to fiddle with the exponent for \(e^{\lambda}\) and the number of times you carry out this sum operation and see how well the series approximates the true value.
The bar plot shows the size of each step, aka how big is \(\lambda^i/i!\), above this is the sum of all of the bars, aka the series estimate of \(e^{\lambda}\) at the number of steps you have specified.
Note that as you increase the exponent the number of steps you need to get a good estimate increases with it, why might this be?
Do you have a favorite series or mathematical construct? Let me know!
Addendum
I really am a large fan of the Poisson distribution, so much so I even have it permanently written on my body.
Finally did it. Guess im stuck with this statistics thing now. #poissonPmf pic.twitter.com/DtTPzOiErd
— Nick Strayer (@NicholasStrayer) August 19, 2017