How vs. Why

licensed from PresenterMedia

I am a big believer in "why". My parents could have told you that from the time I was very small. I never accepted an order to follow an algorithm, and because my father was a math professor, I never had to. He was always able to explain to me the "why" of mathematical formulas and algorithms, often using manipulatives, arrays, or diagrams to make it make sense.

Fast forward to today, taking a statistics class. My dad is no longer available to explain the "why" to me, and everything in the textbook, the lecture notes, on Wikipedia and on YouTube is about the how. This week I learned how to calculate an expected frequency from a cross-tabulation of data. But why is that the expected frequency? It seems logical to me that any calculation that relies only on the data collected is inherently skewed by that data, and could be fairly easily manipulated by adjusting the sampling.

I also learned this week that in order to construct a confidence interval of a proportion, I use z=1.96 or z=2.58, depending on whether I'm looking for a 95% or 99% confidence interval. Why? Where did these magic Z's come from? What do they represent? I understand the whole idea of confidence intervals, I am just frustrated that there is an element in my algorithm that I can't personally account for! Maybe I just need to see it like pi, and know that it just IS. But even pi I got to experiment with using string to prove to myself that it just is...

I can certainly get by running algorithms and using software to construct analyses and tests, and maybe I need to resign myself to that. Maybe it takes a degree in statistics to get to the "why". The sad thing is that if I understood the why, I would always be able to self-check the results of the algorithm to see if they make sense. Without the why, I have to just check my math, and assume that the results are logical.