This week I have been brushing up on noise. Why noise? This series deals with probability and statistics. Noise is related to probability. Random fluctuations are usually dismissed as noise. Many engineers devote their whole careers to fighting noise, knowing that in the end, noise will win. The struggle reminds me of Odin trying to delay the ultimate victory of the giants for as long as possible. Odin and we are all destined to die an entropy death.
One big trouble with noise is that most people are not careful in how they define it. For instance, at a large party, you might struggle to understand the conversation between you and your date, but it is difficult because of the noise of all the other people talking around you. What your date is saying is the desired signal. What other people are saying is noise. Of course the people standing near you who are trying to talk to each other and, thereby, contributing to your background noise, think your conversation is noise and their conversation is the signal.
I once showed an interesting chest radiograph containing an obvious cancer on the patient’s kidney to an orthopedic friend. I asked if he saw anything unusual. He looked at it for a moment and said that the patient was probably in his late fifties based on the vertebral compression and that he had the start of some arthritis. This good bone doctor did not see the cancer on soft tissue because that would normally be noise to him. I doubt if the original treating physician noted the vertebral compression. What is noise and what is signal often depends as much on what we want as on the system properties.
Once we have defined what we mean by noise in a given situation, the next thing to do is to characterize it. We often refer to “white noise,” but even that has some assumptions in it. True white noise with power spread equally over all possible frequencies would take an infinite power source. Since they are not readily available, we intuitively assume a noise spectrum that is weighted by some function. A gaussian bell-shaped weighting function is usually assumed to be the normal shape of a noise function. It is not.
The classic bell-shaped curve is only one of an infinite collection of possible white noise generators. In fact, even things that we normally assume are distributed by gaussian fashion often follow a similar curve which looks like a gaussian near the center, but has different wings. Noise in the price of stocks does not distribute as a gaussian.
Knowing that the gaussian shape is not always to be expected is important. In 1994, Richard J. Herrnstein’s book, The Bell Curve created a controversy in part because he does not seem to have understood possible distributions.
The errors encountered when transferring binary data from an HD are examples of non-gaussian impulsive noise.
Gaussian noise is generally additive, which means we can assume a signal and simply add noise. This type is relatively easy to deal with. Other types of noise such as the mottling on a photograph due to low light is multiplicative and often much more difficult to identify and minimize. Blurring is another multiplicative source of noise.
In general, noise is more interesting and complicated than we think. There is even a phenomenon known as “stochastic resonance” which is essentially adding noise to a weak signal to make it easier to detect! This happens, for instance, when we listen to faint sounds. We can actually hear faint signals better if there is a bit of noise present. This surprising result has some analogs in probability that we can explore another time. (Historically, astronomers would pre-expose glass plates with a uniform weak white light to increase sensitivity to the faint signals from the heavens. Is this the same thing?)
In response to the interest my original tutorial generated, I have completely rewritten and expanded it. Check out the tutorial availability through Lockergnome. The new version is over 100 pages long with chapters that alternate between discussion of the theoretical aspects and puzzles just for the fun of it. Puzzle lovers will be glad to know that I included an answers section that includes discussions as to why the answer is correct and how it was obtained. Most of the material has appeared in these columns, but some is new. Most of the discussions are expanded compared to what they were in the original column format.
[tags]statistics, probability, decision theory, noise, signal[/tags]