`` "Like other occult techniques of divination, the statistical method has a
private jargon deliberately contrived to obscure its methods from
non-practitioners."
- G. O. Ashley
Evolution of Information Theory
Information Theory is essentially the creation of one extraordinary
researcher, Claude Shannon, who also continued to shape the development of
this subject for more than a decade. To its shame,
Alcatel-Lucent, the present corporate owner of the venerable Bell
Labs, no longer maintains an old website with interesting
biographical information about Claude Shannon and about
the history of information theory. As an aside,
TelephonyOnline has a very interesting article about the history
and the future of Bell Labs:
No longer protected by AT&T's monopoly and guaranteed stream of
funding, Bell Labs depended on the fortunes of its parent company. After the
telecom bubble burst in 2000, Lucent saw its revenues plummet and it began
laying off staff companywide, including at Bell Labs. Those financial
problems continue to this day as the newly merged Alctael-Lucent fights to
regain its footing.
Fortunately other interesting reports about the history of information
theory exist, for e.g. here
and here
(the latter report is from MIT and claims to have input from some ex-Shannon
collaborators). For a technical account of the development of information
theory, refer S. Verdu, ``Fifty years of Shannon theory", IEEE Trans. on
Inform. Th., Oct 1998. Here is a
survey with comments about personal influences and memories of Claude
Shannon from several people.
Perhaps the most interesting reference on Shannon's work is by Prof. R.
Gallager, "Claude E. Shannon: a retrospective on his life, work, and
impact," IEEE Trans. on Inform. Th., Nov 2001.
Trivia
Here are a few trivia from the biographical literature about Shannon.
- No rough drafts. Information Theory was born
with the publication in 1948 of Claude Shannon's paper, ``
A Mathematical Theory of Communication" in the Bell Labs Tech.
Journal. The next year, this paper was republished under the bolder title
``The Mathematical Theory of Communication". Shannon had been working
on this paper since 1940, and it had so many novel ideas that it took only a
year after publication for its title to be upgraded to the definite article.
- Dissertation. The remarkable thing is Shannon's Ph.D
dissertation had nothing to do with communication theory. It was titled,
``An algebra for Molecular Genetics." Unfortunately Shannon never published
this work, and it remained unknown for a long time. It is included in the
book Claude
Shannon: Collected Papers, published by John Wiley.
- Shannon's heroes. (from Sloane and
Wyner's biography of Shannon) ``... His childhood hero was Edison, who
he later learned was a distant cousin. Both were descendants of John Ogden,
an important colonial leader and the ancestor of many distinguished people.
Shannon's later hero list, without deleting Edison, includes more academic
types such as Newton, Darwin, Einstein and Von Neumann."
- `Entropy'? Information theory folklore holds
that John von Neumann convinced Shannon to call his uncertainty measure
as `entropy', in part because ``nobody knows what entropy really is
and so you will have the upper hand in any debate" (see for e.g. this
Wiki
entry). Actually this story is false, as Shannon himself testified
(see R. Price, ``A Conversation with Claude Shannon", IEEE Commun.
Mag., May 1984).
- Funny Einstein story. (from Prof. Arthur Lewbel's tribute
to Shannon) ``... The story is that Claude was in the middle of giving
a lecture to mathematicians in Princeton, when the door in the back of
the room opens, and in walks Albert Einstein. Einstein stands listening
for a few minutes, whispers something in the ear of someone in the back
of the room, and leaves. At the end of the lecture, Claude hurries to
the back of the room to find the person that Einstein had whispered too,
to find out what the great man had to say about his work. The answer:
Einstein had asked directions to the men's room."
Back to Home