A famous communication theorist

I was reading a paper on MIMO recently, when I noticed that one of the authors was Maher Arar.  Surely it can't be that Maher Arar, I thought to myself. But in fact it is: here he is getting a PhD in EE from the University of Ottawa.

Congratulations, Dr. Arar! He even has a nice blog.

Pigeonholes and entropy, a work in progress

The pigeonhole principle is probably the simplest lemma in combinatorics:

• Say you want to sort k objects into n bins.  If n < k, then at least one bin must contain at least two objects.  

For example, a person can have only one of 366 possible birthdays (counting Feb 29).  By the pigeonhole principle, any gathering of 367 or more people must have people who share a birthday.

The lemma is obvious almost to the point of triviality, so it comes as a surprise that you can use it to prove powerful results; there are some examples in the book.

Lately I've been wondering whether it can be used to say anything interesting about entropy.  Here's the first thing I thought of: let X and Y be random variables on a discrete alphabet, and let S(X) and S(Y) represent the support of the probabilities of X and Y, respectively (i.e., x is in S(X) if and only if p(x) > 0). Then

• Theorem. If |S(X)| > |S(Y)|, then H(X|Y) > 0.
• Proof. By definition of entropy, H(X|Y) >= 0. If H(X|Y) = 0, then there must exist an injective map from S(X) --> S(Y).  However, since |S(X)| > |S(Y)|, no such map exists (by the pigeonhole principle).  Thus, H(X|Y) != 0, and the theorem follows.

 Not very exciting, I will admit.  I'm still thinking about it, any other ideas?

Bad web strategies: A short problem set

Question 1. Consider this tweet:

What is the target of the link?

a) The ComSoc blog.

b) A link to a screenshot of the ComSoc home page, highlighting the "Blog" button in the link bar.  The screenshot is part of a Facebook album, and clicking on the screenshot only navigates to the next picture in the album.  Being an image, text in the screenshot (including the blog URL) is not selectable for copy/paste.  From the link, there are no obvious ways to get to either the ComSoc home page or blog.  


Question 2. What is ComSoc's slogan?

a) "The world's leading membership organization for communications professionals"

b) "We have never heard of the internet before today"

What's a PhD worth at the finish line? On hiring committees

The blogosphere is roiling over the worth of a PhD.  Should PhD programs take all qualified students? Is the academic "meritocracy" a myth, i.e., are some PhDs more equal than others? Even an outlet as erudite as The Economist suggests doing a PhD might be a waste of time (the always-nameless correspondent calling her ecology PhD "pointless").

In all of these pieces, the worth of a PhD is measured by its convertibility into a tenure-track position. There's lots of ink to be spilled on this topic, and I don't have a grand, philosophical point to add about the worth of the PhD.  However, I'd like to talk about the "finish line" viewed by many PhD students: the hiring committee.

No, I don't want to "talk about" your exam

The college where I did my undergrad had the habit of printing your rank in your class along with your yearly grade report; the top student in each class got a small prize, usually a book. At the end of my third year, I was happy to read on my report that I ranked first out of thirty-something electrical engineers (it was a small college).

I left for the summer and returned for my senior year, but hadn't received my prize.  When I asked around, it turned out that the second-place finisher had complained to all our profs about his exam grades, and managed to get enough extra marks to take first place -- my book became his.

I realize it's a small, petty thing to remember after all these years.  But I think about it every time a student comes to my office wanting to "talk about" their mark on the final exam.

A look back

At the beginning of the year, I resolved to blog more seriously in 2010.  I didn't quite hit my goal of 52 blog posts, but I'm still pretty happy with how it turned out.

This blog will be on hiatus until the new year.  Happy holidays, and see you in 2011.

Two new papers on molecular communication

I just got back from lovely Cambridge, Mass., where I managed to sneak in a day to see some of the interesting talks at Bionetics before storming back to Toronto to cover my teaching schedule.  December in Boston, what more could you ask for?

I'm a coauthor on a couple of new molecular communication papers that just came out, one at Bionetics, and the other on arXiv:

• For Bionetics, my Ph.D. student, Nariman Farsad, and our collaborators, Satoshi Hiyama and Yuki Moritani of NTT DOCOMO, continued some of our earlier work on microchannels.  This was a short "work-in-progress" paper, where we considered the performance of active transport systems in a more realistic simulation environment.  Surprisingly, Brownian motion with drift usually does better than molecular motors -- probably because every molecule can start propagating right away, rather than waiting to be picked up by a motor. [N. Farsad, A. W. Eckford, S. Hiyama, and Y. Moritani, "Information rates of active propagation in microchannel molecular communication,” in Proc. 5th International Conference on Bio-Inspired Models of Network, Information, and Computing Systems, Boston, MA, USA, 2010. (pdf)]

• On arXiv (and also submitted to Trans. Info. Theory), K. V. Srinivas, Ravi Adve, and I extended Sachin's earlier work on Brownian motion with drift.  We're looking at molecular timing channels, where information is encoded in the transmission time x. The receiver sees the reception time y, where y = x + n, and where n is the first arrival time of a Brownian motion; thus, these are additive noise channels. It turns out that the first arrival time distribution of a Brownian motion with positive drift is given by the inverse Gaussian distribution (kind of an odd name at first glance, since it's not the reciprocal of a Gaussian PDF).  So a molecular timing channel in the presence of positive drift is an additive inverse Gaussian noise channel. There is a rich literature on the inverse Gaussian, which lets us get closed-form solutions and bounds for a number of important quantities, like ML detectors, probabilities of error, and capacities -- sadly, not as clean and nice as the solutions for the AWGN, but we do what we can. [K. V. Srinivas, R. S. Adve, and A. W. Eckford, "Molecular communication in fluid media: The additive inverse Gaussian noise channel," arXiv:1012.0081 (submitted to IEEE Transactions on Information Theory).]

You can also check out more of my molecular communication work here.