It seems to me there are three primary meanings to the statement, “You’re not listening to me!”

1. “You’re not listening to me.”
   You really did fail to pay attention, and notice what was said.
2. “You did not do what I want.”
   They said it, you understood but didn’t follow their idea, which they take as proof of not caring.
3. “You didn’t read my mind.”
   Come on, you must have known what I wanted.

There are probably more, but I think these are pretty common. Which have you heard? What have you said?