Why Empathy Fails
In order to empathize with another person, you have to recognize that he actually exists apart from and without specific reference to you. You must understand that she has a distinct identity and an interior life of her own, with which you might possibly empathize. While there are some interesting exceptions to this rule, it's a useful place to start a discussion of why some people can't empathize, or why their ability to feel empathy is severely restricted. We can look at the spectrum of psychological disorders in terms of ability to recognize and tolerate separateness, then understand the ways that this ability will limit our capacity to feel for other people.
In psychotic disorders, for example, where splitting and projection dominate, other people serve as containers for disowned parts of our own psyche. As a result, we may try to avoid them -- annihilate them, in the most profound cases; or if idealization is involved, we may want to merge with and "own" them instead (think celebrity stalkers, for instance). But because we're so busy projecting into them, we can't see those people for who they really are; we can't empathize with their internal experience. People who present with autism symptoms famously lack the ability to empathize. Autistic defenses seek either to shut out the external world because it feels too threatening (shell-like defenses) or to obscure and erase personal boundaries because separateness is intolerable (confusional defenses); the awareness of other people as separate and distinct is therefore severely compromised. Empathy is virtually impossible.
Lack of empathy is a primary diagnostic criteria for narcissistic personality disorder, as well. For these individuals, other people serve mainly to provide narcissistic feedback or to contain our shame. If you exist (in my personal universe) merely in order to admire me, then I can't see who you are or how you feel. If you exist so that I may feel superior to you (the container for all my projected damage), I likewise can't empathize with your actual feelings. Even in less troubled people, narcissistic behavior of various kinds reflects a limited ability to recognize or take an interest in others, restricting the capacity to empathize with them. This is an experience we've all had in our everyday lives: the self-absorbed friend who talks on and on about herself, or dumps his problems into us, with no interest in us or our feelings. (I discussed this dynamic in my post on the toilet function of friendship.)