The ability to grasp emotional messages in everyday gestures and respond to them is at the core of successful social communication. The hypothesis that abnormalities in socio-emotional behavior in people with autism are linked to a failure to grasp emotional significance conveyed by gestures was explored. We measured brain activity using fMRI during perception of fearful or neutral actions and showed that whereas similar activation of brain regions known to play a role in action perception was revealed in both autistics and controls, autistics failed to activate amygdala, inferior frontal gyrus and premotor cortex when viewing gestures expressing fear. Our results support the notion that dysfunctions in this network may contribute significantly to the characteristic communicative impairments documented in autism.