“Theory of mind,” the ability to make inferences about others' mental states, seems to be a modular cognitive capacity that underlies humans' ability to engage in complex social interaction. It develops in several distinct stages, which can be measured with social reasoning tests of increasing difficulty. Individuals with Asperger's syndrome, a mild form of autism, perform well on simpler theory of mind tests but show deficits on more developmentally advanced theory of mind tests. We tested patients with bilateral damage to orbito-frontal cortex (n = 5) and unilateral damage in left dorsolateral prefrontal cortex (n = 5) on a series of theory of mind tasks varying in difficulty. Bilateral orbito-frontal lesion patients performed similarly to individuals with Asperger's syndrome, performing well on simpler tests and showing deficits on tasks requiring more subtle social reasoning, such as the ability to recognize a faux pas. In contrast, no specific theory of mind deficits were evident in the unilateral dorsolateral frontal lesion patients. The dorsolateral lesion patients had difficulty only on versions of the tasks that placed demands on working memory.