The neural representation of self is a fundamental question for brain research. Employing activation likelihood estimation (ALE) meta-analyses, we assessed the commonalities and distinctions between different components of the self by focusing on the ‘physical’ self and the ‘psychological’ self – assessed respectively through face processing and self-referential tasks. We first conducted ALE meta-analyses by computing the convergence of findings on brain activation in self-face recognition and self-referential studies respectively. Contrast and conjunction analyses of these two meta-analytic results were then applied to extract the distinctions and commonalities in self-face and self-reference tasks. Facial self processing was particularly associated with lateral brain regions with a right hemispheric dominance, while processing psychological self predominantly activated cortical midline structures, more specifically the anterior cingulate cortex/superior frontal cortex. In contrast, the conjunction analyses showed that the two aspects of self-processing recruit the dorsal anterior cingulate cortex and the left inferior frontal gyrus extending to the insula. A framework including both distinct and common neural representation of selfs is discussed.