The use of peer assessment through a multisource feedback process has gained recognition as a reliable and valid method to assess the characteristics of professionals and trainees. A total of 168 first-year medical students completed a 15-item questionnaire to self-assess their professional work habits and interpersonal abilities. Each student was expected to identify 8 first-year classmates to complete a corresponding 15-item peer assessment. Although the self and peer assessment questionnaires had strong reliability (Cronbach’s α = 0.85 and 0.91, respectively), an exploratory factor analysis resulted in a 3- and 2- factor solution, respectively. The third factor was associated with items related to students’ personal attributes. Significantly lower mean score differences for the self-report assessment were found for all 15 items (Cohen’s d = 0.27 to 1.39, p < 0.001). A decision study analysis found that 7 peer assessors were needed to achieve a generalizability coefficient of 0.70. The findings suggest some inconsistencies in regards to the construct validity and stability of measures between self and peer assessment measures. The need for self-awareness of students’ strengths and limitations, however, is recommended as part of their development in a profession that emphasizes self-regulation.