Based on Item Response Theory, a theory frame to determine validity of teaching evaluation scores was developed, then rater leniency and rater self consistency from Rasch model were selected to determine validity. An example illustrated how to use rater leniency and rater self consistency of Rasch model to determine validity. Data collected from Rasch model indicated that leniencies of some raters were significantly different and self consistency of some raters were not good, then some student evaluation scores were valid but other student evaluation scores were invalid. Then, contributions and limitations in the paper were discussed.