Measuring Creativity: A Case Study Probing Rubric Effectiveness for Evaluation of Project-Based Learning Solutions

Renee M. Clary, Robert F. Brzuszek, C. Taze Fulford

.

DOI: 10.4236/ce.2011.24047                

Abstract

This research investigation focused upon whether creativity in project outcomes can be consistently measured through assessment tools, such as rubrics. Our case study research involved student-development of landscape design solutions for the Tennessee Williams Visitors Center. Junior and senior level undergraduates (N = 40) in landscape architecture design classes were assigned into equitable groups (n = 11) by an educational psychologist. Groups were subsequently assigned into either a literary narrative or abstract treatment classroom. We investigated whether student groups who were guided in their project development with abstract treatments were more likely to produce creative abstract design solutions when compared to those student groups who were guided with literary narrative interpretations. Final design solutions were presented before an audience and a panel of jurors (n = 9), who determined the outstanding project solutions through the use of a rubric, custom-designed to assess the project outcomes. Although our assumption was that the measurement of the creativity of groups’ designs would be consistent through the use of the rubric, we uncovered some discrepancies between rubric score sheets and jurors’ top choices. We subjected jurors’ score sheets and results to a thorough analysis, and four persistent themes emerged: 1) Most jurors did not fully understand the rubric’s use, including the difference between dichotomous categories and scored topics; 2) Jurors were in agreement that 6 of the 11 projects scored were outstanding submissions; 3) Jurors who had directly worked with a classroom were more likely to score that class’ groups higher; and 4) Most jurors, with the exception of two raters, scored the abstract treatment group projects as higher and more creative. We propose that while the rubric appeared to be effective in assessing creative solutions, a more thorough introduction to its use is warranted for jurors. More research is also needed as to whether prior interaction with student groups influences juror ratings.

Keywords

Creativity Assessments, Rubric, Creativity Measurement, Rubric Consistency, Problem Based Learning

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Black, P. C. (2007). The mississippi story. MS: University of Mississippi Press.
[2]	Csikszentmihalyi, M. (1996). Creativity: Flow and the psychology of discovery and invention. New York: Harper Collins.
[3]	Delisle, R. (1997). Use Problem-Based Learning in the Classroom. Alexandria. VA: Association for Supervision and Curriculum Development.
[4]	Fulford, C. T., Brzuszek, R., Clary, R., Gainer, D. & Lackey, B. (in press). Influencing the story: Literal versus abstract instruction. Proceedings of the Council of Educators in Landscape Architecture (CELA) 2011 Urban Nature.
[5]	Guildford, J. P. (1959). Three faces of intellect. American Psychologist, 14, 469-479. doi:10.1037/h0046827
[6]	Guildford, J. P. (1986). Creative talents: their nature, use and development. Buffalo, NY: Bearly Ltd.
[7]	Guildford, J. P. (1988). Some changes in the Structure-of-Intellect model. Educational and Psychological Measurement, 48, 1-6. doi:10.1177/001316448804800102
[8]	Isaksen, S. G., & Treffinger, D. J. (1985). Creative problem solving: the basic course. Buffalo, NY: Bearly, Ltd.
[9]	Johnson, R. L., Penny, J., & Gordon, B. (2000). The relation between score resolution methods and interrater reliability: An empirical study of an analytic scoring rubric. Applied Measurement in Education, 13, 121-138. doi:10.1207/S15324818AME1302_1
[10]	Jonnson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity, and educational consequences. Educational Research Review, 2, 130-144. doi:10.1016/j.edurev.2007.05.002
[11]	Markham, T. (2011). Can we really teach creativity? ASCD Edge. Association for Supervision and Curriculum Development. http://edge.ascd.org/_Can-we-really-teach-creativity/blog/3577828/127586.html
[12]	Nasar, J. L., & Johnson, E. (1990). The personality of the profession. Landscape Journal, 9, 102-108.
[13]	Niaz, M. (1993). Research and teaching: Problem solving in science. Journal of College Science Teaching, 23, 18-23.
[14]	Neuendorf, K. A. (2002). The content analysis guidebook. Thousand Oaks, CA: Sage.
[15]	Osborn, A. F. (1963). Applied Imagination (3rd ed.). New York: Scribner’s.
[16]	Parnes, S. J. (1981). Magic of your mind. Buffalo, NY: Bearly Ltd.
[17]	Shepherd, C. M., & Mullane, A. M. (2008). Rubrics: The key to fairness in performance based assessments. Journal of College Teaching and Learning, 5, 27-32.
[18]	Shores, M., & Weseley, A. J. (2007). When the A is for agreement: Factors that affect educators’ evaluations of student essays. Action in Teacher Education, 29, 4-11.
[19]	Starko, A. J. (1995). Developing creativity in the classroom: Schools of curious delight. White Plains, NY: Longman Publishers.
[20]	Sternberg, R. J. (2010). Teach creativity, not memorization. The Chronicle of Higher Educaiton. http://chronicle.com/article/Teach-Creativity-Not/124879/
[21]	Tan, O.-S. (2008). Problem-based learning and creativity. Cengage Learning Asia.
[22]	Taylor, A. R., Jones, M., & Broadwell, B. (2008). Creativity, inquiry, or accountability? Scientists’ and teachers’ perceptions of science education. Science Education, 92, 1056-1075.
[23]	Wiggins, G., & McTighe, J. (2005). Understanding by design. Alexandria, VA: Association for Supervision and Curriculum Development.
[24]	Yager, R. E. (2000). A vision for what science education should be like for the first 25 years of a new millennium. School Science and Mathematics, 100, 327-341. doi:10.1111/j.1949-8594.2000.tb17327.x
[25]	Yin, R. K. (2008). Case study research: design and methods (4th ed.). Thousand Oaks, CA: Sage Publications.