An Investigation of the State of Creativity and Critical Thinking in Engineering Undergraduates ()
1. Introduction
Engineers are in the business of creativity and innovation. In the last two hundred years, “more than half of the major life-altering, technological and social innovations introduced to the world came into being”, and engineers were responsible for a large portion of those technological breakthroughs ( Puccio & Cabra, 2010 ). Now more than ever, the world is in a state of continuous innovation, in need of engineers with the creative and critical thinking skills necessary to solve present and future challenges ( Tucker, 2001 ; Twohill, 2012 ).
The Partnership for 21st Century Skills, a consortium focused on infusing 21st century skills into education, conducted a study to determine which skills employers found most essential for performance in today’s workplace, with 382 to 409 employer respondents ( Casner-Lotto & Barrington, 2006 ). Based on the employers’ answers, the study gathered a list of twenty attributes and categorized them as either basic or applied skills. These included simple items such as reading comprehension, as well as more advanced abilities, such as leadership and social responsibility. The study found that creativity/innovation and critical thinking were some of the most desired skills for workplace entrants with four- year college diplomas. Specifically, both of these skill sets were ranked within the top ten. It is apparent that creativity and critical thinking are essential to workplace performance. This suggests that critical thinking and creativity must be given the same level of emphasis in engineering education as math and sciences to ensure engineers are ready for the 21st century workplace.
Innovation as a process requires both creativity and critical thinking in order to be successful. Theoretical frameworks on the innovation process generally distill into stages: idea generation, development of the concept, evaluation and selection of the concept, product development, and finally implementation ( Dreiling & Recker, 2013 ; Berry, Shankar, Parish, Cadwallader, & Dotzel, 2006 ; Nagji & Tuff, 2012 ). Critical thinking skills are valued in this framework, since multiple stages require the definition and understanding of problems, as well as defining the original problem the company needs to solve through innovation. Creativity also plays a key role in ideation steps within the innovation process, through the many stages of the product or service’s lifecycle. The innovation process for business is driven by these two critical skills. As such, it is essential to understand the state of both among college engineering students, and find methods to address any deficiencies found.
The business need for innovation shows the importance of teaching undergraduate engineering students how to be better creative and critical thinkers. Unfortunately, the literature in this area suggests critical thinking stagnation and creative decline in undergraduate engineers instead. This research further explored these areas of concern to determine the state of creativity and critical thinking in engineering undergraduates through a comparative study between incoming and outgoing engineering students.
This paper will explore the current state of creativity literature in pertinent areas, the hypotheses will be clearly stated, the methodologies undertaken to test the hypotheses will be discussed, the results of the experimental analyses will be presented and finally the results and their implications will be discussed.
2. Literature Review
2.1. What is Creativity?
Creativity was once considered a divine gift, sometimes completely disassociated from the person who developed the creative works. The Greeks thought creativity to be the result of a person’s daimon, or guardian spirit ( Runco & Albert, 2010 ). For much of history, creativity was considered the ability of a few, until research began to suggest it might possibly be a tool for all. Researchers found that creativity was not the sole ability of only privileged individuals, but the potential ability of anyone with a capacity to learn ( Andreasen, 2006 ; Gelb, 2000 ; Starko, 2014 ).
Creativity is difficult to define. Its reach is so wide and its implementation so broad that a complete definition is elusive within the research. Guilford (1967a, 1967b, 1959 ) provided one of the first definitions for research as part of his Structure-of-Intellect model, defining creativity in a similar way to other types of human thought. His is one of the first definitions; Starko (2014) and others point to a plethora of creativity research suggesting many different definitions of creativity across the psychology disciplines ( Parkhurst, 1999 ; Kozbelt, Beghetto, & Runco, 2010 ). The lack of a universal theory of creativity makes research measures difficult to reconcile.
Though the varied theories do not agree completely, there are consistent elements that appear in many historical and current studies supporting the research and study of creativity. The following exploration synopsis of literature is purposefully concise, as these topics can be extensive. The consistent elements seen throughout creativity theory are:
1) Connectedness, Emotional Thinking and Metaphorical Thinking
Connected and metaphorical thinking are the types typically associated with a creative individual’s ability to elaborate on ideas and synthesize. By using metaphor and analogy, typically unassociated ideas are related through an indirect path to form a further refined or completely new idea ( Mednick, 1962 ). Kohn et al. (2011) “argue[d] that people’s ability to combine categories is related to their ability to produce original, high-quality products”, which would make their resulting products creative.
In writing, metaphors are the most commonly found type of expression of novel ideas. Piirto (2004) found that the use of metaphor is an important part of the works of young creative writers. The power of ideational connection allowed by the use of metaphor allows for complex or unrelated ideas to be brought together in a novel form. Additional studies have looked at how metaphors progress with age and their relation to creativity ( Gardner & Winner, 1982 ).
Analogies are another simple way to generate and communicate novel ideas, as they provide a common starting point for others to understand the potential of a new concept. This can lead to incremental changes that in turn become innovations, or breakthrough ideas ( Chan & Schunn, 2014 ). Empathizing is also a studied approach which, like the use of analogies, provides a means for others to understand the innovation. The basis of this process is the human need to empathize with the feelings of others. By understanding how others may struggle with products or services, a creative individual can craft solutions that may totally redefine an industry ( Genco, Johnson, Holtta-Otto, & Conner Seepersad, 2011 ). By connecting ideas through metaphor, analogies and emotional connection, new concepts are generated.
2) Flexibility and Elaboration
Flexibility and elaboration are largely based on divergent thinking processes. Guilford (1950, 1967a, 1959) proposed the concept of divergent thinking based on his observations that creative people tended to utilize this style of thought more than others. Additionally, Guilford detailed divergent thinking in terms of fluency, flexibility, originality and elaboration. Divergent thinking is the process by which many ideas are generated based on a core concept. This type of thinking allows the exploration of multiple concepts, ideas and combinations without constraint ( Starko, 2014 ).
Creativity literature lends great credence to the concept of divergent thinking. Many creative assessments use divergent thinking concepts in their development, including the Torrance Test of Creative Thinking ( Torrance, 1988 ). Additionally, much of the literature of creative training also relies on divergent thinking as a means of improving creativity, and assessing its improvement or decline ( Cropley, 2000 ; Clapham, 1997 ; Clapham & Schuster, 1992 ; Runco, Millar, Acar, & Cramond, 2010 ; Kim, 2011 ; Cramond, Matthews-Morgan, Bandalos, & Zuo, 2005 ). Divergent thinking and its associated characteristics are a vital part of creativity; its value in determining creative ability has been verified through extensive proven correlations in the field.
3) Openness to Experience and Originality
Openness to experience is key to creativity, as demonstrated in those who have tendencies toward curiosity and imagination ( Feist, 2010 ). The acceptance of new experiences and novel concepts allow the individual to explore previously unknown ideas. These new experiences are catalysts to novel innovations, combinations and the development of new concepts.
Dollinger et al. (2004) compared several personality tests to the Test of Creative Thinking-Drawing Production (TCT-DP) in a study. The personality tests captured the attributes of individuals involved in the study, to determine how the different personality traits correlated with their creative assessments. The only personality dimension found with consistent correlation through the creativity assessment was the quality of openness. Many other studies have researched this very relationship and found similar results ( George & Zhou, 2001 ; Wutrich & Bates, 2001 ; Furham, 1999 ; Perrine & Brodersen, 2005 ).
The close connection between openness and creativity is a key trait for creative individuals. Based on its importance to the creative process, this trait proves useful in the work of creativity assessment, as it is an outwardly demonstrable trait.
4) Problem Finding and Problem Solving
Problem finding and solving are crucially important for the generation of creative ideas. Though generating many novel ideas can support creative solutions, the combination and association of non-obvious concepts is essential to creating creative products ( Root-Bernstein & Root-Bernstein, 2001 ). Studies throughout creativity research point to problem finding and problem solving as a basis of creative product generation ( Runco & Nemiro, 1994 ; Belski, 2009 ; English, 1997 ; Lai, Roan, Greenberg, & Yang, 2008 ; Ozyurt & Ozyurt, 2015 ; Wakefield, 1985 ).
Problem finding has been shown to be a “primary condition of creative performance” ( Wakefield, 1985 ). The results of a study on fifth graders’ ability to find novel solutions to math problems concurred with the finding of Wakefield’s study, as the participants’ novel approaches to solving the problems improved through training showing a link between creativity and problem finding ( English, 1997 ). Rostan (1994) found that the most creative individuals within another study dedicated the most time to problem finding during their creative exploration. Based on these and the other studies mentioned, problem finding and solving skills are a vital characteristics of the creative individual.
5) Willingness for Risk-Taking
In his seminal work, McClelland (1963) suggested that calculated risks form one essential aspect of scientific performance. The willingness to take chances is a cornerstone of creativity literature, due to the groundbreaking nature of creativity.
Many studies seek to determine the association between risk-taking and creativity. Tyagi et al. (2017) conducted two studies to explore the relationship between creativity and risk-taking. The results suggested risk-taking was the “strongest predictor of creative personality and ideations scores” ( Tyagi, Hanoch, Hall, Runco, & Denham, 2017 ). The strong relationship between risk- taking and creativity holds true across multiple other research efforts ( Eisenman, 1987 ; Pankove & Kogan, 1968 ; Sternberg, 2012 ; Steele, McIntosh, & Higgs, 2016 ). Based on the available literature in the field of creativity research, risk-taking is an individual characteristic closely related with both creative ability and production.
6) Curiosity
Curiosity, though a well-known trait, is still under investigation in terms of its underlying mechanisms and general cognitive theories. Litman (2005) defined curiosity as a desire or a want of previously unknown information. His paper likens it cognitively to other appetitive desires such as food. Based on this explanation, curiosity is an exploratory force driving creative efforts.
Other research on curiosity and creativity has yielded similar results. Conflicting viewpoints on whether curiosity drives creativity, creativity drives curiosity, or a combination of both, lead to further difficulties ( Harrison, 2016 ; Hunter, Abraham, Hunter, Goldberg, & Eastwood, 2016 ). Windahl (2017) suggests that curiosity begins the process of creative ideation and synthesis in market analysis. Alternatively, Litman (2005) proposes that curiosity forms a “motivational backbone” present throughout the life of creative ideas from their inception, through their adoption and dispersion. Though the interactions between creativity and curiosity are not understood, the literature clearly indicates curiosity is essential to creativity.
Based on the research suggesting these characteristics are essential to creativity, they may be a vital key to researching creativity. These cross-creativity theory elements are not singular in their effects. Each element influences the others intricately, yet may be individually studied based on creative production. As such, these elements will be used as the basis of creativity assessment in this study. Further in the discussion, the elements will be linked to the associated testing measures used by the Test of Creative Thinking-Drawing Production (TCT- DP), the creativity assessment instrument utilized in this study. This analysis will link the elements of the assessment to other known theories and long-used tests of creativity assessment.
2.2. State of Creativity in Engineering Undergraduates
Though industry demands creativity and innovation from the 21st century workforce, signs of negative trends in creativity appear in various studies ( Kim, 2011 ; Genco, Holtta-Otto, & Conner Seepersad, 2012 ). The findings of these studies suggest current pre-college and college education, including engineering education, have driven creative thinking down over many years to its present state. Based on the seriousness and urgency of this problem, special attention should be paid in determining its severity and how best to reverse the negative effects that have already occurred. One study used archival data for the Torrance Tests of Creative Thinking (TTCT) on a total of 272,599 kindergarten through twelfth grade students and adults in order to determine trends on their ability to think divergently ( Kim, 2011 ). The study shows a downturn in creative thinking skills (i.e. fluency, originality and elaboration within the TTCT) from 1966 through 1974, and from 1990 to the present. These results represent a significant loss of creative individuals capable of being trained in various technical careers that require innovation for true success. Though already troublesome, there is evidence this scenario is not improved by exposure to the engineering curriculum.
Genco et al. (2012) conducted a study on the innovation potential of undergraduate engineers. This evaluated students’ creative skills by asking for a new design for an alarm clock, a standardized real-world design problem. Through the use of such a ubiquitous object, this experiment lends itself to solutions demonstrating the negative effects of design fixation and expert bias ( Jansson & Smith, 1991 ; Woltz, Gardner, & Bell, 2000 ). Design fixation is the implementation of previously learned information or patterns in order to solve a problem, regardless of their applicability to the current dilemma. Though both groups (freshmen and seniors) had experience with alarm clocks in everyday life, seniors may have been at a greater disadvantage, as they would likely have a more developed knowledge of the internal workings and functions of an alarm clock as gained from their coursework. This may have driven them to less creative potential solutions, re-using common alarm clock design elements regardless of their actual design necessity. Researchers noted that senior-level students did indeed demonstrate abilities indicative of their more advanced engineering education, but lacked the more free-form creativity the freshmen displayed. The study concluded that, overall, freshman engineering students had a statistically significant advantage in terms of creative output. Other studies also point to similar conclusions of diminished creative abilities after completing engineering coursework ( Genco, Holtta-Otto, & Conner Seepersad, 2012 ; Kazerounian & Foley, 2007 ).
Atwood and Pretz (2016) discovered that creative engineering students are less likely to complete their engineering degrees. The authors suggest this results directly from the ineffectual way creativity is taught within the engineering curriculum. Student surveys also lend credibility to the findings of creativity research on those studying engineering. Kazerounian and Foley (2007) conducted a survey asking students and professors how specific Maxims in Creativity were addressed through their engineering coursework. These maxims included learning to fail, leading by example and the search for multiple answers. Engineering students largely felt that these creative maxims were generally not addressed within their coursework. These results concur with the research seen throughout the literature review. In many cases, engineering education does not support the creative processes the students require for their work outside academic life.
2.3. Evaluating Creativity
As seen throughout current studies, creativity is still a primary research topic. It is of great consequence to the human condition as it drives our knowledge acquisition process. As such, it is important both academically and socially to develop ways to evaluate the creative skills within individuals, as well as understand how that skill changes. Currently this is accomplished through a myriad of creativity assessments designed to examine the well-known attributes of creative individuals. As mentioned earlier, the Torrance Tests of Creative Thinking is a hallmark test in the study of creativity.
There is significant research as to the applicability and validity of the TTCT in reference to its use within creativity research. Some argue against its accuracy in determining creative skill on the basis of its heavy reliance on divergent thinking ( Zeng, Proctor, & Salvendy, 2011 ). Though divergent thinking is considered to be an important aspect of creative skill, it is not the only one. As such, the TTCT, with its emphasis on divergent thinking as an instrument of creative skill, would only give a partial picture of an individual’s creative potential.
Though the TTCT can provide a measure of creativity in terms of divergent thinking ability, and studies have shown it to be consistent with long-term creativity as judged by the participants, an approach is necessary that is more sensitive to overall creativity ( Cramond, Matthews-Morgan, Bandalos, & Zuo, 2005 ; Runco, Millar, Acar, & Cramond, 2010 ). Divergent thinking forms an essential but incomplete picture of creative ability ( Cropley, 2000 ). To this aim, Jellen and Urban (1986) created the Test of Creative Thinking-Drawing Production.
Carl Rogers’ theory of creativity makes up the primary basis of the TCT-DP. “Roger’s theoretical approach to creativity describes the nature of the creative act, the conditions under which it occurs, and the manner in which it may constructively be fostered” ( Jellen & Bugingo, 1989 ). As such, the TCT-DP allows each participant “to create, develop, expand and/or extend something unique or novel that is satisfying to the creator” ( Jellen & Bugingo, 1989 ). This allows for creativity to be witnessed in a more freeform medium, rather than a rigid construct such as the verbal portions of the TTCT. In addition, the TCT-DP attempts to capture more dimensions of an individual’s creativity ( Jellen & Urban, 1986 ).
The TCT-DP evaluates creative ability with the use of 11 + 4 criteria, developed as a complete analysis of an individual’s creativity through Continuations (Cn), Completions (Cm), New Elements (Ne), Connections made with a line (Cl), Connections made to produce a theme (Cth), Boundary breaking that is fragment dependent (Bfd), Boundary breaking that is fragment independent (Bfi), Perspective (Pe), Humor and affectivity (Hu), Unconventionality, a (Uc, a), Unconventionality, b (Uc, b), Unconventionality, c (Uc, c), Unconventionality, d (Uc, d), and Speed (Sp) ( Jellen & Urban, 1986 ). For this research, speed was not used as a measure within the analyses due to the limitations imposed by participant testing.
Traditional creativity measures focus on divergent thinking as one of the more important measures of creativity. These include aspects such as fluency, elaboration, flexibility and originality. Since traditional tests are primarily concerned with divergent thinking, some of the more whole-picture creativity measures set forth in the TCT-DP do not have direct links to the traditional measures. As such, Jellen and Bugingo (1989) developed a table linking the TCT-DP measures to traditional measures of creativity, and provided possible new measures for those without direct links to traditional research measures. Using this table and current creativity research as a reference, Table 1 provides a synopsis of the TCT-DP variables, associated mental processes and their links to traditional and current creativity research. The new measures are interesting, as they account
![]()
Table 1. Links and Definitions for TCT-DP Measures.
Note: aTraditionally categorized as flexibility. bNot traditional measures in the evaluation of creativity within current creativity literature, but suggested as new measures for creativity measurement by Jellen and Bugingo (1989) . Definitions and supporting literature have been generated by the researcher with reference to measurement intent and literature referring to these characteristics of creative individuals.
for the additional dimensions of creative individuals, such as risk-taking and sensitivity ( Starko, 2014 ).
The theoretical construct of the TCT-DP hinges on concepts supported by creativity research. The test itself demonstrates internal and construct validity through many studies ( Dollinger, Urban, & James, 2004 ; Cropley & Cropley, 2000 ; Rodrigues Virgolim, 2005 ). As such, several researchers found direct correlations between high test scores on the TCT-DP and high levels of real-world creativity ( Cropley & Cropley, 2000 ; Urban, 2004 ; Jellen & Bugingo, 1989 ; Leung, 2013 ; Danial, 2015 ). Urban (2004) also cites several other studies performed in multiple countries that have similar ties to real world creativity ( Togrol, 2012 ). Based on these studies developed using the TCT-DP, there is evidence for the overall reliability and validity of the test as an assessment of creative ability. The correlation between real world creative tasks and a high score on the TCT-DP suggests the test can support comparative studies, as the difference in scores can be tied to higher real-world creative outputs.
2.4. State of Critical Thinking
Critical thinking is a vital part of the engineering profession. Engineers are taught to analyze and think through problems from multiple perspectives in order to reach an appropriate solution. They must design useful items such as alarm clocks and vehicles based on critical requirements, material limitations, cost limitations and many unknowns. Most courses focus on teaching fundamental mathematical concepts and problem solving as a means to teach both engineering as well as critical thinking in problem solving ( Zhou, 2012 ; Badran, 2007 ).
Since critical thinking is important not only as a means to analytically solve problems, but also as a driver for creativity, it is quite important to understand the effects of the engineering curriculum on the critical thinking skills of undergraduates. Studies on critical thinking provide a less clear picture of the effects of engineering education on critical thinking skills. Douglas’ (2012) study found evidence that undergraduate engineering students performed better on a standard critical thinking assessment than graduate engineering students. The researcher noted the variance was likely due to graduate students answering fewer questions during testing. As such, it was likely there were no differences between the groups. Another study by Ozyurt and Ozyurt (2015) agreed with this result. Though this would suggest a reduction in critical thinking skills, other studies and the Watson-Glaser Critical Thinking Assessment (WGCTA) normative data suggest an improvement in overall critical thinking abilities as a result of involvement in both academic and extracurricular college life.
Research largely suggests that critical thinking improves as a result of college experiences, including both coursework and college life. Mines et al. (1990) studied how senior college students’ critical thinking abilities were affected by their college experience as compared to freshman college students. Researchers evaluated a group composed of freshman, senior and graduate participants using the Reflective Judgement interview, Watson-Glaser Critical Thinking Appraisal and Cornell Critical Thinking Test. The most relevant finding of this research was that seniors scored significantly higher than freshman students. When compared to the normative data of the general student populations for the WGCTA, the freshman engineers scored lower in critical thinking skills than their equivalent freshman norms, while the senior engineers scored at their respective normative average. Many other studies suggest this result as well ( Watson & Glaser, 2008 ; Pascarella, 1987 ; Keeley, 1992 ; Spaulding & Kleiner, 1992 ).
Some studies also suggest campus culture and out-of-class experiences may play a role in the improvement of critical thinking ( Tsui, 2000 ; Terenzini, Springer, Pascarella, & Nora, 1995 ). According to these studies, by participating in college courses and college life, students experience a significant improvement in their overall critical thinking abilities ( Pascarella, 1987 ; Terenzini, Springer, Pascarella, & Nora, 1995 ). Keeley et al. (1982) found that seniors provided more appropriate critiques of sample passages than freshman participants. Though the researchers noted that the seniors’ commentary still lacked some details, the level of critical thinking demonstrated was significantly higher than that of the freshman students. Another study found that, though critical thinking ability was not related to grade level, engineers showed high-level critical thinking skills when tested ( Ozyurt & Ozyurt, 2015 ).
Research indicates engineers as a group are more likely to have thinking styles inclined towards critical thinking, such as preferring “more highly prioritized thinking” ( Gridley, 2007 ). Additionally, several colleges are actively integrating engineering-based critical thinking instruction within their coursework ( Mokhtar, 2010 ). Since engineers typically focus more on analytically-driven fields such as physics and mathematics, this suggests very positive improvements in critical thinking abilities near the completion of their degrees, through both their experiences and coursework. The large repository of studies suggests that engineers should develop improved critical thinking skills as a result of their engineering education.
2.5. Assessing Critical Thinking Skill
Engineering education typically focuses on training students to think critically about a given problem, in order to forge solutions from experience and learned concepts. This is necessary for any engineer to be able to function within the realm of laws that govern physics and mathematics. Critical thinking is also an essential part of the creative thinking process, specifically in the process of synthesis ( Starko, 2014 ; Root-Bernstein & Root-Bernstein, 2001 ; Andreasen, 2006 ). Based on this, it was postulated that the critical thinking abilities of senior engineering students would be higher than that of the freshman students, due to the intensive coursework intended to hone their critical thinking skills (i.e. analysis, synthesis, etc.). To measure the difference in critical thinking abilities, the Watson-Glaser Critical Thinking Appraisal Form A was used for this study.
Measuring any cognitive trait requires accurate assessment instruments, capable of capturing the trait while resisting confounding from other factors. As a means to measure critical thinking in individuals, the Watson-Glaser Critical Thinking Appraisal Form A provides a consistent and simple means of gathering critical thinking data. The WGCTA Form A Manual details the use of this test on high school students, college students and business professionals. Since the test is applicable to a wide subset of late teens to adult individuals, the test is acceptable for use in assessing the critical thinking abilities of freshman and senior undergraduates, as done in this research. In addition to being age and level appropriate, there is strong evidence of reliability and validity of the test when data are appropriately evaluated and interpreted ( Watson & Glaser, 2008 ; Pascarella, 1987 ; Mines, King, Hood, & Wood, 1990 ; Wilson & Wagner, 1981 ).
As with all tests used for scientific study, test reliability and validity are essential to successful data interpretation. The WGCTA was chosen for these experiments based on its extensive history as a reliable and consistent measure of individual critical thinking skills. The WGCTA Form A Manual, as well as additional recent studies, find good reliability and validity evaluations in real world studies, as well as good correlation with other known tests of critical thinking ( Pascarella, 1987 ; Behrens, 1996 ; Gadzella & Baloglu, 2003 ; Watson & Glaser, 2008 ; Mines, King, Hood, & Wood, 1990 ).
2.6. Study Goals
This research expands the current knowledge by correlating the results of studies using non-standard methods of creativity and critical thinking assessment (i.e. judgment of creative works, GPA, SAT, etc.) and standardized tools. Since non-standard tools of assessment are difficult to compare and reproduce, the framework of this study provides a simple and easy-to-administer set of tools. This set of tools is necessary for the longitudinal studies which must be undertaken to gain a complete understanding of creativity and critical thinking.
Additionally, this study expands the understanding of critical thinking in engineering students. By exploring the critical thinking abilities of freshmen and seniors, trends requiring additional study can be identified and addressed.
3. Hypothesis
This study has two hypotheses. They are:
1) Senior engineering students will have lower levels of creative ability than the freshman engineering participants.
2) Senior engineering students will have higher critical thinking scores than freshman engineering students.
The literature review supports these two hypotheses. Several studies demonstrate a lack of creativity in the engineering curriculum ( Atwood &Pretz, 2016 ; Genco, Holtta-Otto, & Conner Seepersad, 2012 ; Kazerounian & Foley, 2007 ). The results of Weinstein et al. (2014) imply that individuals practicing within traditionally creative fields may not be experiencing the creativity decline seen by other studies. This suggests that the decline in creativity may be focused in fields traditionally thought of as less creative, such as engineering.
Though critical thinking may not be advanced through engineering coursework ( Douglas, 2012 ), many studies have shown that the exposure and advancement of studies positively affects critical thinking ( Mines, King, Hood, & Wood, 1990 ; Ozyurt & Ozyurt, 2015 ; Pascarella, 1987 ; Spaulding & Kleiner, 1992 ; Terenzini, Springer, Pascarella, & Nora, 1995 ). There is strong evidence to suggest that a pursuit traditionally linked with analytical thinking would trend toward improved critical thinking skills.
4. Methodology
4.1. Research Participants
Study participants were recruited from both an introductory course (EGS 1006C: Introduction to the Engineering Profession) and a senior-level course in engineering. The participants in the introductory course were first year freshmen in the College of Engineering, and provided pre-engineering education data regarding their creative and critical thinking capabilities through the testing (group FA). A total of 62 freshman engineering students volunteered to participate in the TCT-DP testing from the EGS 1006C course (nFA = 62). Of the 62 freshman engineering participants, 59 freshman engineering students completed the WGCTA (nFA, WGCTA = 59). Due to the multidisciplinary nature of the introductory course (i.e. multiple engineering major types), this sample is indicative of the base state of creativity and critical thinking in the average incoming engineering student, regardless of their field specialty.
Engineering senior students enrolled in a leadership senior-level capstone course were asked to participate in the study (group CA). A total of 112 senior engineering students volunteered to take the TCT-DP (nCA, TCT = 112). Of the 112 volunteers, 95 opted to take the WGCTA also. Only 86 completed the WGCTA (nCA, WGCTA = 86). Due to a delay in testing materials, the testing was postponed until a few weeks into the course, and some inadvertent creativity training took place through a combination of exposure to the project-based curricula, and the known creative expectations of the course ( Rietzchel, Nijstad, & Stroebe, 2014 ). Since the baseline creativity scores for the CA senior group were skewed from this sample, the baseline creativity scores used for comparison to the freshman group (FA) came from a separate set of senior engineering students (group WA). This group was given the same TCT-DP test as the CA senior group, for a separate but related study. These students were at the beginning of their senior design course, and had no prior knowledge of creativity training or testing taking place before the pretest was provided. A total of 68 senior engineering students volunteered to participate in the TCT-DP testing from a workshop conducted as part of a capstone course in engineering (nWA = 68).
These courses enroll students from a wide variety of engineering majors. Participants were randomly requested to participate from the capstone course in order to gather a sample more indicative of the general population of engineering students, as seen in the introductory freshman course. This allowed for a sample with adequate representation.
4.2. Procedure
A group of undergraduate engineering freshman and senior students were asked to participate in a study investigating creativity and critical thinking. Study participants were informed that their participation in the study was completely voluntary, would not have any effect on their coursework, positive or negative, and that they could request to be removed from the study at any time. Consent forms were distributed, thoroughly discussed and clarified before obtaining verbal consent and continuing with the testing. Participants were not asked for identifiable data. Only age and gender were collected in order to perform demographic analyses. Additionally, participants were notified that if they decided to no longer participate in the study, they could turn in their testing material and it would be destroyed.
After giving consent, study participants completed a test with two distinct parts. The first section was an assessment of creativity through the Test of Creative Thinking-Drawing Production. The TCT-DP uses an incomplete drawing as a starting point, and asks study participants to complete the drawing in whatever manner they wish. Their completed drawings were then evaluated based on 11 + 4 unique factors, which together formed a complete evaluation of the creativity of the individual. The test took a maximum of fifteen minutes for students to complete.
The second section of the testing asked participants to complete the Watson-Glaser Critical Thinking Appraisal. The WGCTA provides an assessment of the logical and critical thinking capabilities of the participants. The appraisal assesses critical thinking skills through questions of logic, including inference and the evaluation of arguments. This test required approximately forty minutes to complete.
Both assessments were bundled together as a single testing package and given to participants with sections that collected the necessary identifiers. Assessments of freshman and senior students were administered as similarly as possible within the constraints of testing. Senior testing was conducted in a classroom environment with an entire class. The freshmen were tested in groups as well, but in a non-class setting with fifteen persons or less at a time.
4.3. Data Analysis
The study utilized a correlational design. The TCT-DP data was analyzed and compared between the two population groups. The same analyses were performed on the WGCTA data. This study tested the relationships between engineering education, critical thinking and creative abilities. Additionally, several other variables were tested for additional effects, such as age and gender.
TCT-DP forms were evaluated using the criteria provided by the test publishers. Three raters were trained, using randomly generated samples as a group. These raters included an associate professor in the College of Engineering and two graduate students. The raters were then provided with a completely randomized set of forms for blind evaluation. After an initial round of evaluation, scores were evaluated for rater agreement. Nearly half the tests were sent for regrading, with no mention of the source group or the degree to which raters were set apart. After re-grading, the test scores approached high levels of agreeability. As such, scores from all three raters were averaged, and the average scores were used for statistical analysis.
The Intraclass Correlation Coefficient (ICC) was calculated for the raters’ combined scores to determine the degree of agreement between the raters. Each testing group’s data were evaluated using a two-way random effects model to calculate the ICC. The two-way random effects model could be utilized effectively, as the evaluation had a consistent set of raters, and the raters were randomly chosen from a pool of possible raters (i.e. other graduate students) ( McGraw & Wong, 1996 ; Shrout& Fleiss, 1979 ).
The TCT-DP and WGCTA test data gathered were analyzed using Multivariate Analysis of Variance (MANOVA) tests. MANOVAs were evaluated based on a degree of certainty, α, in the 95th percentile. Data determined to be non- normal was analyzed using non-parametric analysis techniques.
With the combined data analyzed, comparative analyses were performed on the sub scores of both the TCT-DP and the WGCTA for each group. The comparative analyses were conducted in order to determine which sub scores were significantly affected by the independent variable: time spent in the engineering curriculum. Since the data is ordinal due to the rating system used on both the TCT-DP and the WGCTA, a non-parametric statistical analysis compared the sub scores between groups. The data were analyzed using a Mann-Whitney U Test, the non-parametric equivalent of a two-sample T-Test. The Mann-Whitney U Test results were interpreted based on a degree of certainty, α, in the 95th percentile.
5. Results
5.1. Participant Demographics
Participants were instructed to provide gender and age information at the commencement of all testing. Participants consisted of freshman students from EGS 1006C (Introduction to the Engineering Profession), and senior students from EGS 4624 (Engineering Leadership and Innovation) and an engineering capstone course. Though instructions were given before testing to provide gender and age on the TCT-DP, some participants did not provide all necessary information. As such, the gender and age analysis of the data were limited to the participant data for which the demographic information was provided. For analysis requiring age, all individual data not containing gender information was excluded from the sample. For analysis requiring both, all participant data without gender and age information was excluded for the purposes of that specific analysis. No evaluations were found that omitted gender but provided age; therefore, no data were removed for gender alone. It is important to restate that the data were only excluded on the specific analyses that required gender and/or age; other analyses which only required TCT-DP or WGCTA score data were evaluated using all available data.
The participant demographics provided by the participants are detailed below in Table 2. Though the age and gender information is assumed to be indicative of the overall population, not all participants provided all requested information. Two-way ANOVA analyses were performed on all groups studied to determine
![]()
Table 2. Participant demographics and test completion.
the effects of age and gender on TCT-DP score.
The second portion of testing was focused on critical thinking assessment through the WGCTA. This test was taken by a subset of those who volunteered for the TCT-DP testing. Incomplete WGCTA tests were removed from the analysis.
A two-way ANOVA was performed on the senior participant WGCTA data (CA). The results provided no significant evidence that age (F = 1.522, P-Value = 0.099) or gender (F = 0.84, P-Value = 0.772) influenced critical thinking scores. Though the age effect did not reach the significance level being considered in this testing (α = 0.05), there are signs of a trend in this data.
A two-way ANOVA analysis was also performed on the freshman (FA) WGCTA data. Like the senior results, the two-way ANOVA showed neither gender nor age had a significant effect (F = 0.22, P-Value = 0.639; F = 0.21, P-Value = 0.808). Due to the limited range of ages in the incoming freshman participants, no significant effects were expected in terms of age. Based on the analyses of the demographic data, the data represents an adequate sample in terms of gender and age.
5.2. TCT-DP Evaluation and Agreement
Both single measure and average measure ICCs are reported. The average measures report the overall agreement between the three raters who scored the tests for the studies covered in this analysis. The overall agreement of the raters directly lends validity to the results of the study. The single measures detail the rater reliability expected of the TCT-DP based on the provided rating instructions.
The single measures ICC results for the pre-training creativity course seniors (ICCCA, SM = 0.866), freshman participants (ICCFA, SM = 0.811) and pre- training creativity workshop seniors (ICCWA, SM = 0.877) show almost perfect agreement between raters (ICC > 0.80). The average measures ICC results for the pre-training creativity course seniors (ICCCA, AM = 0.951), freshman participants (ICCFA, AM = 0.928) and pre-training creativity workshop seniors (ICCWA, AM = 0.955) also showed strong agreement between raters (ICC > 0.80). This provides strong evidence of consistency and agreement between the three raters.
Based on these statistics, the study data are sufficiently robust to draw conclusions from the results.
5.3. Creativity Testing
A MANOVA statistical analysis was performed on the incoming freshman data (FA) and the senior data (WA). As mentioned in the previous section, unforeseen factors prevented the pre-training creativity course senior participant sample group (CA) from being considered as the unaltered senior engineering student condition to compare against the freshman student group (FA). The creativity course senior participant sample group (CA) scored significantly higher, by an average of seven points, than the creativity workshop senior participants (WA) on the TCT-DP (see Figure 1). As such, the conditions of the CA group’s creativity testing were revisited and it was found that due to testing material delays, the students were not given testing until six weeks after the start of their course. Though this portion of the course did not contain explicit creativity training, the students were informed of the upcoming creativity portion of the course, as well as the potential of participating in a creativity study. It is posited that the creativity course group underwent inadvertent training as a result of their exposure to creative expectations, and were given implicit permission to be creative as a result of the anticipated course content. As such, the pre-training creativity workshop senior group (WA) was used as the comparative senior condition for the creativity portion of this study against the freshman group (FA), as this group did not have potential exposure to inadvertent creativity training before their testing.
A MANOVA statistical analysis was conducted between the senior participant data (WA) and the incoming freshman participant data (FA). There was strong evidence that the freshmen students performed better in the sub score of “Evaluation of Arguments” (F = 5.084, P-Value = 0.026). The statistical analysis also shows a trend that the freshman group (FA) had higher “Inference” scores than the senior group (WA) (F = 3.159, P-Value = 0.078). This trend suggests that freshman scores (FA) may be higher than those of the creativity workshop seniors (WA) before creativity training was provided.
Comparative analyses were performed to determine which creativity sub score characteristics influenced overall test scores the most between the freshman
![]()
Figure 1. Creativity course senior engineering students (CA) & creativity workshop senior engineering students (WA).
![]()
Table 3. Comparative analyses of TCT-DP sub scores between freshman (FA) and senior (WA) participant groups.
group (FA) and the senior group (WA). There was strong evidence that freshmen earned significantly higher scores in Ne, Cl, Cth, Pe and Ucd. Senior engineering students’ scores demonstrated lower levels of divergent thinking (Ne) and synthesis thinking (Cl and Cth) skills, as well as some impact on their ability to produce novel solutions to problems (Ucd) as compared to the freshman group. This is detrimental in terms of overall creativity, and directly impacts the seniors’ abilities to produce original solutions to problems. This may be due to several factors, including the methodologies used in traditional engineering education. Table 3 details the comparative analyses performed.
5.4. Critical Thinking Testing
The WGCTA scores of the pre-training creativity course seniors (CA) were compared to the incoming freshman (FA) WGCTA scores using a MANOVA test. It is postulated that neither senior group was exposed to critical thinking training, and therefore the critical thinking performance of the pre-training creativity course seniors (CA) and creativity workshop seniors (WA) were considered to be equivalent. The results did not provide enough evidence to reject the hypothesis. There was no statistically significant difference between the two samples, suggesting that there was no significant critical thinking change between incoming freshman students, and seniors nearing completion of their engineering coursework (F = 1.054, P-Value = 0.306).
In fact, the comparative analyses showed two sub scores where freshman participants performed better than senior participants. A statistically significant difference was found for the evaluation of arguments (F = 5.084, P-Value = 0.026). Table 4 shows the detailed results for the WGCTA sub scores. There is a trend (P-Value = 0.072) suggesting freshman engineering students out-performed senior engineering students in inference as well. Based on the test descriptions provided within the manual, this suggests the freshmen were better able to discriminate “among digress of truth or falsity of inferences drawn from data”, as well as distinguish “between arguments that are strong and relevant” ( Watson & Glaser, 2008 ).
The WGCTA manual provides normative data for the groups tested in this study. To better understand how the engineering students in the study group compared to the general population of students, statistical tests were conducted against the related norms. This allowed for the comparison of the “expected” average of the various samples (incoming freshman and upper division students in four-year colleges, respectively), and the actual data collected for both the engineering freshman and senior participants ( Watson & Glaser, 2008 ). Comparing the sample data to the normative data provided in the WGCTA manual, there was a statistically significant difference between both groups and their respective norms. The results of the statistical analyses are detailed in Table 5.
The incoming freshman engineers scored significantly higher than the average student score within in their normative group. This is an understandable difference, as the norm takes the entire student population into account. Those wishing to be engineers are presumed to have some innate problem-solving capabilities that would allow their average to be above the general population’s average.
![]()
Table 4. Comparative analyses of WGCTA sub scores between senior (CA) and freshman (FA) participant groups.
![]()
Table 5. T-test results for freshman (FA) and creativity course senior (CA) WGCTA scores against respective norms.
![]()
Figure 2. Study participant WGCTA data vs. WGCTA manual normative data.
The study freshman group’s average is at the 70th percentile (freshman average) when compared to the normative average (50th percentile). The senior participants, on the other hand, scored significantly lower than their corresponding normative group (50th percentile, norm average), at slightly above the 35th percentile (senior average). Figure 2 shows these comparative analyses graphically.
6. Discussion
Creative and critical thinking is essential components of the engineering profession. With these characteristics in action, engineers push boundaries and develop the improved tools and products that improve our lives. By understanding how these vital skills are approached in undergraduate degree programs, we can better deduce how graduates will perform in the real world, and course-correct should it be necessary. This study sought to understand if senior engineering undergraduate students were less creative but had higher critical thinking capabilities than freshmen undergraduate engineering students. To this end, the study exposed some expected issues but also other effects that were not so immediately apparent.
6.1. Senior Student Inadvertent Training
It was postulated that the two senior groups (CA and WA) were initially assumed to be similar in creative and critical thinking abilities based on their similar development level within their academic careers. Considering the scores of these two groups should have been within small deviations of each other, the differences must have been caused by outside factors. The creativity course seniors (CA) were administered the pre-training creativity test six weeks after the course began, due to a delay in testing materials. Though no specific creativity training was given during this time, the course was taught by the same professor who also taught the later creativity portion. The students were made aware of future creativity content, as well as the creativity testing. Some studies suggest creative expectation and creative permission can play a large role in creative training. This appears to have had a large influence on the pre-training TCT-DP scores of the affected senior group (CA).
Rietzchel et al. (2014) found that when asked to be creative, participants generated more creative products, though they felt it was not very creative. The expectation and anticipation of being creative are postulated as the reason for the majority of the difference between the scores of the two senior groups, which should not otherwise have been significantly different. Other studies also point to mere creative expectation and anticipation as a means of creativity improvement in study participants ( Kelley, Littman, &Peters, 2001 ; Scott & Bruce, 1994 ; Tierney & Farmer, 2004 ; Shalley, 1995 ; James, Hartman, Stebbins, & Jones, 1977 ; Chima, 2011 ; Starko, 2014 ; Twohill, 2012 ).
6.2. Creative Thinking in Engineering Undergraduates
This study provides strong evidence that incoming freshmen were more creative than senior engineering students, as measured by the TCT-DP. The results suggest a decline in creativity between the freshman and senior years of engineering students. This could be due to several possible scenarios: the engineering curriculum had adverse effects, freshman students with creative skills moved away from engineering, a change of technologies, educational drift, changing economic conditions or some combination thereof. Though there are many potential reasons for these changes, all scenarios are detrimental to the engineering profession. Studies show evidence of a definite lack of creativity training within the engineering curriculum, as well as extensive evidence of attrition from engineering majors to other programs ( Kazerounian & Foley, 2007 ; Daempfle, 2003 ; Astin & Astin, 1992 ; Marra, Rodgers, Shen, & Bogue, 2012 ; Shuman, Delaney, Wolfe, Scalise, & Besterfield-Sacre, 1999 ).
Steps must be taken to understand the reasons behind this trend of creative decline in engineering students. Longitudinal studies on the set academic assessments used in this study could provide vital information regarding the root causes of creativity decline. Determination of these causes could provide opportunities to redirect the education of young engineers, and increase their ability to think more creatively. Additionally, this study provided evidence that even small steps in creativity training (i.e. the effect of mere creative expectation on the CA senior group’s scores over the WA senior group’s scores in pre-training testing) can provide significant improvements in creative ability.
6.3. Critical Thinking in Engineering Undergraduates
The study of the critical thinking scores provided no evidence that senior participants demonstrated higher critical thinking performance than freshmen. This result suggests either there is inadequate support for developing the critical thinking skills of engineering undergraduates, or that those who are more capable of critical thinking are lost through attrition.
The results of the engineering students’ WGCTA scores in comparison to the normative data scores of the general population shed light on an even larger concern than the study results. Analysis showed the freshmen (µ = 57.76, SD = 8.33) entered their engineering education significantly above their normative group’s average (µ = 53.8, SD = 9.2), up to the 70th percentile of the norm for critical thinking abilities. On the other hand, the senior group’s scores (µ = 56.16, SD = 9.78) were significantly below those of their normative group (µ = 59.2, SD = 8.4) of the general population of upperclassman students, down to the 35th percentile of the norm.
The data suggests that critical thinking was stagnant between the two groups, and degraded when compared to normative data. As with the decline in creativity, this may be due to a host of factors. Though a comparison of seniors and freshmen is not a one-to-one comparison of critical thinking, the data trends suggest additional work must be done in order to understand how critical thinking is evolving in engineering students.
6.4. Discussion on Critical and Creative Thinking in Engineering Undergraduates
This study provides further support of several findings in the existing literature, but also further develops our understanding of creativity and critical thinking skill development in engineering students. That creativity is on the decline in engineering undergraduates, is apparent. The experiment also further confirmed the critical thinking stagnation findings of Douglas (2012) and Ozyurt and Ozyurt (2015) , but revealed additional insights when comparing this study’s data to normative data provided by the WGCTA. Critical thinking skills were significantly lower in engineering senior students as compared to students in other majors. This finding is significant, as it suggests the problem-solving capabilities of engineering students is not improving over the course of their training, or that students with skill in critical thinking abandoned engineering during the course of their program of study. This significant problem must be addressed through the development of critical thinking instruction, coursework development, and ultimately a restructured curriculum in order to better meet the needs of engineering students. Without these vital skills, engineers are unprepared and ultimately at a large disadvantage when faced with the challenges of a constantly- evolving world.
Longitudinal studies must be undertaken to identify and address deficiencies, in order improve creative outputs and critical thinking in engineering students. Focused courses addressing creativity and critical thinking can provide a stop- gap measure to address these issues in the short-term, but more long-term solutions are ultimately necessary.