Determining the Difficulty and Discrimination Parameters of a Mathematics Performance-Based Assessment ()
1. Introduction
Student achievement in mathematics has been hindered by various challenges related to assessment (Gao, 2012; Nortvedt & Buchholtz, 2018; Suurtamm et al., 2016). Two challenges are “a focus on recall of isolated items of knowledge… [and] inadequacy in aligning assessment tasks with students’ real-life situations” (Gao, 2012: p. 63). Performance assessment, also called performance-based assessment (PBA), addresses these concerns in ways that traditional assessments do not. On a PBA, examinees perform the actual skills that the test was designed to measure (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014) . In contrast to most traditional tests, PBA requires students to think critically and apply their knowledge to create a response or a product. Furthermore, PBA incorporates real-world tasks and contexts, thus making the assessment more relevant to students than multiple-choice tests (Wiggins & McTighe, 2005). Implementing performance assessment in mathematics classes not only enhances the quality of instruction and improves student achievement (Mulana et al., 2021; Stone & Lane, 2003), the practice motivates students in their learning as well (Arhin, 2015; Balik, 2012).
The use of performance assessment comes with its own set of challenges. For decades, experts have reported that “the usual suspects” of administering PBA on a large scale are costs and scoring subjectivity (e.g., Darling-Hammond & Adamson, 2010; Madaus & O’Dwyer, 1999; Stiggins & Bridgeford, 1984; Wren & Gareis, 2019). Another problem is the typical lack of psychometric evidence to support the use of PBA in educational settings (Davey et al., 2015; DiCerbo et al., 2016; Stecher et al., 2000). A related issue is developing performance assessments that are appropriate for students with diverse ability levels, as “it is quite conceivable that a well-designed performance assessment could be administered and yet fail to provide useful data if the assessment is delivered at a level that is either too difficult or too easy for the student being assessed” (Bahr, 2007: p. 34). Performance assessments are seldom analyzed to determine the degree of difficulty or how well they discriminate between students at different proficiency levels.
Classical test theory (CTT) has served the educational and psychological measurement community admirably for over 100 years, but CTT models have shortcomings due to the theory’s assumptions. One problem associated with CTT is that item statistics—specifically difficulty and discrimination indices—are dependent on the general ability level of the sample of examinees employed during the test development process (Hambleton, 2000). For example, a homogeneous sample of high-ability examinees might yield data that indicates the items are easier and less variant than if data from a heterogeneous sample was used to develop the test. Item response theory (IRT), which appeared on the educational and psychological measurement scene much later, involves the relationship between examinees’ ability levels and their responses to test items. For this reason, contemporary researchers and practitioners in the measurement field tend to prefer IRT models to calculate item difficulty and item discrimination parameters.
2. Methodology
The aim of the study was to apply item response theory to determine the difficulty and discrimination parameters of items on a performance-based assessment developed by the primary author. The PBA comprised five mathematics computation items presented in real-life scenarios. Each item was designed to assess the proficiency of senior high school (SHS) students in these math domains: transformation, descriptive statistics, mensuration, geometric construction, and linear equations. Item 4, the geometric construction item, is shown in Figure 1.
A total of 750 SHS students in the Western Region of Ghana were administered the PBA. The sample consisted exclusively of SHS 3 students (i.e., students in their third and final year of high school) from two randomly selected classes at 15 high schools, stratified according to the Ghana Education Service’s SHS categorization system. There were five schools from each of the three highest SHS categories: A, B, and C. Senior high schools in the highest categories are government or public schools, which are considered the best schools in the traditionally competitive Ghanaian secondary school system.
Using a standardized scoring rubric developed by the primary author, three different examiners—one each for the A, B, and C categories—scored the students’ responses to the PBA items. Data were analyzed with Samejima’s (1969) two-parameter logistic (2PL) graded response model, a popular IRT model used to estimate the difficulty and discrimination levels of polytomously scored items such as those on performance assessments.
3. Results
Data analyses yielded values that indicated the discrimination and difficulty parameters of the PBA items (see Table 1). Discrimination, known in IRT as location, is represented by a, and difficulty, or slope, is denoted by b in the table. Although the theoretical range of both parameters in IRT is −4 to +4, ranges that are observed in practice are usually −2.8 ≤ a ≤ +2.8 and −3 ≤ b ≤ +3 (Baker, 2001).
![]()
Figure 1. Geometric construction item for mathematics performance-based assessment.
![]()
Table 1. Parameter estimates for mathematics performance-based assessment items.
a = item discrimination (slope), bi = item difficulty (location).
Baker’s (2001) cut-off ranges were used to determine how well the items discriminated between examinees with different levels of proficiency in each domain. The cut-off ranges were as follows: very low discrimination = 0.01 to 0.34, low discrimination = 0.35 to 0.64, moderate discrimination = 0.65 to 1.34, high discrimination = 1.35 to 1.69, and very high discrimination > 1.70. While the transformation and geometric construction items discriminated moderately between students at different points on the proficiency continuum, the descriptive statistics, mensuration, and linear equation items all demonstrated very high discrimination power.
Unlike CTT with its single-value approximations of item difficulty, IRT models provide varied estimates of an item’s difficulty depending on the estimated ability of examinees. In other words, the values at each level (bi) explain how an item performs along the ability scale. The statistical definition of item difficulty in IRT is the point on the ability scale at which the probability of answering the item correctly is 0.5 (Baker, 2001).
The b values in Table 1 show that each item performed as expected, as b increased progressively in value from the lowest ability level to the highest ability level, (i.e., b1 to b9). The items with the greatest range of difficulty were the transformation and linear equations items, with b1 and b9 values approaching the typical minimum and maximum b values of −3 to +3. The least difficult and most difficult items were the descriptive statistics and mensuration items, respectively. Overall, the functioning of items along the ability scale indicated that the difficulty level of every item was acceptable.
4. Recommendations
The results of the study have implications for improving classroom and large-scale mathematics assessments in Ghana. At present, the degree of difficulty and discrimination of items on SHS classroom tests is largely unknown. Well-designed performance-based assessments would provide better information about students’ proficiency in mathematics than the traditional assessments currently do. In addition, high-quality PBA items would enhance the core mathematics section of the high-stakes West African Senior Secondary Certificate Examination (WASSCE) by effectively discriminating between examinees who are proficient in a specific domain and examinees who are not.
Ghanaian mathematics teachers and assessors associated with the West African Examinations Council (WAEC) who have access to software for analyzing polytomous test items could use this study to guide them as they create and calibrate new PBA items for their tests. The WAEC’s vision is “To be a world-class examining body adding value to the educational goals of its stakeholders” (WAEC, 2022), so it seems logical that the organization would embrace the best assessment methods and statistical models for obtaining comprehensive psychometric evidence for items on the WASSCE and other assessments.
Other recommendations mirror those made by Arhin (2015) in earlier research conducted with SHS mathematics students in Ghana. After finding that performance assessment-driven instruction “had an encouraging effect on students’ attitude towards mathematics especially on students’ motivation, independent thinking and understanding in solving mathematical problems” (p. 114), Arhin suggested that SHS mathematics teachers include PBA tasks in their lessons. He also proposed that math teachers receive training on the use of performance assessment-driven instruction. Along the same lines, the authors of the present study recommend that PBA become an integral part of methods and assessment courses at colleges and universities where students are trained to teach mathematics. These recommendations pertain not only to Ghana, but to other countries where performance-based learning and assessment have not been fully actualized in secondary classrooms. Providing current and future educators with the tools and knowledge to develop lessons and tests that incorporate PBA would have a catalytic effect on mathematics education by “improving the quality, realism, and utility of instruction for students” (Baker, 2019: p. vii).
Acknowledgements
The authors would like to thank the heads of the senior high schools in the Western Region of Ghana that participated in the study, as well as Stephen Court for his help during the preparation of this paper.