Validity and Reliability of an Instrument Measuring Eating Habits, Weight-Related Health Awareness and Body Weight Perception among Malaysian TVET Students ()
1. Introduction
The quality of any quantitative study depends directly on the quality of the instrument used to measure its variables. An instrument that is not valid or not reliable produces misleading data, inaccurate conclusions and misdirected interventions, regardless of how sophisticated the applied analysis may be. For this reason, the assessment of instrument validity and reliability is recognised as a fundamental step in survey research methodology (Cohen et al., 2018; Creswell & Creswell, 2018). Validity refers to the extent to which an instrument truly measures the construct it is intended to measure, while reliability refers to the consistency and stability of the scores it produces (Mohajan, 2017; Taber, 2018). Both properties must be demonstrated empirically rather than assumed, particularly when an instrument is adapted from a different cultural context or population (Boateng et al., 2018).
This requirement is especially pressing in adolescent health behaviour research in Malaysia. The high rate of overweight and obesity among adolescents, with approximately one in three adolescents affected (Institute for Public Health, 2022), demands high-quality local research on behavioural determinants such as eating habits, self-esteem and physical activity. Nationally representative studies have likewise linked socioeconomic circumstances to adolescent obesity, underlining the need for measurement tools that work across diverse Malaysian settings (Ahmad et al., 2018). Globally, excess body weight remains one of the most serious public health challenges of this decade (World Health Organization, 2024), while insufficient physical activity continues to rise among adults and young people alike (Strain et al., 2024). Behavioural mechanisms such as reward sensitivity in eating further complicate the picture and require carefully measured constructs (Sutton et al., 2022).
Most instruments for these constructs, however, were developed in Western contexts and for mainstream school or university populations. The Technical and Vocational Education and Training (TVET) population has almost no purpose-validated instruments, even though vocational students, especially those in culinary arts and bakery and pastry programmes, face a distinctive food environment that generic instruments may not fully capture. The TVET sector itself is a strategic pillar of human capital development in Southeast Asia, and its students face distinctive learning and career environments that merit dedicated research attention (Adnan et al., 2021). The absence of validated instruments for this population hampers the development of a local evidence base and the evaluation of health intervention effectiveness in vocational institutions.
Adapting an existing instrument is an efficient strategy for addressing this gap, but it demands a rigorous validation process. An instrument whose item wording has been modified, whose items have been added, or which has been translated into another language must undergo pilot testing to establish content and construct validity before it is used in the actual study (Tsang et al., 2017). Content validation is itself a structured process, typically involving expert judgement of the relevance and representativeness of every item, and it can be quantified through content validity indices (Almanasreh et al., 2019; Polit & Beck, 2006). Exploratory factor analysis is the customary method for assessing the construct validity of adapted instruments, with the suitability of the data evaluated through the Kaiser-Meyer-Olkin measure and Bartlett’s test of sphericity, and the contribution of each item evaluated through its factor loading (Hair et al., 2019; Shrestha, 2021). Internal consistency reliability, in turn, is usually assessed using Cronbach’s alpha coefficient (Cronbach, 1951; Taber, 2018).
This study aims to report the adaptation process and the validity and reliability evidence for an instrument measuring perception of excess body weight, eating habits, weight-related health awareness and perceived benefits of physical activity among Malaysian TVET students. Specifically, the study sets out three objectives. First, it describes the adaptation and item-screening process applied to the original instrument for the Malaysian vocational context. Second, it assesses the construct validity of the instrument through exploratory factor analysis in a pilot study. Third, it assesses the internal consistency reliability of each construct using Cronbach’s alpha coefficient. By documenting this process and evidence transparently, the study provides a ready-to-use instrument for researchers and institutions that wish to study the health behaviour of vocational students, thereby contributing to the development of health research capacity in the Malaysian TVET sector.
2. Literature Review
2.1. Validity and Reliability in Instrument Development
Validity and reliability are the two psychometric pillars that determine the quality of a measurement instrument. Content validity assesses the extent to which the items represent the construct domain comprehensively, construct validity assesses the extent to which the instrument reflects the theoretical structure of the construct, and internal consistency reliability assesses the extent to which the items within one construct move in the same direction (Boateng et al., 2018; Mohajan, 2017). Best practice in scale development and validation prescribes a clear sequence of steps, beginning with item generation and review, followed by pilot testing, factor analysis and reliability assessment (DeVellis & Thorpe, 2021; Hinkin, 1998; Tsang et al., 2017). Comprehensive reviews of scale development research similarly emphasise that decisions taken at each stage, from item pool construction to factor retention, must be justified and reported so that later users can evaluate the instrument on its merits (Worthington & Whittaker, 2006).
Cronbach’s alpha coefficient remains the most widely used index of reliability, although its interpretation must be made carefully, taking into account the number of items and the dimensionality of the construct (Cortina, 1993; Streiner, 2003; Taber, 2018). Alpha values of 0.70 and above are generally accepted as satisfactory for social science research, while values approaching 0.80 and above are considered good (Ghozali, 2018; Hair et al., 2019; Nunnally & Bernstein, 1994). Classical treatments caution, however, that a high alpha is a property of the scores in a particular sample rather than a fixed property of the instrument itself, which is why reliability should be re-estimated whenever the instrument is applied to a new population (Streiner, 2003; Taber, 2018).
2.2. Content Validity and Expert Judgement
Content validation precedes any statistical testing and rests on systematic expert judgement. Long-established guidance recommends that a panel of experts rate the relevance of every item to the construct definition, with quantification through the content validity index at both item and scale level (Davis, 1992; Lynn, 1986). Subsequent methodological reviews have refined these procedures, clarified the interpretation of index values and warned against common reporting ambiguities (Almanasreh et al., 2019; Polit & Beck, 2006). Practical primers tailored to health professions education set out step-by-step calculation procedures that make the approach accessible to applied researchers (Yusoff, 2019), and instrument design studies illustrate how content validation is implemented within a full development cycle (Zamanzadeh et al., 2015). For the Malaysian setting, local validation studies in education have demonstrated the value of combining expert review with subsequent empirical screening, providing precedents that the present study follows (Lee Abdullah & Wei, 2017; Sidek Mohd Noah, 2002).
2.3. Exploratory Factor Analysis in Construct Validation
Exploratory factor analysis (EFA) is widely used to assess the construct validity of newly developed or adapted instruments. Two preliminary tests determine the suitability of the data for factor analysis: the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy, with values of 0.70 and above indicating good suitability (Kaiser, 1974; Shrestha, 2021), and Bartlett’s test of sphericity, which must be significant to confirm the existence of sufficient inter-item correlations (Bartlett, 1954). The factor loading of an item indicates the strength of its relationship with the underlying factor, and items with loadings of 0.50 and above are generally regarded as contributing meaningfully to the construct (Hair et al., 2019), although some guidelines accept a threshold of 0.40 depending on sample size and study purpose (Howard, 2016).
Methodological reviews stress that the decisions made within EFA, including the extraction method, the rotation method and the factor retention criteria, should be made explicitly and reported transparently because they shape the final structure of the instrument (Fabrigar et al., 1999; Watkins, 2018). Poorly justified defaults, such as retaining factors purely on the eigenvalue-greater-than-one rule, are among the most common errors in applied factor analysis and can produce unstable or uninterpretable solutions (Costello & Osborne, 2005). Items that fail to reach the loading threshold should be removed to preserve construct coherence, a screening practice recognised in the scale development literature (Boateng et al., 2018; Worthington & Whittaker, 2006). Pilot sample size also influences the stability of the factor solution: while classical rules of thumb favour larger samples (Comrey & Lee, 1992), simulation evidence shows that stability depends on communalities and the item-to-factor ratio rather than on absolute size alone (MacCallum et al., 1999), and samples of around one hundred respondents are considered adequate for instruments with a moderate number of items (Kyriazos, 2018). For estimating Cronbach’s alpha itself, methodological guidance for medical and social research similarly supports pilot samples of this magnitude (Bujang et al., 2018).
2.4. Internal Consistency and the Alpha Debate
Although Cronbach’s alpha has dominated reliability reporting since its introduction (Cronbach, 1951), the recent psychometric literature has debated the assumptions underlying it, particularly the tau-equivalence assumption that real data rarely satisfy in full (McNeish, 2018). Several scholars recommend McDonald’s omega as a more realistic reliability estimator because it does not require uniform factor loadings across items (Hayes & Coutts, 2020). Others emphasise that alpha remains informative when its limits are respected: it should be computed per unidimensional construct, interpreted alongside the number of items, and not treated as evidence of unidimensionality (Cortina, 1993; Streiner, 2003; Taber, 2018). The position adopted in this study is pragmatic. Alpha is used because the purpose is pilot-stage reliability screening and comparison with values reported by earlier users of the same instrument, while future studies are encouraged to report omega as a complement.
2.5. Instruments Measuring Adolescent Health Behaviour
A variety of instruments have been developed to measure adolescent eating behaviour, weight perception, self-esteem and physical activity. The Rosenberg Self-Esteem Scale, for instance, is the most widely used measure of global self-worth (Rosenberg, 1965), while numerous eating habit and physical activity instruments have been validated among college and university students (Sogari et al., 2018). Body image and appearance-related constructs have likewise been measured extensively across age groups, confirming that dissatisfaction with body weight is prevalent well beyond adolescence (Quittkat et al., 2019). Nevertheless, most of these instruments were developed for Western populations or mainstream higher education students, and their suitability for Malaysian vocational students cannot be assumed. Differences in culture, language, literacy level and learning environment can affect how items are understood and answered, which necessitates revalidation in the new context (Tsang et al., 2017). The instrument adapted in this study originates in the master’s thesis research of Riggs (2017) and was published by McArthur, Riggs, Uribe and Spaulding (2018), who administered it to 516 undergraduates at a university in North Carolina within a Health Belief Model framework. Its construct coverage matches the focus of research on excess body weight, but it was developed in English for a North American university population and therefore requires both formal cross-cultural adaptation and local psychometric evidence before it can be used with confidence for the Malaysian TVET population.
2.6. Cross-Cultural and Cross-Context Adaptation of Instruments
Adapting an existing instrument to a new context involves more than language translation. A rigorous adaptation process requires the assessment of conceptual equivalence, that is, whether the measured construct carries the same meaning in the target culture; item equivalence, that is, whether each statement is understood as intended; and operational equivalence, that is, whether the format and administration method suit the target population (Souza et al., 2017; Tsang et al., 2017). Established cross-cultural adaptation guidelines further prescribe stages of translation, synthesis, expert review and pretesting before an instrument can be considered equivalent to its original version (Beaton et al., 2000). Failure to address these aspects can produce an instrument that appears to function but actually measures a different construct, thereby contaminating the study’s conclusions. These principles guided the adaptation process reported in this study; items were reviewed for suitability to the Malaysian vocational context and screened on the basis of empirical factor-loading evidence.
2.7. Research Gap
Overall, the literature provides clear methodological standards for instrument validation but reveals a marked shortage of validated instruments for the Malaysian TVET population. Local adolescent obesity and health behaviour studies rely almost entirely on mainstream school samples, and the instruments used rarely pass through a transparently documented validation process for the vocational context. This study addresses that gap by reporting in detail the adaptation, item screening, construct validity testing and reliability assessment of a four-construct instrument for vocational college students, thereby providing a psychometric foundation that future research can cite and build upon.
3. Methodology
3.1. Research Design and Overall Procedure
This study is an instrument validation study conducted as the pilot phase of a larger correlational survey on excess body weight among TVET students. The validation procedure followed the best-practice sequence of scale development (Boateng et al., 2018; DeVellis & Thorpe, 2021; Tsang et al., 2017), namely item adaptation and review, content validation, pilot testing, exploratory factor analysis for construct validity and internal consistency reliability assessment. All analyses were carried out using the Statistical Package for the Social Sciences (SPSS) version 26. Figure 1 presents the conceptual measurement framework of the study, in which the four measured constructs jointly describe the weight-related health behaviour profile of TVET students, supported from below by the psychometric evidence assembled in this study and feeding forward into institutional health monitoring and intervention evaluation.
Figure 1. Conceptual measurement framework of the study.
Figure 2. Five-stage instrument validation workflow.
Figure 2 summarises the five-stage validation workflow. Each stage feeds the next in sequence: the adapted item pool is refined through content and face validation, tested empirically in the pilot study, screened through exploratory factor analysis and finally assessed for internal consistency, producing the validated 46-item instrument used in the main study.
3.2. Original Instrument and Its Provenance
The study instrument was adapted from the weight-related beliefs questionnaire developed by Riggs (2017) and published in peer-reviewed form by McArthur, Riggs, Uribe and Spaulding (2018). The original questionnaire was constructed within the Health Belief Model framework and administered in English as an online survey to 516 undergraduate students at a university in North Carolina, United States, with beliefs rated on five-point scales. It was selected for the present study because its belief domains correspond closely to the weight-related constructs of interest and because the five-point response format suits the reading level of vocational college students. All four attitudinal sections used here are drawn from that instrument.
The complete adapted questionnaire contains five sections. Section A records demographic information including gender, programme of study, year of study, age, household income and parental education background, with occupational classification following the Malaysia Standard Classification of Occupations 2020, together with anthropometric measurements of height and weight for the calculation of body mass index. Weight was measured using a Tanita HD-314 digital scale with a maximum capacity of 150 kilograms and a precision of approximately 100 grams. The scale was calibrated before each measurement session in accordance with the manufacturer’s manual, height was measured with a stadiometer with the participant standing erect, heels together and the head in the Frankfurt plane, and each measurement was repeated two to three times with the mean taken as the final value. Measurements were carried out by the researcher together with a trained assistant.
Section B measures perception of the effects of excess body weight and retains 12 of the 17 original items. Section C measures eating habits, covering lifestyle, attitudes towards eating and food-related behaviour, and retains 12 of the 20 original items. Section D measures weight-related health awareness and retains 13 of the 14 original items. Section E measures perceived benefits of physical activity and retains 9 of the 15 original items. All attitudinal items are rated on a five-point Likert scale, namely 1 = Strongly Disagree, 2 = Disagree, 3 = Not Sure, 4 = Agree and 5 = Strongly Agree. Before screening, the four sections contained 66 attitudinal items in total. The single criterion by which 20 of these were removed is specified in Section 3.7 and its outcome is reported in Section 4.3; no item-removal rationale is given here, so that the criterion is described in one place only.
Two construct labels have been changed relative to the original study, and the reason should be stated plainly. Section D was previously labelled self-esteem, but its items concern awareness of weight-related health risks, sources of health information, advice received from significant others, and motivation to attain and maintain a healthy weight. It does not measure global self-worth and is not equivalent to the Rosenberg Self-Esteem Scale (Rosenberg, 1965); it is therefore reported throughout this paper as weight-related health awareness. Section E was previously labelled physical activity, but its items measure beliefs about the benefits of being active rather than the frequency, duration or intensity of activity performed; it is therefore reported as perceived benefits of physical activity. Naming a scale after a construct it does not measure is a recognised threat to construct validity (Boateng et al., 2018; Worthington & Whittaker, 2006), and the renaming aligns each label with the item content so that users of the instrument are not misled. Readers comparing these results with the original instrument of McArthur et al. (2018) or with the wider study from which this validation is drawn should note the correspondence: weight-related health awareness corresponds to the former self-esteem section, and perceived benefits of physical activity to the former physical activity section.
3.3. Translation and Cross-Cultural Adaptation
The original instrument is in English, whereas the language of instruction and daily use of the target population is Malay. The instrument was therefore translated into Malay through the staged procedure recommended for cross-cultural adaptation of self-report measures (Beaton et al., 2000; Souza et al., 2017; Tsang et al., 2017). A forward translation into Malay was produced by a single translator, a back translation into English was then prepared, and both versions were reviewed by a panel of lecturers experienced in language and in vocational education. The panel assessed semantic equivalence, the accuracy of technical terminology and cultural appropriateness, and any term judged likely to cause confusion was refined through discussion until consensus was reached. The resulting pre-final Malay version was then taken forward to content and face validation and to the pilot study.
Beyond linguistic equivalence, the adaptation addressed conceptual, item and operational equivalence (Souza et al., 2017; Tsang et al., 2017). Items referring to situations without a counterpart in the Malaysian vocational setting were revised, and the mode of administration was changed from the original self-completed online survey to supervised face-to-face completion in order to suit the target population.
3.4. Content and Face Validation
The pre-final Malay version was reviewed for content and face validity before the pilot study. Two subject experts conducted the review: the Head of the Language Unit of Kolej Vokasional Keningau and a lecturer in English holding a doctoral degree. The experts examined every item for accuracy of construct representation, clarity of wording and suitability for the vocational student context, in line with the principle that expert judgement should precede statistical testing (Davis, 1992; Lynn, 1986). Items containing cultural references or situations irrelevant to the local environment were revised. Once both experts had endorsed the instrument, a face validity check was carried out with students to establish whether the questionnaire was understood as intended, and the questionnaire was additionally approved by the research supervisor.
Two aspects of this procedure fall short of current recommended practice and are reported here for transparency rather than omitted. First, the panel comprised two experts, whereas a minimum of three to five is generally advised, and both were language and education specialists rather than specialists in nutrition or health behaviour. Second, expert judgement was recorded qualitatively; relevance was not rated on an ordinal scale and item-level and scale-level content validity indices were therefore not computed (Almanasreh et al., 2019; Polit & Beck, 2006; Yusoff, 2019). The content validation reported in this study should accordingly be read as a qualitative expert review rather than as a quantified content validity assessment, and this is restated among the limitations in Section 5.7.
3.5. Pilot Study
The pilot study was conducted in May 2021 at one public vocational college in Sabah, Malaysia, with 100 student respondents. The respondents represented four programmes of study, namely Culinary Arts, Bakery and Pastry, Cosmetology and Technology, so that the instrument was tested across the diversity of vocational student backgrounds. They were not included in the main study sample of 400 students; the pilot served solely to confirm that the instrument reached an acceptable standard before actual data collection, which is a recognised step for identifying instrument weaknesses in advance (Cohen et al., 2018; Creswell & Creswell, 2018). Questionnaires were distributed and completed face to face, and respondents were briefed on the purpose of the study, asked to answer honestly and assured of confidentiality before completion.
A sample of 100 respondents satisfies the recommended minimum for stable estimation of Cronbach’s alpha (Bujang et al., 2018) and falls within the range that can be adequate for exploratory factor analysis of an instrument with a moderate number of items when communalities are high and factors are well determined (Kyriazos, 2018; MacCallum et al., 1999). Those conditions are demanding, however, and Section 3.7 sets out the respondent-to-item ratios achieved and the consequent limits on the strength of the factor-analytic evidence.
Detailed demographic characteristics of the pilot respondents, the number of students invited relative to the number who completed the questionnaire, and the resulting response rate were not recorded separately from the main study records. This is acknowledged as a reporting limitation in Section 5.7.
3.6. Research Approvals and Ethical Considerations
Before data collection, written approval to conduct the study was obtained from the Educational Planning and Research Division (EPRD) of the Ministry of Education Malaysia, from the Sabah State Education Department, and from the director of the participating vocational college. The questionnaire itself was approved by the research supervisor and endorsed by the two subject experts described in Section 3.4. The researcher met the college director in person to explain the purpose of the study and the administration procedure before distribution began.
Participation was voluntary. Respondents were briefed on the purpose of the study, given sufficient time to complete the questionnaire and assured that their responses would remain confidential, and no personally identifying information was retained in the analysis dataset. Anthropometric measurements were taken individually. The study was carried out under institutional and departmental approval rather than through review by a named research ethics committee, and individual written informed consent and, where applicable, parental consent and participant assent were not separately documented; both points are stated among the limitations in Section 5.7.
3.7. Data Analysis
All analyses were carried out using SPSS version 26. Construct validity was assessed through exploratory factor analysis conducted separately for each of the four constructs rather than on the combined item pool, so that each analysis addressed the internal structure of a single scale. The suitability of each correlation matrix for factoring was examined using the Kaiser-Meyer-Olkin measure of sampling adequacy, with values of 0.70 and above indicating good suitability, and Bartlett’s test of sphericity, which was required to be significant (Bartlett, 1954; Kaiser, 1974; Shrestha, 2021).
A single screening criterion was applied to the 66 attitudinal items. Items with a factor loading below 0.50 on their intended factor were removed as not contributing meaningfully to the construct, and the analysis was re-estimated after removal until an acceptable solution was obtained (Hair et al., 2019). This criterion, and no other, governed the removal of all 20 items reported in Section 4.3; earlier accounts of this instrument that attributed some removals to low reliability values or contextual unsuitability are superseded by the present description, which applies one criterion uniformly across the four constructs. Items removed or revised on grounds of language or cultural fit were dealt with earlier, at the adaptation and expert review stages described in Sections 3.3 and 3.4, and are not counted among the 20.
The adequacy of the pilot sample for factor analysis requires explicit comment. One hundred respondents provide approximately 1.5 respondents per item across the initial 66-item pool and approximately 2.2 per item across the final 46 items, both well below the ratios of 5:1 to 10:1 conventionally recommended (Comrey & Lee, 1992; Hair et al., 2019). Analysing each construct separately improves the position materially, since the largest single analysis involves 20 items rather than 66, but it does not remove the concern. The factor structure reported in this paper is therefore presented as a preliminary solution requiring replication in an independent and larger sample. The sample is adequate for the more modest purpose of estimating Cronbach’s alpha with reasonable precision (Bujang et al., 2018).
The internal consistency reliability of each construct was assessed using Cronbach’s alpha computed on its retained items. Guided by local and international guidelines, an alpha value below 0.60 indicates that the instrument needs to be rebuilt, values from 0.60 to 0.79 indicate acceptable reliability, values from 0.80 to 0.89 indicate a good level, and values of 0.90 and above indicate an excellent level (Ghozali, 2018; Hair et al., 2019; Mohd Konting, 2004). Table 1 summarises the interpretation guideline used in this study. Because alpha assumes tau-equivalence, which real data rarely satisfy, McDonald’s omega is recommended as a complement in future applications of the instrument (Hayes & Coutts, 2020; McNeish, 2018).
Table 1. Interpretation guideline for Cronbach’s alpha coefficient (adapted from Mohd Konting, 2004; Ghozali, 2018).
Alpha Coefficient Range |
Level of Reliability |
0.90 and above |
Excellent |
0.80 to 0.89 |
Good |
0.60 to 0.79 |
Acceptable |
Below 0.60 |
Needs to be rebuilt |
4. Results
4.1. Content and Face Validity
Before the pilot test, the adapted Malay instrument underwent the expert review described in Section 3.4. Both reviewers endorsed the instrument for use after items with unclear wording or with cultural references irrelevant to the Malaysian vocational environment had been revised, and the subsequent face validity check with students confirmed that the questionnaire was understood as intended. Because relevance was recorded qualitatively rather than rated on an ordinal scale, item-level and scale-level content validity indices are not available for this instrument. The outcome of the review formed the item set that was subsequently tested empirically in the pilot study.
4.2. Suitability of the Data for Factor Analysis
Exploratory factor analysis was conducted separately for each construct on the pilot data of 100 respondents. Sampling adequacy was acceptable for all four constructs, with Kaiser-Meyer-Olkin values ranging from 0.70 for perceived benefits of physical activity to 0.78 for weight-related health awareness, and Bartlett’s test of sphericity was significant for every construct (p < 0.001), confirming sufficient inter-item correlation for factoring (Bartlett, 1954; Kaiser, 1974; Shrestha, 2021). All retained items loaded at 0.50 or above on their intended factor, with loadings ranging from 0.50 to 0.81 across the four constructs. Table 2 reports these results by construct.
Table 2. Sampling adequacy, sphericity and factor loading range by construct in the pilot study (n = 100).
Construct |
Items retained |
KMO |
Bartlett’s test of sphericity |
Factor loading range |
Perception of excess
body weight |
12 |
0.75 |
p < 0.001 |
0.55 - 0.79 |
Eating habits |
12 |
0.72 |
p < 0.001 |
0.50 - 0.77 |
Weight-related health awareness |
13 |
0.78 |
p < 0.001 |
0.52 - 0.81 |
Perceived benefits of physical activity |
9 |
0.70 |
p < 0.001 |
0.53 - 0.76 |
4.3. Construct Validity
Applying the 0.50 loading criterion removed 20 of the 66 attitudinal items, namely five items from the perception of excess body weight construct, eight items from the eating habits construct, one item from the weight-related health awareness construct and six items from the perceived benefits of physical activity construct, leaving a 46-item instrument. The resulting structure is presented in Table 3, and the effect of the screening on the item count of each construct is visualised in Figure 3. These results provide preliminary support for the construct validity of each dimension studied in this sample. They should be read as a preliminary structure: with approximately two respondents per item, the solution is susceptible to sampling variability and requires confirmation in an independent sample before the factor structure can be regarded as established.
Figure 3. Original versus retained items for each construct after factor-loading screening.
Table 3. Structure of the instrument after the adaptation and item screening process.
Section |
Construct |
Original Items |
Retained Items |
Scale |
B |
Perception of excess body weight |
17 |
12 |
5-point Likert |
C |
Eating habits |
20 |
12 |
5-point Likert |
D |
Weight-related health awareness |
14 |
13 |
5-point Likert |
E |
Perceived benefits of physical activity |
15 |
9 |
5-point Likert |
4.4. Internal Consistency Reliability
Table 4 reports the Cronbach’s alpha coefficient for each construct based on the pilot data of one hundred respondents, and Figure 4 displays the coefficients against the conventional 0.70 acceptability threshold. The perception of excess body weight construct recorded an alpha of 0.79, eating habits 0.80, weight-related health awareness 0.83 and perceived benefits of physical activity 0.81. The value for physical activity is close to the value of 0.85 reported by McArthur et al. (2018) for the corresponding original subscale, although the two coefficients rest on different item sets, a different language of administration and a different student population, so their similarity is not in itself evidence of measurement equivalence. All four values fall within the acceptable-to-good range according to the guidelines adopted (Ghozali, 2018; Mohd Konting, 2004) and all exceed the 0.70 threshold customarily accepted in social science research (Hair et al., 2019; Nunnally & Bernstein, 1994; Taber, 2018). These results indicate that every construct possesses sufficient internal consistency to be used in the actual study.
![]()
Figure 4. Cronbach’s alpha coefficients of the four constructs against the 0.70 acceptability threshold.
Table 4. Cronbach’s alpha reliability coefficients for each construct (n = 100).
Construct |
No. of Items |
Cronbach’s Alpha (Pilot) |
Level |
Perception of excess body weight |
12 |
0.79 |
Acceptable |
Eating habits |
12 |
0.80 |
Good |
Weight-related health awareness |
13 |
0.83 |
Good |
Perceived benefits of physical activity |
9 |
0.81 |
Good |
4.5. Sample Items for Each Construct
To give an impression of the content of the instrument, Table 5 presents a sample item for each construct. The perception of excess body weight items address respondents’ views on the consequences of body weight, the eating habits items cover lifestyle and attitudes towards eating, the weight-related health awareness items focus on awareness of weight-related health risks and advice received from others, and the perceived benefits of physical activity items address the benefits students associate with being active. All items are answered on a five-point Likert scale.
Table 5. Sample items for each construct of the instrument.
Construct |
Sample Item |
Perception of excess body weight |
It will adversely affect my health in the future. |
Eating habits |
I consume sugary drinks, foods or snacks every day or on most days. |
Weight-related health awareness |
A doctor, nurse or nutritionist has advised me about my health. |
Perceived benefits of physical activity |
Physical activity reduces my depression,
stress and anxiety. |
Taken together, the pilot findings provide consistent evidence that the adapted instrument satisfies psychometric standards for construct validity and internal consistency reliability. The instrument was subsequently used in the main study of four hundred vocational college students, and its psychometric performance in the pilot provides confidence in the quality of the data collected. The following discussion interprets these findings and examines their implications for research and practice.
5. Discussion
This study reported the validity and reliability evidence for a four-construct instrument for the Malaysian TVET population. The findings show that the instrument satisfies the psychometric standards set, with good factorability, satisfactory factor loadings for all retained items and reliability coefficients between 0.79 and 0.83 across constructs. The following discussion interprets these results in the context of the psychometric literature, evaluates the strength of the adaptation process, examines the implications for research and institutional practice, and acknowledges the limitations that future studies should address.
5.1. Interpreting the Construct Validity Evidence
The construct validity evidence obtained meets the conventional thresholds, though its strength is bounded by the size of the pilot sample. The Kaiser-Meyer-Olkin values of 0.70 to 0.78 indicate acceptable sampling adequacy, and the significant Bartlett’s test confirms that the correlation matrix is suitable for factoring (Kaiser, 1974; Bartlett, 1954; Shrestha, 2021). More importantly, retaining only items with factor loadings of 0.50 and above ensures that each construct is represented by items that contribute substantively, in line with the thresholds recommended in the multivariate analysis literature (Hair et al., 2019; Tabachnick & Fidell, 2019). The screening process that removed a total of twenty items across four constructs may appear aggressive, but it reflects the principle that construct coherence takes precedence over the number of items; a shorter but homogeneous instrument produces more precise measurement and reduces respondent burden (Boateng et al., 2018; Costello & Osborne, 2005). For a vocational student population whose diversity of literacy levels must be taken into account, a concise and clear instrument also reduces the risk of item misunderstanding, thereby contributing indirectly to data quality.
5.2. Interpreting the Reliability Evidence
Cronbach’s alpha coefficients between 0.79 and 0.83 place all four constructs within the acceptable-to-good range (Ghozali, 2018; Hair et al., 2019; Mohd Konting, 2004). Two points deserve attention in interpreting these values. First, the alpha value for physical activity (0.81) is close to the 0.85 reported by McArthur et al. (2018) for the corresponding original subscale. This is reassuring, but a claim of equivalence across populations would require a formal measurement invariance analysis. Second, the fact that the alpha values are not excessively high, that is, they do not exceed 0.90, should actually be read positively; extremely high alpha values can signal item redundancy, meaning items that are so similar that they add no new information (Streiner, 2003; Taber, 2018). The obtained range of 0.79 to 0.83 suggests that the items within each construct are consistent without being repetitive, a desirable psychometric balance.
Although Cronbach’s alpha remains the most widely reported reliability index, it should be acknowledged that recent psychometric literature has debated its underlying assumptions, particularly the tau-equivalence assumption that real data rarely satisfy in full (McNeish, 2018). Some scholars recommend McDonald’s omega as a more realistic reliability estimator because it does not require uniform factor loadings across items (Hayes & Coutts, 2020). In the context of this study, the use of alpha is justified given that the purpose was pilot-stage reliability screening and comparison with values reported by earlier researchers who also used alpha. Nevertheless, future studies using this instrument are encouraged to report the omega coefficient as a complement to alpha, in line with current psychometric reporting practice. Acknowledging this debate shows that the reliability interpretation in this study was made with full awareness of its methodological limits.
5.3. Comparison with Other Validation Studies
The reliability values obtained are comparable to those of other health behaviour instrument validation studies. The recorded alpha values of 0.79 to 0.83 fall within the range commonly reported for eating habit and health perception instruments among higher education students (Sogari et al., 2018) and match the standards achieved by local educational instrument validation studies (Lee Abdullah & Wei, 2017). This comparability matters because it demonstrates that the vocational population, although different in background from the university populations usually studied, is capable of giving consistent responses to a well-adapted instrument. The finding counters any assumption that survey instruments are less suitable for vocational students and instead affirms that, with a careful adaptation process, the TVET population can be studied to the same psychometric standard as other populations. This opens the way for the broader inclusion of vocational populations in national adolescent health research, thereby addressing the representation imbalance that has long existed in the literature.
5.4. Contribution to Malaysian TVET Research
The main contribution of this study is the provision of a validated instrument specific to a population that has so far been marginalised in Malaysian adolescent health research. The absence of validated instruments for vocational students is not merely a technical gap; it limits the ability of researchers to generate trustworthy local evidence on the health behaviour of this group, and in turn limits the ability of policymakers to develop evidence-based interventions. By documenting the adaptation process and psychometric evidence transparently, this study enables other researchers to use the same instrument, compare findings across studies and build a cumulative evidence base. Such transparent reporting is an open science practice increasingly demanded in psychometric research (Boateng et al., 2018; Watkins, 2018). The instrument also opens up opportunities for comparative studies between vocational and mainstream populations, a comparison that has so far been difficult because of instrument differences.
5.5. Implications for Institutional Practice
At the level of practice, this validated instrument offers a practical monitoring tool for vocational institutions. Colleges can use the instrument for periodic screening of students’ eating behaviour, weight-related health awareness and beliefs about physical activity, for detecting at-risk groups and for evaluating the effectiveness of institutional health programmes over time. Because the instrument is concise and self-administered, it is suitable for use in institutional routines without demanding substantial resources. Such implementation naturally requires the support of institutional leadership and the readiness of teaching staff.
The successful use of the instrument as an institutional monitoring tool depends on committed instructional leadership (Osman & Kamrozzaman, 2024). The integration of the instrument into digital learning platforms can also be leveraged, in line with the development of blended learning in Malaysian higher education (Rais, Kamrozzaman, & Jie, 2025), while self-determined learning approaches can encourage students to monitor their own health using feedback from this instrument (Kamrozzaman & Jie, 2025).
5.6. Guidance for Researchers Using the Instrument
To facilitate reuse, several administration and scoring guidelines should be stated. The instrument is suitable for face-to-face or online administration and takes approximately fifteen to twenty minutes to complete. The score for each construct is obtained by summing or averaging the item responses on the five-point Likert scale, with higher scores indicating a higher level of the construct. The anthropometric section requires a calibrated digital scale and stadiometer, and measurement should be carried out by two people to ensure accurate recording. Researchers using this instrument in different populations or regions are advised to conduct a reliability check on their own sample and report the alpha values obtained, given that reliability is a property of scores within a particular sample rather than a fixed property of the instrument (Streiner, 2003; Taber, 2018). Such reporting practice will allow the psychometric evidence base for this instrument to accumulate systematically across studies.
5.7. Limitations and Recommendations for Future Research
Several limitations should be acknowledged. First, the pilot study was conducted at one vocational college in Sabah, and although the respondents represented four programmes of study, generalisation to other zones and states requires further confirmation. Fifth, construct validity was assessed through exploratory factor analysis only; future studies should conduct confirmatory factor analysis (CFA) on a separate sample to confirm the factor structure obtained, in line with the exploratory-to-confirmatory sequence recommended in the literature (Boateng et al., 2018; Hair et al., 2019; Kline, 2023). Sixth, reliability was assessed through internal consistency only; an assessment of test-retest stability would complete the reliability profile of the instrument. Seventh, the four constructs are attitudinal and perceptual throughout; none of them captures observed behaviour, so studies needing behavioural outcomes should pair this instrument with direct measures. Eighth, measurement invariance across gender and programme of study has not been tested; measurement invariance analysis would ensure valid group comparisons in future studies.
The most important limitation concerns sample size in relation to the number of items. One hundred respondents provide approximately 1.5 respondents per item across the initial 66-item pool and approximately 2.2 per item across the final 46 items, both well below the 5:1 to 10:1 ratios conventionally recommended for exploratory factor analysis (Comrey & Lee, 1992; Hair et al., 2019). Analysing each construct separately improves the ratio but does not eliminate the risk that the solution reported here is unstable and would not replicate exactly in a new sample. The factor structure should accordingly be treated as a working hypothesis rather than a settled property of the instrument.
A second limitation concerns the completeness of the psychometric reporting. The factor extraction and rotation methods, the factor retention rule, and the eigenvalues, communalities and percentages of variance explained were not recorded at the time of analysis, and only the sampling adequacy statistics, loading ranges and alpha coefficients reported in Table 2 and Table 4 are available. Content validity was established through qualitative expert review by two language and education specialists rather than through quantified content validity indices rated by a larger and disciplinarily broader panel. Response rate and detailed demographic characteristics of the pilot respondents were likewise not recorded separately. Researchers replicating this validation should report these quantities in full, and the present factor structure should not be regarded as independently reproducible until they do.
A third limitation concerns approvals and consent. The study was conducted under written approval from the Educational Planning and Research Division, the Sabah State Education Department and the participating college, but not under review by a named research ethics committee, and individual written informed consent and, where applicable, parental consent and participant assent were not separately documented. Future applications of this instrument should secure formal ethics committee approval and document consent and assent procedures explicitly.
A fourth point concerns construct coverage rather than labelling. The constructs have been renamed to match their item content, as explained in Section 3.2, but this means the instrument does not measure global self-worth or physical activity behaviour at all. Researchers requiring those constructs should administer purpose-built measures alongside this instrument, such as the Rosenberg Self-Esteem Scale (Rosenberg, 1965) for global self-worth or a validated activity recall or accelerometry protocol for activity behaviour.
6. Conclusion
This study reported the adaptation process and the validity and reliability evidence for an instrument measuring perception of excess body weight, eating habits, weight-related health awareness and perceived benefits of physical activity among Malaysian TVET students. Through a pilot study of one hundred vocational college students in Sabah, the instrument demonstrated acceptable factorability, with Kaiser-Meyer-Olkin values of 0.70 to 0.78, factor loadings from 0.50 to 0.81 for all retained items, and Cronbach’s alpha coefficients between 0.79 and 0.83 across the four constructs. These results indicate that the instrument reaches a psychometric standard suitable for use in research on the vocational population, with the qualification that the factor structure is preliminary and awaits confirmation in an independent sample.
The contribution of this study is twofold. Methodologically, it provides transparent documentation of an adaptation and validation process that other researchers can use as a reference. Practically, it offers a ready-to-use tool for vocational institutions to monitor student health behaviour and evaluate interventions. Future studies should extend this psychometric evidence through confirmatory factor analysis, test-retest stability testing and measurement invariance analysis, as well as by extending the validation sample to other zones and states. With a solid psychometric foundation, health behaviour research on Malaysian TVET students can grow on a foundation of trustworthy measurement.
Acknowledgements
The authors would like to thank UNITAR International University for supporting this research.
Author Contributions
The first author developed the study concept, carried out the instrument adaptation, conducted the pilot study, analysed the data and prepared the draft manuscript. The second author supervised the research, contributed to the interpretation of the psychometric findings and critically reviewed the manuscript. Both authors have read and approved the final version of this paper.