The Validity and Reliability of Social and Psychological Skills (SP-Skills) Module ()
1. Introduction
Human capital, especially the highly educated, has been recognized as an important component in contemporary economic growth. This is due to the rapid advancement of technology and increased globalization competition, which has led to major changes to the labor market and increased demand for higher education. However, being in university life demands various aspects of a student’s life, such as emotional, psychological, intellectual, social and moral compared to other levels of life. Apart from that, in this stage of life, most students also experience new pressures and challenges that contribute to psychological difficulties such as low psychological well-being (Shahira et al., 2018; Auerbach et al., 2016) . In addition, there is also a large proportion of university students in Malaysia who cannot adapt to new environments and social relationships, different cultures, as well as have a low level of social skills such as not being assertive (Sieng & Yussof, 2017; Siang-Ang, 2016) , having communicative competence and moderate skills in terms of future orientation, critical thinking, identity formation, and prosocial tendencies (Khairani, Ahmad, & Sutatminingsih, 2021; Yoke & Ngang, 2017) . All these problems if not dealt with properly can cause students to be exposed to various mental health problems.
Therefore, the most important goal for all universities is not only to provide knowledge to students, but also to ensure psychological well-being, and encourage students to develop assertiveness and social skills for their current academic and professional goals. In addition, some research results show that assertiveness, social skills and psychological well-being can be formed and improved in a person while at school, college or university level through intervention or training. These three aspects can be improved and developed through various programs such as psychoeducation, counseling, workshops or training sessions given (Cabrera, Daya, & Echague, 2020; Howard & Gutworth, 2020; Ishak, 2016) . Nevertheless, some research that has been carried out shows that psychoeducational approaches are often used in institutions such as colleges and universities to help counselors or even facilitators to deliver information in a timely manner that is short, especially with the increasing number of students (Hood, Jelbert, & Santos, 2021; Robson & Simone, 2019) . Therefore, this whole problem requires a psychoeducational intervention that can play a dominant role in promoting psychological well-being, developing social skills and assertiveness and at the same time providing satisfaction and success at the personal, academic and professional levels of students. Thus, the SP-Skills module is developed based on this educational approach in order to meet the current needs.
2. Research Background
A good design will help to achieve the learning outcomes. In this research, SP-Skills module was developed by referring to the Sidek Module Development Model (MPMS) (Noah & Ahmad, 2005) . The justification for choosing this model is because this model is the only model that is considered as a comprehensive and integration model. The model is complete and has systematic and specific steps to develop module. In fact, this model also emphasizes on pilot study to identify weaknesses and advantages of the module. The information obtained during pilot study is useful to improve the module. In addition, several researchers have used the Sidek Module Development Model and they have proven in their studies that this model has high validity and reliability (Yusoff, Mahfar, & Saud, 2019; Zainuddin, 2018; Mahmud, 2017) .
Besides that, the Sidek Module Development Model is chosen due to its structured stages, which helps the researchers to plan, design and organize the systematic steps and this model is considered complete as it can explain the development of the draft module to test the effectiveness of the module. This model comprises of two stages, each with different purposes. The first stage produced a draft module. There are nine steps. It starts off with developing an aim of the module and then identifying theories, rational, philosophy, concept target and time allocation. After that, it will be followed by needing an assessment, setting the objectives, selecting the content, strategy, logistic and media. Lastly, it ends with integrating the draft module. Afterward, at the second stage, the module itself will be tested and evaluated. This draft module will undergo a pilot test to assess its validity, reliability and norms. If it is proven that this module is high in validity and reliability, then it can be considered as a complete module and can be proceed onto the last step; effective evaluation. After following these two stages, the module can be employed to target population.
Three theories are integrated in the development of the SP-Skills module, namely the Rathus Assertiveness Schedule (RAS) (Rathus, 1973) , the Del Prette Social Skills Inventory (SSI) (Del Prette & Del Prette, 2013) , and the Psychological Well-Being Theory (Ryff, 1989) . Rathus Assertiveness Schedule (RAS) has five sub-constructs which are the ability to respect, the ability to express feeling, probability of making mistakes, the ability to say “no”, and interacting and communicating. Next, there are five social skills factors in the Social Skills Inventory (SSI) built by Del Prette and Del Prette in 2001 namely coping and self-assertion with risk, self-assertion in the expression of positive effect, conversation and social confidence, self-exposure to unknown people and new situations and self-control of aggressiveness. While there are six dimensions of psychological well-being in Ryff’s Model of Psychological Well-being, namely autonomy, environmental mastery, personal growth, positive relationship with others, purpose in life and self-acceptance. The integration of theory can make the module more effective and guarantee the effectiveness of this SP-Skills module because each selected theory will complement each other.
This module also applies a psychoeducational approach that focuses on individual psychological aspects and educates participants about educational techniques or strategies. It also plays a role similar to counseling and psychotherapy in shaping individuals in a more positive direction. This method of psychoeducational intervention is the best strategy for channeling useful information and being able to explore oneself, i.e. strengths and weaknesses, and build various skills. In the context of this study, psychoeducation acts as an alternative teaching intervention to help students in the development of social and psychological skills. The detailed theoretical framework of SP-Skills module is illustrated in Figure 1.
![]()
Figure 1. SP-Skills module theoretical framework.
3. Methodology
This study is a survey study conducted to test the validity and reliability of the draft module that was built. The main focus is to obtain validity values through written responses by a group of expert panelists and reliability values through written responses (module reliability instrument) from 45 respondents who were the first-year students of a public university in Malaysia, after they have completed each activity in the module.
For content validity purpose, a total of eight experts have been appointed as panelists for the content evaluation of the SP-Skills module. The selection criteria for expert panelists are randomly based on; 1) expertise and knowledge in the field of psychology and counselling, 2) over five years of experience working in the field of psychology and counselling, 3) expertise and experience in the psychology and counselling modules development, and 4) academicians or registered practitioners in the field of psychology and counselling. Therefore, for this study, the appointment made based on the background of the expert panels is coincidental and appropriate. Table 1 shows the background of each expert involved. All of them are registered counselors from the Malaysia Board of Counselors and have more than 10 years of experience as counselors. Apart from that, they also conduct talks, workshops, and courses related to counselling.
The validity of a module refers to the accuracy of the concept and content of a module. This means that a module with high validity will produce achievements according to the objectives that the researcher wants to measure. Based on Russell’s (1974) model in order to obtain validity items, the module must meet five
![]()
Table 1. Expert panelists’ profile.
points, namely: 1) meet the target population; 2) the teaching situation or module implementation method is appropriate; 3) the time allocated to carry out or the method of carrying out the module is sufficient; 4) the module successfully causes an increase in student achievement in the targeted aspect; and 5) the module succeeds in changing students’ attitudes towards excellence. To determine the validity of a measurement tool or module, the method that can be used is to ask for feedback and views from experts (Konting, 2000) . Evaluation and criticism from an expert panel was used by Russell (1974) and also local researchers to obtain the validity of the tested module (Ahmad, 2013; Mohamed Arip, 2010; Bistamam, 2006) .
The content validity of the SP-Skills is examined in this study through feedback and evaluation by expert panels to assure the quality of the module and effective execution of the module (Noah & Ahmad, 2005; Siong & Syed Ali, 2023) . The expert panel must assess and verify the module’s overall content based on Russell’s (1974) view, which was modified by Noah and Ahmad (2005) contained five statements namely: 1) meet the target population; 2) make sure the situation of teaching or method of implementation of the module is appropriate; 3) the time allocated to run the module or the module implementation method is sufficient; 4) the module has successfully led to an increase in student achievement in the targeted area; and 5) module managed to modify the attitude of student. The expert panels were also requested to provide suggestions for improvement, feedback and response in the space provided. Suggestions for improvement, reviews and feedback from the expert panels have been applied to SP-Skills draft improvements. For the purpose of data collection, this study uses a variety of approaches, namely face-to-face (direct approach), and online (email) according to the comfort and needs of experts (Brinkman, 2009) .
At the initial stage, expert panelists were contacted via email to obtain consent in addition to explaining the purpose and procedure of the study. This study contacted a total of fifteen experts to be a panel, but only eight experts agreed with each out of them six professional experts and two field experts. However, the number of these experts is more than sufficient for this study to be done. For the purpose of evaluating the validity of the module content, the researcher has prepared a complete copy of the module that contains the introduction, the general objective of the module, the theoretical basis, and the overall content of the module prepared to be examined and evaluated by the panel of experts. Experts are also required to give comments, suggestions, criticism and feedback on the space provided. The scale for this evaluation is a ten-point likert scale, which is 1 (strongly disagree) to 10 (strongly agree). Meanwhile, the calculation method for content validity is based on the formula that was suggested by Noah and Ahmad (2005) who is based from Russell’s (1974) formula. The total score that given by each texpert (x) will be divided by the total maximum score (y) and then multiplied by 100. A module has high content validity when it obtains 70 percent and is achieve a high level of achievement (Noah & Ahmad, 2005; Tuckman, 1988) and vice versa if the percentage is less than 70 percent, this module does not have good content validity. The formula is shown in Figure 2.
![]()
Figure 2. Formula to analyze the content validity of SP-Skills.
Reliability is linked to consistency. This means that a test is said to have high reliability if the same score is obtained from the same individual at different times. According to Mohamed Arip (2010) , module reliability refers to the consistency and stability of a module in treating what should be treated as found in the objective of a module. Testing the reliability of a module can be seen through the extent to which students can follow the content of a module (Russell, 1974) . According to Noah and Ahmad (2005) , the method of determining the reliability of a module is similar to the method of determining the reliability of a measurement instruments. This is because both are instruments developed specifically for a specific purpose.
Therefore, the reliability of a module can be tested by employing a questionnaire based on the objectives of a module or the implementation steps in the module as recommended by Noah and Ahmad (2005) . A study by Mohamed Arip (2010) in the designing of a self-concept improvement module has constructed reliability items based on the steps of implementing the module and obtained a reliability coefficient value of 0.84. While the study by Nawawi et al. (2016) in the development counseling intervention Self-Excellence Module based on six values PANS Transformation also constructed reliability items based on steps and obtained a reliability coefficient value of 0.95. Therefore, in this study, the researcher chose to develop a reliability item questionnaire based on the objectives of each activity found in the module. This is because there is a time limitation and this pilot study was tested on a sample that has very strict time constraints.
To determine the module’s reliability (Cronbach’s alpha value), there are two methods that can be used, namely the construction of questionnaire items based on module objectives or implementation steps within the module (Noah & Ahmad, 2005) . In this study, the researchers have used the method of constructing the module reliability questionnaire based on module objectives to obtain the reliability coefficient for the module. Thus, based on the objectives of the learning units in the module that have been assessed and approved by the experts in the module validation process, the statements in the reliability questionnaire for this study were constructed. The questionnaire, in which comprises of 93 statements based on five sub-topics in the module’s learning units, is used to assess the reliability of this module. The 5-point Likert scale was used to answer the questionnaire. The questionnaire had been validated by three experts. Based on the reviews by the experts, researcher had modified and removed a few terms used in the questionnaire items. After correcting the questionnaire, the researcher conducted a pilot test on 45 undergraduate students.
A pilot study is a mini version of a research or a trial run to conduct a full-scale study and may be conducted specifically to pre-test a research instrument (Dikko, 2016) . In this pilot test study, the researcher’s intention is to evaluate the reliability of SP-Skills modules as module capabilities in helping students achieve various objectives that have been set. According to Cooper and Schindler (2006) , the sample size in the pilot study does not need to be determined base on statistics because it is only to test the validity of the test instrument. Hertzog (2008) made several different recommendations for sample size depending on the purpose of the pilot study in her recent and comprehensive article. For a feasibility study, her recommendations were, “samples as small as 10 - 15 per group sometimes being sufficient” (p. 190). For instrument development, her recommendation was 25 to 40. Hertzog (2008) recommended 20 to 25 for intervention efficacy pilots, given reasonable effect sizes, but 30 to 40 per group for pilot studies comparing groups. Whereas, the Central Limit Theorem (CLT) states that the distribution of sample means approximates a normal distribution as the sample size gets larger, regardless of the population’s distribution. Sample sizes equal to or greater than 30 are often considered sufficient for the CLT to hold. In order to meet the central limit theorem, the sample size for the pilot study is 45 randomly selected first year bachelor’s degree student from the targeted population. Initially, these samples have homogeneity characteristics with the research respondents.
The participants are chosen at random using the following inclusion criteria: 1) first year Bachelor’s Degree student; and 2) attend public university. For the selection process, all first-semester bachelor’s degree students at University Malaysia Pahang were invited to participate in the study. The selection of the respondents is suitable according to the needs of the pilot study because it has characteristics that are similar to the actual study subject and the number of participants is suitable to assist the researcher to obtain the overall reliability value of the module and the sun units in the SP-Skills required by using Cronbach’s alpha method (Ali, Amat, & Mohd Kari, 2019; Zahir, Saper, & Bistamam, 2019) . Written informed consent was obtained from all those who voluntarily agreed to participate in the study before enrollment. Students who did not met the criteria, not provide consent or were absent when the data was being collected were not included in this study. After that, the researcher can identify the weaknesses of the module itself in terms of the suitability of activities, time usage to finish the module and its activities, suitability the activities of module with the target sample, clarity of the objectives, steps of activities and instructions and others.
In terms of reliability value, at least a minimum reliability value of 0.60 should be achieved to indicate that the module has a good, acceptable and consistent level of reliability (Ali, Amat, & Mohd Kari, 2019; Pallant, 2013; Sekaran & Bougie, 2016) . If less than 0.60, the module is in poor consistency and needs to be repaired. The value of the Cronbach’s alpha less than 0.6 is considered weak and the closer the Cronbach’s alpha to 1.0 is better, and when the value of Cronbach’s alpha is more than 0.7, this is considered good and acceptable (Fraenkel, Wallen, & Hyun, 2012) . Conversely, Bond and Fox (2015) and Sekaran and Bougie (2016) stated that a Cronbach’s alpha more than 0.8 is considered good and acceptable. This high reliability indicates that the respondents have mastered the objectives and are able to successfully follow the steps of each activity in this module. The high reliability values of the SP-Skills Cronbach’s alpha reliability indicate that the content of this module is reliable, excellent, and can be used for experimental studies. Improvements have been made based on expert panel feedback. The smoothness of the target group following each activity indicates that the modules produced are good and have an impact on improving assertiveness, social skills as well as psychological wellbeing.
4. Results and Discussion
The findings in this study are divided into two which involve the validity of the module content and the reliability of the module. The content validity of the module was obtained through written responses by eight experts. While the reliability of the module was obtained through written responses from 45 participants who followed the pilot study.
4.1. Validity of SP-Skills Module
The evaluation of the validity of the module content carried out by experts will be discussed based on the content level of the entire content of the SP-Skills module and based on the sub-module that is the session and activity of the SP-Skills module.
Content Validity of the SP-Skills Module
The content validity of the SP-Skills module was obtained as a result of expert feedback on the modified instrument. The feedback includes the target population, implementation method, and appropriateness of content, timing, and the module’s ability to improve assertiveness, social skills, and psychological well-being of university students. To determine the content validity coefficient of the SP-Skills module, the descriptive method of calculating the content validity of the module proposed by Tuckman (1988) was used.
Table 2 shows the validity assessment given by the expert panel for each statement related to the content of the module. The results of the analysis show that the percentage of 92.5% and 91.3% of the module content meets the target population and can be implemented perfectly. While the percentage value of 87.5% for the content of the SP-Skills module corresponds to the allotted time. While the maximum percentage of 93.8% for the content of the SP-Skills module can increase assertiveness, social skills and psychological well-being of university students.
Based on the results of the analysis shown in Table 3, the content validity coefficient value of the module is 0.92. The level of module content validity has been set at 0.70 as a condition for determining the level of module content validity. This condition coincides with the recommendations presented by previous researchers. Therefore, this SP-Skills module has high validity and can be used.
![]()
Table 2. Expert evaluation of SP-Skills module.
![]()
Table 3. Overall content validity of SP-Skills module.
Validity of the SP-Skills Sub-Module
Table 4 shows the validity coefficient of the SP-Skills sub-module. The content validity coefficient for all SP-Skills sub-modules is above 0.85. The minimum content validity coefficient of 0.88 is for the social skills sub-module, whereas the maximum content validity coefficient of 0.93 is for the conclusion sub-module. This means that the level of content validity of the SP-Skills sub-module is good and reliable.
The results of the eight experts’ evaluation found that the value of the content validity of the SP-Skills module was 0.92 which is above the minimum level of 0.70. This condition this shows that the value of the content validity of the SP-Skills module is high coincides with the recommendations presented by Tuckman (1988) , as well as previous researchers (Mohamed Arip, 2010; Bistamam, 2006) . The value shows a good validity of the content. Apart from that, comments and feedback on module content are only focused on the sentence structure and the duration of the activity. No major faults such as irrelevant content or inappropriate activity are identified. This shows the module is on track. In addition, the agreement of the experts who suggested that the next action should be carried out, which is a pilot study, is also a clear indicator that this module is suitable and relevant to be used in the study to help students in the development of assertiveness, social skills and psychological well-being.
The content validity instrument given to experts contains an opinion column that allows experts to give written comments, suggestions, criticisms and feedback on the entire content of the SP-Skills module. The comments and improvement suggestions are also strengthening ideas in strengthening the SP-Skills module. However, overall based on the responses from the experts, it shows that this SP-Skills module can be implemented, meet the objectives, target group, and time that has been set.
![]()
Table 4. Content validity of SP-Skills sub-module.
![]()
Table 5. Reliability of sub-modules and overall SP-Skills modules.
4.2. Reliability of SP-Skills Module
In order to determine the reliability value of the module, the written responses by 45 undergraduate students in the reliability instrument were analyzed for Cronbach’s alpha value. The results of reliability testing are shown in Table 5.
Based on the results of the analysis in Table 5, the overall reliability value of the SP-Skills module is 0.958. While for the sub-module, the highest Cronbach’s alpha reliability value is 0.938 which is the social skills sub-module and the lowest Cronbach’s alpha value which is 0.901 is the introduction module unit.
The process of getting the reliability value done is in line with what has been done by previous researchers on their module (Jusoh et al., 2011; Mohamed Arip, 2010) . Based on the results of the analysis, the overall reliability value of the SP-Skills module is 0.958. While for the sub-module, the highest Cronbach’s alpha coefficient reliability value is 0.938 which is the social skills sub-module and the lowest Cronbach’s alpha value is 0.901 which is the introduction unit. According to Konting (2000) , if the reliability value obtained is high, at least 0.60 means that the module has a good level of consistency. On the other hand, a reliability value that does not reach the value of 0.60 means that the module is at a poor level of consistency and needs to be improved. This means that the overall reliability of the SP-Skills module is good, acceptable, reliable and can be used in interventions to improve assertiveness, social skills and psychological well-being of university students because it exceeds the level of 0.60.
5. Conclusion
In a nutshell, this study has developed a SP-Skills module based on the Sidek Module Development Model with a high value of validity. Based on the findings from the present study, the overall value of reliability is also high. This module is expected to have a positive impact on university students as it has the capability to improve students’ assertiveness, social skills and psychological well-being. Since this SP-Skills module has high validity and reliability, the researchers have a few recommendations for further research in this area. It is suggested that in the future, other researchers can refer to the Sidek Module Development (Noah & Ahmad, 2005) in order to develop new modules. Meanwhile, to assess the content validity of module, the experts’ evaluation can be used and the calculation method of its validity can be referred to Russell’s (1974) formula. Lastly, the quantitative and qualitative approaches can also be employed in order to confirm the effectiveness of the module.