Paper Menu >>
Journal Menu >>
![]() Journal of Software Engineering and Applications, 2013, 6, 571-581 Published Online November 2013 (http://www.scirp.org/journal/jsea) http://dx.doi.org/10.4236/jsea.2013.611069 Open Access JSEA 571 Standardization of Quality Evaluation of Educational Software and Electronic Learning Tools—Analysis of Opinions of Selected Experts* Štefan Karolčík, Elena Čipková, Milan Veselský, Helena Hrubišková Department of Didactics in Science, Psychology and Pedagogy, Faculty of Natural Sciences, Comenius University, Bratislava, Slovak Republic. Email: karolcik@fns.uniba.sk, cipkova@fns.uniba.sk, veselsky@fns.uniba.sk, hrubiskova@fns.uniba.sk Received September 27th, 2013; revised October 24th, 2013; accepted October 31st, 2013 Copyright © 2013 Štefan Karolčík et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. ABSTRACT Our research was focused on the identification of features, which was essential for educational digital products and the determination of their quality. The introductory analytical part of our research is focused on the analysis of existing sources of information related to the problems of research, production, appropriate use and evaluation of educational software environments. Consequently, we have divided the existing software products into three basic groups according to our main distinguishing feature. Second part of our paper is focused on various aspects, which are to be considered when assessing the quality of software solutions. The final part contains the presentation of results of our findings re- lated to the most important features expected and required from digital learning tools by professional experts and spe- cialists in given field. Keywords: Educational Technologies; Learning Tools; Evaluation Tools 1. Introduction Reliable and in-depth quality evaluation of electronic learning tools presupposes the existence of specific evaluation tools, which are able to evaluate the specific content and functionality of software applications with educational ambitions. Only the clearly defined, impar- tial, professionally correct and understandable criteria can distinguish a high quality educational product from other products—less appropriate or even socially dan- gerous electronic didactic tools. In spite of the fact that educational technologies influence the current educa- tional process, professional and methodically elaborated assessment of their quality has not been a subject of ma- jor research projects by now. One reason is the excep- tional dynamics of technologies development and con- stantly expanding list of new devices and their functions. It disables the standardization of evaluation related ex- clusively to one specific software product or device. Which features characterize a modern and attractive digital learning tool? Which is useful for educational practice? What distinguishes it from other less appropri- ate software applications? Is it possible to measure the quality of any software solution by existing tools? Efforts to obtain relevant answers to these questions prompted us to carry out this research. 2. Educational Technologies and Their Categorization Determination of clear criteria, by which we could dis- tinguish the educational (pedagogical, academic) soft- ware, learning environment or more comprehensive in- formation system from other digital products, is an ex- tremely demanding task. From the point of view of pedagogical practice, this is a critical issue. When defin- ing the terms associated with the use of digital technolo- gies in the teaching process, most of the authors focuses on the pedagogical character of a product, which was designed, created, tested in practice and further devel- oped especially for educational purposes—as a tool for teaching and learning [4,6,14,17,20,27]. In this sense we understand the educational technologies as technological solutions, tools and devices designed specifically for the *This work was supported by the Slovak Research and Development Agency under the contract No. APVV-0266-11. ![]() Standardization of Quality Evaluation of Educational Software and Electronic Learning Tools—Analysis of Opinions of Selected Experts 572 needs of education and various educational and training activities. When studying the problems of introduction of the digital technologies into the educational process more deeply, we can also find the expressions considering any strict delimitation of a set of educational software as dangerous and incorrect. These attitudes are largely based on the premise, that any software becomes really educational only at the moment of its application in the educational process, or learning process, it means by the way of its usage [12,17,21,29]. Taking into account these opinions, it is necessary to enlarge the group of educa- tional technologies with specific software products, which in spite of the fact, that they were not specifically designed to support the pedagogical process, they have naturally found its application and became popular among teachers or students. A huge range and variety of educational software ap- plications and interactive educational environments dis- ables the introduction of uniform templates for evalua- tion of their quality. It is necessary to define the basic criterion/criteria for the categorization of existing soft- ware solutions and to evaluate all the products within one defined group. In addition there are several software classifications, and at the highest level we can divide the software products into free products (free of charge, shareware, they can be copied) and proprietary products (their usage is conditioned by the purchase of license; they cannot be modified or copied). Other divisions de- pend on the selection of criteria. Educational software can be divided by the way of presentation of processed information, role of the computer in the educational process, learning subject for which it is primarily in- tended, by educational paradigm, cognitive process phase of a learning person, for whom the software is primarily intended, by the software’s main function, recipient’s age, way of work etc. [1,3,20]. If we use a primary purpose of usage as a main criterion, most of the educational soft- ware environments used in schools is intended for prac- tising and studying of curriculum. Their popularity emerges from the fact, that they help teachers save time, facilitate the preparation of teaching aids (interactive tests, quizzes a exercises) and simplify the organization of learning process (individual and group work of stu- dents). The evaluation of results is accurate, fast and automatic. Reverse side of their frequent application in the educational process are repetitive activities and ques- tions focused most frequently on the knowledge of facts. When designing the propositions of standards for quality measuring of software with educational usage, we can start from the basic division of software solutions into three main groups: complex learning environments; instructive programs and Applets and finally software modules and accessories. As a main distinguishing fea- ture, we have defined the extent of provided and elec- tronically processed information and the associated range of application possibilities of usage of products in the educational process. A separate category consists of software tools for management, administration and sup- port of learning activities—LMS (learning management system), video conference systems, presentation programs and other unilaterally focused and highly specified digital teaching aids, e.g. interactive worksheets, crosswords, gap-fills, electronic forms etc. Into this category we can also add the web portals with video lectures, which can- not be changed and which are very often a part of e- learning courses and lessons. Complex learning environments contain robust web portals with a large database of educational materials (objects) for a number of learning subjects and classes, and also tools, which simplify the work with them (searching, authentication, preparation of students/teach- ers, education management, creation of individual tasks, interactive tasks, methodical guidelines, recommended procedures, worksheets, automated monitoring of student results etc.). Technical support and continuous updates of the electronic content is a matter of course. Instructive programs are specific software solutions, which partially replace a teacher in the interpretation of new curriculum, its practising and automatic evaluation of a level of student skills and knowledge. The electronic content processed by the program is usually limited to one specific discipline, thematic unit or teaching subject. Typical instructive programs include various comer- cially sold learning CD and DVD media with installa- tion .exe files (Terasoft, LANGMaster, SILCOM Multi- media...). Applets, software modules and accessories represent the smallest program units, which can be launched sepa- rately or as plugins of web pages. Their didactic usage is limited mainly to the mathematical calculations in more complex tasks, addition of missing data, explanation of problems or simulation of phenomena or processes. The interactive presentation mainly consists of animations supplemented with graphical and sound effects. Their progress can be influenced, e.g. by the change of the in- put quantities. There are also quite well-known Java ap- plets focused on the explanation of mathematical and physical relations (GeoGebra, FyzWeb-applets...). A special group of software products designed to sup- port the distance forms of education is so called LMS (Learning management system). It is an e-learning envi- ronment intended for creation, edition and management of educational projects (lessons, courses). Its application into the educational process requires not only a good knowledge of chosen environment, but also the complete preparation of the electronic content. This is not a part of Open Access JSEA ![]() Standardization of Quality Evaluation of Educational Software and Electronic Learning Tools—Analysis of Opinions of Selected Experts 573 an installation package and its quality depends exclu- sively on skills and experiences of the teacher. The most widely used system in Slovakia is LMS Moodle. Other used systems are LMS Claroline, Microsoft Class Server or Webcity. Video conference systems also do not provide the me- thodically elaborated electronic content. Only the topic of a virtual meeting, discussion character and ways of the provision of the relevant data can transform a video con- ference to the highly effective education tool. Video conference is actually a web-oriented environment, which enables the mutual interconnection of several par- ticipants in one video conference room. Voice and image transfer is supplemented with possibility of writing the text messages (chat) and presentation of multimedia data files. The high-quality products enable to share the run- ning applications and sophisticated tools of a team work. E. g. a widely used system EVO received a large support in the scientific circles. Its commercially sold alternative Adobe Connect Pro offers a simple access to the video conference. Presentation programs conclude a special group of software programs, which do not provide the users with materials of the educational nature. However, they help teachers and students present the information available in various digital formats in a very interesting form. Thanks to the programs the outputs have a multimedia character with interactive features. In this way they become dy- namic and visually attractive for students. An example of a typical presentation program in schools is the most widely used Microsoft PowerPoint. Good competition represent programs like Prezi, Adobe Captivate, Asham- poo Presentations and programs intended for the creation of concept maps—Xmind, MindMap, SmartDraw etc. 3. Criteria for the Quality Evaluation of Educational Software and Electronic Educational Tools Despite the fact that digital technologies are more and more applied in the educational and learning process, selection of an appropriate (professionally and didacti- cally correct) software is becoming an increasingly de- manding pedagogical problem—there is no thorough re- search in this field. Most of the published papers are fo- cused on the assessment of specific technological solu- tions (e.g. selected Open Source Software, LMS, CAL software, Zooming Editor Software, Electronic mobile devices, Cloud Learning, Cloud Computing etc.) and determination of their learning potential (see e.g. [2,5,15, 33]). There are many results presented, which analyze the effectiveness of application of selected digital tech- nologies in a specific learning situation and specific educational space (see e.g. [16,48]). The research pro- jects considering all the general principles of creation and objectivity of quality evaluation of modern electronic educational tools are, however, missing. In order to evaluate the quality of the educational software and elec- tronic learning materials, it is necessary to consider sev- eral aspects. From the point of view of the teacher and education, psychological-didactic and professional aspect of the software is extremely important. The evaluation from this point of view should include the fact how the software supports the management of education and evaluation of students. Equally important are the soft- ware technical parameters and user perspective on the evaluated digital product taking into account its individ- ual peculiarities. Educational aspect assesses the software value as a means of support of student various cognitive processes within the meaningful learning process and also the ex- tent, to which it contributes to their involvement in the learning process and to stimulation of their cognitive interests. The software should particularly respect the basic relations of perception, memory and thinking. From this perspective it is, for example, important to evaluate the clearness and intensity of incentives, mutual arrange- ment and use of various classes of multimedia operating objects, text, graphics (including animations and video) and sound. For efficient perception, understanding and learning it is important to simultaneously represent both basic modalities of provided information—it means vis- ual and verbal. To avoid overloading the attention and memory of a learning person, it is necessary to avoid the division of students’ attention to overlapping sources. Educational objects should also support cognitive proc- esses of selection, organization and integration. Illustra- tion of hierarchy and reciprocality between the blocks of information helps better understand their organization (principle of hierarchy). Organization of information helps learning person create the adequate mental repre- sentations (see e.g. [24]). From an educational point of view, it is essential to find out a purpose, validity, content complexity and ade- quacy, technical correctness of provided information, their compliance with curriculum, and also a range of possibilities of the software use in the educational proc- ess, motivation impulses and activities created by the software, provided feedback tools, as well as multimedia variety of the electronic content and preferred style of teachers and students work with the software. Attention should be paid to the fact, whether the software is able to take into account the variability of students, which is determined by their various abilities, cognitive style or preferred learning style, or by their handicap (see e.g. [22,35]). Another evaluated issue is also the interactivity of a software work, which can be understood as an ability Open Access JSEA ![]() Standardization of Quality Evaluation of Educational Software and Electronic Learning Tools—Analysis of Opinions of Selected Experts 574 to influence the behaviour and the course of the events in dependence on user’s requirements, and its openness, it means to what extent it is possible to change the default software environment and to enter its database (possibil- ity of adding own objects, of editing already created ob- jects etc.). User’s point of view pays attention mainly to the fea- tures of user environment also known as graphical user interface (GUI). An important thing is an overall clarity of a symbolic language and tools used by the software when communicating with user, as well as the transpar- ency of drivers’ location. We evaluate the software’s control, various settings options, appearance, quality of multimedia presented, overall stability (resistance to in- correct (unsupported) interventions, localization, soft- ware price, as well as the availability of trial version be- fore purchase and use of current technological trends. Another advantage can also be an obligation of the soft- ware producers to upgrade, extend and update the origi- nal product and its compatibility with other products. The learning process is also connected with teacher’s administrative activities, e.g. students data archiving, creation of various statistical summaries and final evalua- tions. Therefore it is important to find out how the edu- cation software helps teachers in the education manage- ment, students records and evaluation. We are monitor- ing the software performance, capacity, support of import, export, backup and archiving of student records, the abil- ity to record the structured data about students (classes), to manage and control the access to information sources, provision of possibility to evaluate the partial perform- ance, monitor students progress, create the personal stu- dent reports, summarizing tables, statistics, schemes and graphs. The real software application also depends on its tech- nical parameters (technical aspect). The subject of eva- luation should be in this case the software compatibility (start-up under various operating systems and their ver- sions), its hardware requirements, installation method, technical support, security and level of protection against unauthorized attacks. Specific objectives and various methods of use of in- dividual types of educational technologies require vari- ous approaches and evaluation criteria, which take into account their particularities. Regarding the instructive programs intended for the curriculum practising, the es- sential fact is whether they provide various levels of complexity of tasks or issues, whether the program set- tings enable to determine their number, order or varia- tions in the test, as well as the method by which they will be managed (options of restart, variation of time of the response, evaluation of answers correctness in blocks or by individual test items etc.). Concerning the simulation programs, the important thing is whether they allow the user to make decisions on every critical step, whether they provide realistic and credible consequences (effects) of selected actions and to what extent the factors facili- tating the proper understanding of a simulated pheno- menon or process are highlighted. 4. Research Methodology For the purpose of determining the users’ expectations and requirements in relation to the digital learning tools, we realized a questionnaire survey. We used the ques- tionnaires of own design with only open items. Since it is a very specific field of research (educational technolo- gies), only a relatively small sample of respondents was included–it means those who have met the requirements of experts or professional users of digital technologies. We have selected the Delphi method as a basis of the methodology. The questionnaires were provided inten- tionally to 3 selected groups of surveyed experts regu- larly using the digital technologies in everyday work or personal life. The experts gave their opinions on 3 areas of problems related to the most important expected fea- tures of software applications and electronic materials intended and created specifically for the needs of educa- tion and learning process support. We have addressed several experts and the following ones have participated in the research: 12 specialists on information technolo- gies, 11 teachers, who have been experimenting with application of digital technologies into the learning proc- ess in the long term and 12 respondents—users of infor- mation technologies, who are regularly using various digital tools and technologies in work. In order to ensure the respondents a quick and simple access to the ques- tionnaires, we have published them on our website: for IT specialists: http://www.evaluedu.sk /sk/home/formular-it-specialista, for users: http://www.evaluedu.sk /sk/home/formulare/formular-pou zivatel/ for teachers: http://www.evaluedu.sk /sk/home/formulare/formular-ucit el. 5. Research Results 5.1. IT Specialists Evaluation When taking into account all the selections of features made by IT specialists during the evaluation of the im- portance of the educational software features for its users, the following features occurred most frequently: clarity and simplicity, graphical processing, design variety, pos- sibility to access the software through the internet, clear- ness expressed by illustrative examples, videos or pic- Open Access JSEA ![]() Standardization of Quality Evaluation of Educational Software and Electronic Learning Tools—Analysis of Opinions of Selected Experts 575 tures, software interactivity and also its attractiveness for students-motivational function. The specific frequencies of selected features are stated in the Table 1. Table 1. Frequencies of answers of IT specialists to item 1: Try to identify at least 5 most important features of educa- tional software (electronic learning material), which you as a user would expect from this learning tool. Feature Frequency Frequency % clarity/simplicity/user friendly 9 75.0 graphical processing/design variety 6 50.0 possibility to access the software through the internet 5 41.7 clearness with illustrative examples, animations, pictures 3 25.0 interactivity 3 25.0 motivational/attractive 3 25.0 completeness of content with multimedia elements 2 16.7 intuitiveness 2 16.7 compatibility/universality for all OS 2 16.7 factuality/sim plicity of the text 2 16.7 perfection of data and information/proficiency 2 16.7 meets a learning objective/meaningfulness 2 16.7 contains a number of tasks for practising 2 16.7 Undo function 2 16.7 ergonomics 2 16.7 enables further work with information 2 16.7 free/open 1 8.3 Slovak language mutation/comprehensibility 1 8.3 simple and quick installation 1 8.3 quick searching of topics and information/quick feedback 1 8.3 intersubject relations/interconnection1 8.3 interestingly formulated text 1 8.3 possibility to deal with multiple functions 1 8.3 up-to-dateness 1 8.3 possibility to create the simple outputs 1 8.3 contains the interactive tasks 1 8.3 possibility to download the tasks 1 8.3 didactically consistent 1 8.3 Total 12 100.0 An interesting fact is that one third of respondents stated on the first position the software clarity and sim- plicity, what can be freely interpreted as an expression of preference of these features. The answers of other re- spondents related to the features stated on this position differed. The subject of our research was also focused on fea- tures of the educational software, which are important for the education manager, it means for teacher, instructor, lecturer etc. Most often, the IT specialists considered such features to be clarity, simplicity and comprehensi- bility, then features related to free sharing and software financial affordability, then features like testing of stu- dents knowledge, intermediation of results of student tests in a structured form, meeting the requirements for graphical processing and design of presented information, focus on essential content and forms of the curriculum, intermediation of curriculum in a form, which presup- poses its meaningful learning and remembering, then up- to-dateness, overall software attractiveness for students, options of editing of intermediated content and compati- bility of software with various operating systems and devices. Regarding the item No. 1: Try to identify at least 5 most important features of educational software (elec- tronic learning material), which you as a user would ex- pect from this learning tool, 33.3% of respondents stated the features clarity and simplicity in the first position, other responses occurred in this position only once. Taking into account all the selected features regardless of order, the most frequently reported features were clar- ity and simplicity, followed by graphical processing, de- sign variety, then option of accessing the software through the internet, clearness expressed by illustrative examples, videos or pictures, software interactivity and finally its attractiveness for students-motivational func- tion. Item No. 2 revealed the similar findings: Try to iden- tify at least 5 most important features of educational software (electronic learning material), which you as an education manager would expect from this learning tool (instructor, teacher, lecturer etc.). The most important feature of the educational software stated by the respondents was clarity and simplicity. This was stated by 27.3% of respondents. Other features were listed as preferred only once. In general, the most frequently occurring features were again clarity, simplicity and comprehensibility, then fea- tures related to free sharing and software financial af- fordability, then features like testing of students’ knowl- edge, intermediation of results of student tests in a struc- tured form, meeting the requirements for graphical proc- essing and design of presented information, focus on Open Access JSEA ![]() Standardization of Quality Evaluation of Educational Software and Electronic Learning Tools—Analysis of Opinions of Selected Experts 576 essential content and forms of the curriculum, interme- diation of curriculum in a form, which presupposes its meaningful learning and remembering, then up-to-date- ness, overall software attractiveness for students, options of editing of intermediated content and compatibility of software with various operating systems and devices. Item No. 3: Try to identify at least 5 most important technical features of educational software (electronic learning material), which you as a PC classroom admin- istrator (technician) would expect from this tool. This item revealed the following results: First features mentioned most frequently were re- quirements for simplicity of control, start-up and installa- tion; remaining responses were mentioned only once. In general, the most frequently mentioned feature ex- pected by IT specialists was simple administration of users and their accounts, followed by requirements for simplicity of control, installation and start-up of the software, then its compatibility with various operating systems and devices, easy upgrade, update, protection against hackers, system security against the attacks from the outside, minimalization of a need for service calls and finally financial affordability. A group of IT specialists considers a new educational software as a well-arranged and user-friendly educational tool, which is financially affordable for schools, teachers as well as for students. They expect from the software to be graphically interesting and attractive for students, with motivational features and examples using various multi- media tools. The software should be able to intermediate the curriculum on the professional level in a form, which would enable teachers to meet the educational objectives and help students facilitate the meaningful process of learning. It should be compatible with various operating systems and multiple devices such as tablet, iPad, inter- active table etc. PC classroom administrator should dis- pose of simple user’s administration and easy software upgrade with a minimum number of service calls. The software should by stable with sufficient protection against hackers. 5.2. Teachers Evaluation A group of teachers involved in the research consisted of teachers who are very well oriented in given field of digital technologies and who are regularly using the available technological solutions in their teaching prac- tice. Regarding the item No 1: Try to identify at least 5 most important features of educational software (elec- tronic learning material), which you as a user would ex- pect from this learning tool, 36.4% of respondents chose clarity and simplicity as a first feature, followed by Slo- vak language mutation, comprehensibility (27.3%) and finally completeness of content with multimedia ele- ments (18.2%). Taking into account all the mentioned features regard- less of order, the most frequently reported features were clarity and simplicity, followed by completeness of con- tent with multimedia elements, interactivity, Slovak lan- guage mutation, comprehensibility and graphical proc- essing, design variety. The specific frequencies of se- lected features are stated in the Table 2. Regarding the item No. 2: Try to identify at least 5 Table 2. Responses of teachers to item 1: Try to identify at least 5 most important features of educational software (electronic learning material), which you as a user would expect from this learning tool. Feature Frequency Frequency % clarity/simplicity/user friendly 9 81.8 completeness of content with multimedia elements 5 45.5 interactivity 5 45.5 Slovak language mutation/comprehensibility 4 36.4 graphical processing/design variety 4 36.4 shareware/financial affordability 3 27.3 clearness with illustrative examples, animations, pictures 3 27.3 quick searching of topics and information/auick feedback 3 27.3 compatibility/universality for all OS 2 18.2 factuality/simplicity of the text 2 18.2 free/open 1 9.1 localization 1 9.1 suitability for students considering their age 1 9.1 simple and quick installation 1 9.1 intuitiveness 1 9.1 up-to-dateness 1 9.1 intersubject relations/interconnection 1 9.1 consistent preparation for practising/verifying of knowledge 1 9.1 perfection of data and information/proficiency 1 9.1 interestingly formulated text 1 9.1 possibility to deal with multiple functions 1 9.1 information about experiments 1 9.1 Total 11 100.0 Open Access JSEA ![]() Standardization of Quality Evaluation of Educational Software and Electronic Learning Tools—Analysis of Opinions of Selected Experts 577 most important features of educational software (elec- tronic learning material), which would help you in the management (organization) of the educational process; the respondents stated different first features and opin- ions on the intuitively most important feature differed. In general, the most frequently occurring feature of the educational software was clarity, simplicity, comprehen- sibility and interactivity. These were followed by fea- tures like multimedia character, possibility to keep re- cords of students work, appropriate graphical processing, design and clearness with illustrative examples, anima- tions and pictures. Furthermore, from high quality soft- ware the teachers expect that it will be a shareware pro- gram, financially affordable and it will provide several difficulty levels and function “Undo”. Regarding the item No. 3: Try to identify at least 5 most important technical features of educational software (electronic learning material), including its formal proc- essing, which you as a teacher would expect from this tool, the opinions of respondents on the feature men- tioned as a first one, were for each respondent different. The most frequently mentioned features were an inter- esting design of user interface and its attractive graphical processing. These were followed by up-to-dateness and compliance of content with curriculum and textbooks. Another features expected by teachers from the software are financial affordability, content completeness, simple control, start-up and installation. Segmentation of the text should be well-arranged with logical interconnection of related topics and use of interactive elements. A group of innovative teachers, who are regularly us- ing the digital technologies in their learning process fo- cuses on the clarity and simplicity of educational soft- ware, not only on the control, but also on the start-up and orientation in it, logically interconnected topics with complete up-to-date content using the multimedia tools. The software should be graphically interesting with sev- eral difficulty levels and financially affordable for teach- ers as well as for students. The teacher should have the possibility to keep records of students work and interac- tively use all the existing electronic media formats, e.g. animations, pictures and videos. An important feature is Slovak language mutation of the software. 5.3. Users Evaluation Participated users daily use the available digital tech- nologies in their professional work and in this regard may be considered as sufficiently informed and experi- enced. Their opinions on the features of electronic edu- cational aids are therefore considered to be qualified. Regarding the item No. 1: Try to identify at least 5 most important features of educational software (elec- tronic learning material), which you as a user would ex- pect from this learning tool, 25% of respondents stated clarity and simplicity as a first feature, perfection of data and proficiency occurred less frequently (16.7%). Other responses occurred only once. When evaluating all the mentioned features regardless of order, the most frequently reported features were clar- ity and simplicity as well as perfection of data and in- formation and proficiency. These were followed by fea- tures like suitability for students considering their age, clearness with illustrative examples, animations, pictures, interesting graphical processing, design variety, up-to- dateness and ability to activate a user. The specific fre- quencies of selected features are stated in the Table 3. Other results were revealed by a question No. 2: Try to identify at least 5 most important features of presented content of educational software (electronic learning ma- terial), including its formal processing, which you as a user would expect from this learning tool. The respon- dents considered proficiency and quality to be the most important features in this case. This was stated by 33.3% of respondents. In general, the most frequently occurring responses were proficiency and quality followed by focus on stu- dents and clearness with illustrative examples, anima- tions and pictures. From an electronic learning tool the users expect multimedia character, clarity, simplicity, comprehensibility and meaningful content. Segmentation of the text, graphical processing and design of a tool should be appropriate for age of the student and should be focused on what it is supposed to teach. Item No. 3: Try to identify at least 5 most important technical features of educational software (electronic learning material), which you as a user would expect from such product, revealed the following results: First most frequently occurring response was simplic- ity of control, start-up and installation, together with perfect functionality and compatibility with various op- erating systems. In general, the most frequently mentioned feature was simplicity of control, start-up and installation together with compatibility with various operating systems. Fur- ther features were undemanding requirement for hard- ware and possibility of creation of various outputs. The remaining responses were mentioned 2 times or only once. A group of experts actively using the digital technolo- gies would appreciate a perfect and professional educa- tional software with simple control, start-up and installa- tion, which offers—with regard to age of the student—il- lustrative examples with animations, pictures and videos, and which is graphically interesting and attractive, com- patible with various operating systems, as well as with devices like tablet, iPad, interactive table etc. The users Open Access JSEA ![]() Standardization of Quality Evaluation of Educational Software and Electronic Learning Tools—Analysis of Opinions of Selected Experts 578 Table 3. Responses of users to item 1: Try to identify at least 5 most important features of educational software (electronic learning material), which you as a user would expect from this learning tool. Feature Frequency Frequency % clarity/simplicity/user friendly 6 50.0 perfection of data and information/proficiency 6 50.0 suitability for students considering their age 5 41.7 clearness with illustrative exam ples, animations, pictures 4 33.3 graphical processing/design variety4 33.3 up-to-dateness 4 33.3 ability to activate a user 4 33.3 free/open 3 25.0 shareware/financial affordability 3 25.0 interactivity 3 25.0 meets a learning ob_iective/meaningfulness 2 16.7 up-to-dateness 2 16.7 motivational/attractive 2 16.7 Slovak language mutation/comprehensibility 1 8.3 completeness of content with multimedia elements 1 8.3 factuality/simplicity of the text 1 8.3 intersubject relations/interconnection1 8.3 consistent preparation for practising/verifying of knowledge 1 8.3 occupies a small capacity 1 8.3 efficiency/help teachers better organize their work 1 8.3 possibility to access the software through the internet 1 8.3 possibility to combine it with another educational software 1 8.3 various difficulty levels 1 8.3 portable 1 8.3 contains a number of tasks for practising 1 8.3 constructive learning process 1 8.3 ecological 1 8.3 possibility to create the simple outputs1 8.3 update options 1 8.3 Total 12 100.0 also emphasize a well-arranged segmentation of the text, its meaningfulness as well as the possibility to create various kinds of outputs. According to their opinion, the software should not require a demanding hardware, so it will not be necessary to buy new devices and accessories for PC. 5.4. Results Summary The item No. 1: Try to identify at least 5 most important features of educational software (electronic learning ma- terial), which you as a user would expect from this learn- ing tool, was answered by all 3 groups—a total of 35 respondents. The most important feature stated by respondents in the first position was clarity and simplicity. This was mentioned by 31.4% of respondents. Taking into account all the mentioned features regard- less of order, the most frequently reported features were clarity and simplicity, followed by graphical processing, design variety, interactivity, clearness with illustrative examples, videos and pictures, perfection of presented data and information, proficiency and completeness of content with multimedia elements. The specific frequen- cies of selected features are stated in the Table 4. Item No. 2: Try to identify at least 5 most important features of educational software (electronic learning ma- terial), which you as an education manager would expect from this learning tool (instructor, teacher, lecturer etc.), or which would help you in the management of the edu- cational process, was formulated identically for the group of IT specialists and teachers. This group consisted of 23 respondents. According to the respondents, the most important fea- ture is clarity and simplicity. It was stated by 14.3% of respondents. In general, the most frequently occurring features were clarity, simplicity and comprehensibility. These were followed by interactivity, requirement for a free sharing and financial affordability of the software, high-quality graphical processing, design and possibility to keep re- cords of students work. Item No. 3: Try to identify at least 5 most important technical features of educational software (electronic learning material), which you would expect from this tool, was identical for the group of IT specialists and users. This group consisted of 24 experts. First most frequently mentioned responses were sim- plicity of control, start-up and installation, which was started by 27.3% of respondents. In general, the most frequently mentioned features were simplicity of control, start-up and installation and compatibility with various operating systems and devices, followed by simple users’ administration, undemanding Open Access JSEA ![]() Standardization of Quality Evaluation of Educational Software and Electronic Learning Tools—Analysis of Opinions of Selected Experts Open Access JSEA 579 Table 4. Responses of respondents to item 1: Try to identify at least 5 most important features of educational software (elec- tronic learning material), which you as a user would expect from this learning tool. Features Frequency Frequency % clarity/simplicity/user friendly 24 68.6 graphical processing/design variety 14 40.0 interactivity 11 31.4 clearness with illustrative examples, animations, pictures 10 28.6 perfection of data and information/proficiency 9 25.7 completeness of content with multimedia elements 8 22.9 Slovak language mutation/comprehensibility 6 17.1 shareware/financial affordability 6 17.1 suitability for students considering their age 6 17.1 possibility to access the software through the internet 6 17.1 free/open 5 14.3 factuality/simplicity of the text 5 14.3 up-to-dateness 5 14.3 motivational/attractive 5 14.3 quick searching of topics and information/quick feedback 4 11.4 compatibility/universality for all OS 4 11.4 meets a learning objective/meaningfulness 4 11.4 ability to activate a user 4 11.4 intuitiveness 3 8.6 intersubject relations/interconnection 3 8.6 up-to-dateness 3 8.6 contains a number of tasks for practisin g 3 8.6 simple and quick installation 2 5.7 consistent preparation for practising/verifying of knowledge 2 5.7 interestingly formulated text 2 5.7 possibility to deal with multiple functions 2 5.7 possibility to create the simple outputs 2 5.7 Undo function 2 5.7 ergonomics 2 5.7 enables further work with information 2 5.7 localization 1 2.9 information about experiments 1 2.9 occupies a small capacity 1 2.9 efficiency/help teachers better organize their work 1 2.9 possibility to combine it with another educational software 1 2.9 various difficulty levels 1 2.9 portable 1 2.9 constructive learning process 1 2.9 ecological 1 2.9 update options 1 2.9 contains the interactive tasks 1 2.9 possibility to download the tasks 1 2.9 didactically consistent 1 2.9 Total 35 100.0 ![]() Standardization of Quality Evaluation of Educational Software and Electronic Learning Tools—Analysis of Opinions of Selected Experts 580 requirement for hardware, protection against hackers and overall product security. In regard to individual participated groups of respon- dents, we found out a significant similarity in evaluating the importance of features of educational software or educational learning tools. IT specialists, teachers and qualified IT users consider the basic and most important features of these tools to be simplicity and clarity in in- stallation, start-up and orientation, then interesting and attractive graphical interface containing the elaborated technical and visual elements, e.g. animations, videos and pictures, compatibility with various operating sys- tems and devices like tablet, iPad, etc. The respondents also think that the educational software should be finan- cially affordable for schools, teachers and students. It should be complex in terms of the content, and should also meet the professional criteria. When we are talking about individual items, each of the groups stated some- thing special, what should not be missing from its point of view. However, in all the groups and all the items these features were almost identical. 6. Conclusion The introduction of standardized evaluation tools, which are able to competently evaluate the quality of educa- tional software and electronic learning materials, is com- plicated because of a number of factors. One of the most important factors is the determination of appropriate, expected or directly required features, which are crucial for the classification of software product as a recom- mended and high quality learning tool. The research re- alized among respondents—specialists revealed rela- tively clear results. A high level of respondents’ agree- ment defined the simplicity and clarity in installation, start-up and orientation as most important features of educationally focused software products. Other important features were represented by interactivity, multimedia content, various forms of intermediation of information, e.g. texts, pictures, videos, animations, technical and fi- nancial affordability for users, etc. The research also con- firmed that the extent of expected software features de- pended on its content, focus and purpose, for which it was intentionally created. REFERENCES [1] S. Bostock, “Classification of Educational Software,” Keele University, 1995. http://www.keele.ac.uk/depts/aa/landt/lt/docs/atcbttyp.htm [2] S. Bleck, M. Bullinger, A. Lude and S. Schaal, “Elec- tronic Mobile Devices in Environmental Education (EE) and Education for Sustainable Development (ESD) Eva- luation of Concepts and Potentials,” Procedia-Social and Behavioral Sciences, Vol. 46, 2012, pp. 1232-1236. [3] Open Learning Technol- ogy Corporation Limited, “CAL Software Classifications,” 1996. http://www.educationau.edu.au/archives/CP/07.htm [4] co_hodnotit, 2008. http://pdf.uhk.cz/kch/evaluace_SW/co_hodnotit.htm [5] C. Conboy, S. Fletcher, K. Russell and M. Wilson, “An Evaluation of the Potential Use and Impact of Prezi, the Zooming Editor Software, as a Tool to Facilitate Learning in Higher Education,” In: Inovations in Practice, Vol. 7, 2012, pp. 31-45. [6] Criteria for Evaluating Software, 2001. http://www.colleges.org/software_eval/criteria.html [7] Educational Software Evaluation, 2009. http://www.humboldt.edu/~ra7001/pages_courses/sed718 /formb.html [8] Evaluace Výukového Software, 2007. http://pdf.uhk.cz/kch/evaluace_SW/evalindex.htm [9] Educational Software Evaluation, 2002. http://www.sc.edu/etc/eval.etceval.htm [10] Educational Software Evaluation Criteria, “Freshpond Software Review,” 2001. http://www.freshpond.net/treasures/technology/softreview s/softeval.htm [11] R. Fisher, “Učíme děti Myslet a učit se,” Praha, Portál, 2011, 176 p. [12] R. Foshat and M. I. Ahmed, “A Practical Process for Re- viewing and Selecting Educational Software,” PLATO Learning, Inc., 2003. [13] “Kritériá na hodnotenie kvality učebnice pre všeobecno- vzdelávacie predmety,” 2008. http://www.statpedu.sk/buxus/docs//vyskum/ucebnicova_ politika/hodnotenie_kvality_ucebnice.pdf [14] “IT232—Pedagogical Software,” 2009. http://www.idi.ntnu.no/~terjery/IT232/EnglishIT232.html [15] J. Jeff, “Evaluation of CAL Software for Higher Educa- tion: A Task for Three Experts,” Enhancing Learning, Teaching & Curricula with a University-Wide Integrated World Wide Web Framework. The Hong Kong Polytech- nic University, Educational Development Centre, 2013. [16] K.-T. Yang and T.-H. Wang, “Interactive White Board: Effective Interactive Teaching Strategy Designs for Bi- ology Teaching,” In-tech, E-Learning-Engineering, On- Job Training and Interactive Teaching, 2012, pp. 139- 154. [17] I. Kalaš, “Mýty a Vízie O Informatizácii Školy,” In: Zborník Príspevkov z 5. Celoštátnej Konferencie Infovek, Bratislava, 2005, pp. 35-42. [18] S. Kemmis, R. Atkin and E. Wright, “How Do Students Learn?” Working Papers on Computer Assisted Learning, Centre for Applied Research in Education, Norwich, 1977. [19] Z. Kolář and A. Vališová, “Analýza Vyučování,” Grada, Praha, 2009, 232 p. [20] D. Lehotská, “Edukačný Softvér,” In: Matematika Infor- Open Access JSEA ![]() Standardization of Quality Evaluation of Educational Software and Electronic Learning Tools—Analysis of Opinions of Selected Experts 581 matika Fyzika, Roč. 16, č. 30, 2007, pp. 16-23. [21] J. Lever-Duffy and B. J. Mc. Donald, “Teaching and Learning with Technology,” 4th Edition, Pearson Educa- tion, Boston, 2011. [22] J. Mareš, “Styly Učení Žáků a Studentů,” Portál, Praha, 1998, 239 p. [23] S. Papert, “Jean Piaget,” The TIME 100. http://www.time.com/time/time100/scientist/profile/piage t03.html [24] J. Pavlíček, “Projektování Výuky pro Inovativní Výukové Prostředí,” 2. vyd, Ostravská Univerzita v Ostravě, Os- trava, 2008, 100 p. [25] J. Piaget, “Logic and Psychology,” Manchester Univer- sity Press, New York, 1953. [26] J. Piaget, “Psychologie Inteligence,” SPN, Praha, 1970. [27] Program, “Definition and Much More from Answers.com,” 2009. http://www.answers.com/program [28] J. Prucha, “Encyklopedie Pedagogiky,” Portál, Praha, 2009. [29] M. Resnick, “Rethinking Learning in the Digital Age,” In: G. Kirkman, Ed., The Global Information Technology Report: Readiness for the Networked World, Oxford Uni- versity Press, Oxford, 2002. [30] L. P. Rieber, “Microworlds,” In: D. Jonassen, Ed., Hand- book of Research for Educational Communications and Technology, 2nd Edition, Lawrence Erlbaum Associates, Mahwah, 2004, pp. 583-603. [31] M. Roblyer, “Integrating Educational Technology into Teaching,” Pearson Education, Boston, 2012. [32] K. L. Seifert and R. J. Hoffnung, “Child and Adolescent Development,” Hougton Mifflin Comp, New York, 1994. [33] D. B. Shinde and S. D. Gaikwad, “An Evaluation of Se- lected Open Source Software for Digital Libraries,” Seva Sadan’s College of Education, Ulhasnagar, 2013. [34] M. Simpson and F. Payne, “Using Information and Com- munications Technology as a Pedagogical Tool: Who Ed- ucates the Education?” 2009. http://www.leeds.ac.uk/educol/documents/00001292.htm [35] V. Smékal, “Pozvání do Psychologie Osobnosti. Člověk v zrcadle Vědomí a Jednání,” Barrister & Principal, Brno, 2002, 517 p. [36] “Software Evaluation”. http://www.ed.brocku.ca/~jkerr/sftwreva.htm [37] “Software: Definition and Much More from Answers.com”. http://www.answers.com/topic/computer-software [38] R. Taylor, “The Computer in the School: Tutor, Tool, Tutee,” Teachers College Press, New York, 1980. [39] “Teachernet, Learning, Teaching and Managing Using ICT”. http://www.teachernet.gov.uk/wholeschool/ictis/ [40] “The State of Children’s Software Evaluation”. http://www.childrenssoftware.com/evaluation.html [41] “Transforming Teaching and Learning through ICT in Schools,” 2004-2007. http://www.northamptonshire.gov.uk/NR/rdonlyres/301B 8C79-2F78-4A6A-B9E6-24E6AC3A0136/0/TTandLICTi nSchoolsStrategy200407finalversionpagesinorder.pdf [42] M. Veselský, “Pedagogická Psychológia 1. Teória a Prax,” Univerzita Komenského, Bratislava, 2004. [43] M. Veselský, “Pedagogická Psychológia 2. Teória a Prax,” Univerzita Komenského, Bratislava, 2008. [44] A. Vitovský, “Moderní Slovník Softwaru. Výkladový. Anglicko-Český a Česko-Anglický,” AV software, Praha, 2006. [45] “Výkladový Terminologický Slovník Electronických Ko- munikácií”. http://www.vus.sk/iecd/new/Vyklad.asp [46] “What Is Software?—A Word Definition from the We- bopedia Computer Dictionary”. http://www.webopedia.com/TERM/s/software.html [47] “What the Research Says about Using ICT in Science”. http://www.becta.org.uk/page_documents/research/wtrs_s cience.pdf [48] Z. H. Li, “Superiority of Multimedia Technology in Ge- ography Teaching,” In: T. Zhang, Ed., Future Computer, Communication, Control and Automation, Springer-Ver- lag Berlin Heidelberg, 2012, pp. 581-585. Open Access JSEA |