Evaluating a Student MIS in the Tertiary Education Sector: Addressing Functional-Operational Misalignment through Continuous Participative Evaluation

Marian Carcary

.

DOI: 10.4236/jssm.2010.31004                

Abstract

The Information and Communications Technology (ICT) evaluation literature spans several decades. ICT evaluation approaches range from objective and positivistic to subjective and interpretive. While positivistic approaches have dominated the past, there is increasing recognition of the value of interpretivist methods and the need for ongoing project evaluation. Formative continuous participative evaluation (CPE) offers several benefits in terms of project control, enhanced stakeholder relationships and benefit realisation; nonetheless this is often ignored in practice. There is a paucity of ICT evaluation within the Higher Education sector. The 14 Irish Institutes of Technology (IoTs) recently underwent an extensive transformation of their ICT systems, through a nationwide implementation of a suite of integrated IS. This research study, centred on the evaluation of the Student MIS implementation was interpretive in nature; case studies were conducted in five IoTs. This paper focuses specifically on one issue uncovered through the research i.e. the misalignment between the Student MIS and the IoTs requirements. The paper proposes a set of guidelines for addressing this issue through focusing on the theoretical underpinnings of CPE and its importance for organisational learning and benefit realisation.

Keywords

ICT Investment Management, ICT Evaluation, Formative Evaluation, Continuous Participative Evaluation, MIS, ICT in Tertiary Education

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	M. Wilson and D. Howcroft, “Power, politics and persuasion in IS evaluation: A focus on ‘relevant social groups’,” Journal of Strategic Information Systems, Vol. 14, pp. 17–43, 2005.
[2]	W. Chen and R. Hirschheim, “A paradigmatic and methodological examination of information systems research from 1991 to 2001,” Information Systems Journal, Vol. 14, pp. 197–235, 2004.
[3]	S. Cronholm and G. Goldkuhl, “Strategies for information systems evaluation-six generic types,” Electronic Journal of Information Systems Evaluation, Vol. 6, No. 2, pp. 65–74, 2003.
[4]	C. Miskelly, A. Moggridge, C. Stephenson, and M. Williams, “Formative and interpretive approaches to the evaluation of community focused web-fronted projects,” Proceedings of the 11th European Conference on Information Technology Evaluation, Amsterdam, The Netherlands, Academic Conferences, Reading, pp. 281–288, 11– 12 November, 2004.
[5]	A. Roudsari, E. J. Berridge, M. Vodel, and E. Carson, “Design and evaluation of ICT-based patient information systems to support the management of chronic diseases,” Proceedings of the 10th European Conference on Information Technology Evaluation, Madrid, Spain, MCIL, Read- ing, pp. 547–556, 25–26 September, 2003.
[6]	N. Shiratuddin and M. Landoni, “Evaluation of content activities in children’s educational software,” Proceedings of the 8th European Conference on Information Technology Evaluation, Oxford, United Kingdom, MCIL, Reading, pp. 79–87, 17–18 September, 2001.
[7]	M. Gemmell and R. Pagano, “A post-implementation evaluation of a student information system in the UK higher education sector,” Electronic Journal of Information Systems Evaluation, Vol. 6, No. 2, 2003.
[8]	N. Doherty and L. McAulay, “The relationship between the ex ante and ex post information systems evaluation: Reflections from the literature and the practice of evaluating e-commerce investments,” Proceedings of the 9th European Conference on Information Technology Eva- luation, Paris, France, MCIL, Reading, pp. 129–136, 15–16 July, 2002.
[9]	E. Berghout and M. Nijland, “Full lifecycle management and the IT management paradox,” In Remenyi, D. and Brown, A. (eds.), Make or Break Issues in IT Management, Butterworth Heinemann, Oxford, pp. 77–107, 2002.
[10]	N. Doherty and M. King, “The treatment of organisational issues in systems development projects: The implications for the evaluation of information technology investments,” Electronic Journal of Information Systems Evaluation, Vol. 4, No. 1, 2001.
[11]	B. Farbey, F. Land, and D. Targett, “Evaluating investments in IT: Findings and a framework,” In Willcocks, L.P. and Lester, S. (eds.), Beyond the IT Productivity Paradox, Wiley, Chichester, pp. 183–215, 1999.
[12]	B. Farbey, F. Land, and D. Targett, “Moving IS evaluation forward: Learning themes and research issues,” Journal of Strategic Information Systems, Vol. 8, No. 2, pp. 189–207, 1999.
[13]	Z. Irani, A. M. Sharif, and P. E. D. Love, “Linking knowledge transformation to information systems evaluation,” European Journal of Information Systems, Vol. 14, pp. 213–228, 2005.
[14]	L. P. Willcocks, “Evaluating information technology investments: Research findings and reappraisal,” Journal of Information Systems, Vol. 2, No. 3, pp. 243–268, 1992.
[15]	D. Remenyi, M. Sherwood-Smith, and T. White, “Achie- ving maximum value from information systems,” Wiley, Chichester, 1997.
[16]	J. R. Cordoba, “Critical reflection in planning information systems: A contribution from critical systems thinking,” Information Systems Journal, Vol. 19, No. 2, pp. 123–147, 2009.
[17]	V. Serafeimidis and S. Smithson, “Information systems evaluation as an organisational institution experiences from a case study,” Information Systems Journal, Vol. 13, pp. 251–274, 2003.
[18]	L. P. Willcocks, G. Fitzgerald, and M. Lacity, “To outsource IT or not? Research on economics and evaluation practice,” In Willcocks, L. P. and Lester, S. (eds.), Beyond the IT Productivity Paradox. Wiley, Chichester, pp. 293–333, 1999.
[19]	C. E. Hillam and H. M. Edwards, “A case study approach to evaluation of Information Technology/Information Systems (IT/IS) investment evaluation processes within SMEs,” Electronic Journal of Information Systems Evaluation, Vol. 4, No. 1, 2001.
[20]	M. L. Markus, “Technochange management: Using IT to drive organisational change,” Journal of Information Technology, Vol. 19, No. 1, pp. 4–20, 2004.
[21]	M. Carcary, “ICT evaluation in the Irish higher education sector,” Electronic Journal of Information Systems Evaluation, accepted for publication in Vol. 12, No. 2, 2009.
[22]	B. G. Glaser and A. L. Strauss, “The discovery of grounded theory: Strategies for qualitative research,” Aldine De Gruyter, New York, 1967.
[23]	M. Alvesson and K. Sk?ldberg, “Reflexive methodology-new vistas for qualitative research,” Sage Publications, London, 2000.
[24]	B. Czarniawska, “Narratives in social science research- introducing qualitative methods,” Sage Publications, Thou- sand Oaks, London, 2004.
[25]	G. Avram, “Evaluation of investments in ecommerce in the Romanian business environment,” Electronic Journal of Information Systems Evaluation, Vol. 4, No. 1, 2001.
[26]	H. Saastamoinen, “Exception-based approach for information systems evaluation: The method and its benefits to information systems management,” Electronic Journal of Information Systems Evaluation, Vol. 8, No. 1, pp. 51–60, 2005.