Developing a Conceptual Framework for GIS Audit in an Enterprise Setting ()
1. Introduction
1.1. Background
GIS is a growing area of information technology and the increasing demand for geospatial information and tools has made GIS more pervasive (United Nations, 2013). With the massive geospatial data collected due to technological advancements in data capturing equipment and techniques (Song & Wu, 2021), GIS has created an easier and wider access to information for all. In this regard, the conceptual framework aims at identifying GIS audit parameters for a systematic approach in achieving GIS best practices and quality of outputs. The world is facing global geospatial challenges in terms of assessment of geospatial capacity for accurate, timely and comprehensive data to inform policies (Rabiee, 2020). There is a need to assess GIS and examine challenges faced by organizations in sustaining GIS operations (Al-Kodmany, 2012). To realize the success of GIS application, awareness of its context and specificities is necessary (Pereira, Varajão, & Takagi, 2022).
Although human factors are largely responsible for GIS system failures (Madni & Jackson, 2009), lack of monitoring and evaluation equally contributes to failure (Kheybari, Rezaie, Naji, & Javdanmehr, 2020) because this phase of project implementation provides an opportunity to identify areas for improvement.
Decisions made from geospatial data analysis are only viable if the data used is consistent, accurate or up to date among other qualities (Kin & Lazorenko-Hevel, 2021). Unfortunately, this is not the case as some GIS application outcomes usually invalidate the results of geospatial data analysis (Montserrat & Sendra, 2009). This points to inadequate quality checks and quality assurance by GIS users that ensure that data used in GIS applications is fit for use.
Al-Waraqi & Zahary (2013) and Business Mapper (2022) have cited some of the reasons why GIS fails which include: lack of access to appropriate data, underutilization of GIS software, lack of skilled personnel and staff continuity, data management, operational management, information technology infrastructure and lack of focus among others. In order to address these challenges, auditing a GIS system is crucial to ensure an efficient, effective and a responsive GIS that meets user’s need (Al-Kodmany, 2012).
In this conceptual framework, identification of appropriate GIS audit parameters is done through examination of variables that impact GIS performance in an organization.
1.2. Auditing GIS in an Enterprise Setting
With powerful tools and scalability in an enterprise GIS (Zhang et al., 2019), data and information is collected, stored, integrated and accessed across various departments (Peery & Wilson, 2019). This data is subjected to various applications and its quality is vital for an effective and informative outputs. Data and information sharing within an enterprise GIS involves professional and non-professional users, with coordination among many departments. However, there hasn’t been a laid down mechanism of evaluating this data (Budic & Pinto, 2000). There is need to ensure proper integration, analysis, documentation and management of data within an enterprise GIS.
Enterprise GIS should have long-term strategic goals and its operations shaped by the prevailing information technology trends (Peery & Wilson, 2019). This necessitates routine evaluation to ensure sustained performance and appropriate controls for the system. Likewise, due to its broad nature, an enterprise GIS requires GIS functions to be optimized with respect to the resources and application tools (Esri, 2007). There is need for a framework that provides a method to align geospatial technologies and capabilities to business processes (Esri, 2021).
This paper focuses on auditing GIS within an organization to access appropriate implementation, operations and control of its resources. With multiple units within an enterprise GIS (Budic & Pinto, 2000), audit commitment should be higher compared to a project or department GIS where data holding knowledge is exclusive to source department or project team (Peery & Wilson, 2019). This will provide expected benefits of efficiency and effective data management within an organization. It also fulfils the requirements of an entire organization and its flexibility to meet future growth through monitoring and evaluation. Failure of GIS audit in an enterprise setting can be disastrous considering huge investments that usually accompany its establishment (Peery & Wilson, 2019).
The main objective of this paper is to provide a conceptual framework for evaluating an established GIS in an organizational setting.
The paper is organized into four sections. After this introduction, the practice of GIS performance monitoring and evaluation is reviewed in section two, while section three presents and discusses the GIS audit parameters identified in this study. The paper ends with conclusion and recommendation of GIS audit framework appropriateness with an outlook.
2. Existing GIS Audit Frameworks
2.1. Review of Existing GIS Audit Frameworks
Information system auditing is considered as a systematic process of gathering and assessing the existing information to ensure consistency between what has been found and a set of established criteria (Munteanu, 2016). Information system auditing requires an opinion about the information systems as well as data that they process (ALraja & ALomiam, 2013).
GIS is one of the existing information system and its development is a continuous process and the quality of its services has emerged as an important research topic in geoinformatics (Hu, Gui, Cheng, Wu, & McClure, 2019). As much as GIS audit would follow the criterion of information system audit category, a separate audit framework, different from other information system audit is necessary to capture GIS elements not found in other information system audit (Cho, Kim, Kim, & Han, 2012).
From literature there are various distinctions between GIS and other information systems. The spatial context exclusive from other information systems makes GIS distinct and complex from general information systems (Hwang, 2010). The diversification of GIS technology (Cho, Kim, Kim, & Han, 2012), such as the concept of big spatial data (Yue & Jiang, 2014), evolving technologies, hybrid nature and data integration capabilities may warrant new ways of auditing GIS.
During GIS system installation, Functional Requirements Study (FRS) is normally done as a primary planning document (Ferguson, Cartotech, & Antonio, 1990). It provides an evaluation of GIS functionality and any new requirements. It spells out what data is needed and how it must be processed. However, FRS is project specific and may not apply after a project’s implementation period which then warrants that a more generic framework is used. An FRS was done by (Brown, 2016) for auditing GIS to evaluate quality management, consistency and mitigating risk. This mainly concentrates on verification of GIS requirements in terms of data, procedures and personnel. The main focus being to perform GIS needs assessments and identify areas of improvements. It does not provide guidance on major checklists that ensures quality application results.
Mutua & Mwaniki (2017) have done FRS to assess GIS system requirements and functionality i.e. space/office, hardware, software, personnel/staff and data. However this focused on GIS needs assessment by listing crucial GIS components that must be put in place. The element of monitoring and evaluating these components through audit is lacking.
Various attempts on GIS Capability and Maturity Models (CMM) have been made. For example, Slim GIM framework, URISA GIS CMM and PSD Geospatial Maturity Index (GMI). These models are used access existing state of an established enterprise GIS. They define typical properties of a capable and mature enterprise GIS (Johnstone, 2018).
Slim GIM framework incorporates an enterprise architecture to assess challenges in the context of uniqueness and readiness levels within a municipal or local government setting. It ensures understanding of key organizational factors in order to address issues which impact management of information within the organization (Giroux, 2014).
URISA’s GIS CMM, provides a theoretical model of a capable and mature enterprise GIS program within a designated organization (Johnstone, 2018). The model facilitates discussions on appropriate components and characteristics of a capable and well-managed enterprise GIS (Urisa, 2013).
The PSD GMI involves a survey done to assess GIS maturity and capacity in the public sector. The survey captures GIS maturity from respondents in terms of readiness, implementation, and impact. The results are used to support performance benchmarking (Sutton, 2018).
Nevertheless, there is need for a generic GIS audit framework because these GIS CMM are designed for specific GIS operations within public sector. They can be time consuming and require calibration and customization for a more meaningful result in order to fit users need for other sectors (Johnstone, 2018).
Parks Canada Agency (2012) has developed a GIS performance audit that seeks to provide on whether GIS activities within the institution are aligned with and supports the mandate of the agency. Support in relation to appropriate GIS governance structure, efficiency, effectiveness and economic use of allocated funds. Not all aspects of GIS audit have been capture in this performance audit.
GIS Audit checking items have been studied by Cho, Kim, Kim, & Ha (2012). They have examined GIS characteristics, development methodology and spatial data models and suggested GIS supervision check items. The suggested check items only covers GIS technical supervision and does not adequately provide an expansive check that evaluates overall GIS performance.
In terms of auditing competency of GIS workforce, competency models have been developed by DiBiase et al. (2010) and Urisa (2012). These models have identified fundamental geospatial knowledge and abilities that serves as a guide to various groups of people such as geospatial workers, students, educators and certification/accreditation bodies. They provide essential competency areas specified to describe the work of a successful geospatial industry manager but not all-inclusive audit contents on critical GIS functions have been provided.
From the review, existing GIS audit attempts do not reflect an audit checklist that takes into account complete evaluation of an established GIS. Various issues/gaps/weaknesses have been identified namely:
1) They don’t provide audit parameters in relation to basic GIS components that integrate and ensures fitness for use for a functional GIS system. The concentration is on monitoring GIS activities and resources but lacks guidance on some aspects of monitoring and evaluating an overall GIS system. They also don’t provide an element of structured guidelines on frequency of auditing GIS resources in order to understand their state and competencies.
2) No guidelines on limits of exploiting fundamental geospatial datasets.
3) Not exhaustive for overall GIS system competencies in all critical GIS work functions. Focus is more to managerial professional development and no performance measure are described to support competencies that facilitate production of quality GIS results.
4) No guidelines on evaluating extent of utilizing GIS software tools and functionalities.
2.2. Summary of Literature Review
Existing GIS audit attempts presented in literature review indicates major relationships with regards to improving GIS quality of service as presented by FRS, GIS CMM and GIS performance audits reviewed. Though they provide insights on GIS readiness on specific operations or projects within an enterprise GIS, a generic approach of GIS audit that accommodates all aspects of GIS is required. It is necessary to outlines concepts and ideas on fundamental GIS principles and functionalities in order to identify optimal GIS audit parameters considered appropriate for all aspects of GIS system needs. Nevertheless, this research has borrowed ideas from the reviewed literature.
2.3. Conceptual Model of a GIS Audit Framework for an Enterprise GIS
A conceptual model is a description of concepts and ideas that forms the representation of the research topic (Thalheim, 2012). It provides logical orientation and associations of variables that form the underlying structure of entire research (Kivunja1, 2018). It supports the understanding of research context and the problem that needs to be addressed (Robinson, Arbez, Tolk, & Wagner, 2015). In this case, this conceptual model provides a foundation for interpreting appropriate GIS audit parameters as it displays underlying factors for assessing requirements of a functional GIS system. It depicts full dynamics of an enterprise GIS implementation that supports successful performance. It presents expected relationships of variables within a GIS system. Figure 1 explores the concept of a conceptual model for GIS audit.
Independent variables identified include GIS components (Data, Software/Hardware, People and Methods) and organization culture. GIS performance has been identifies as the dependent variable. Intervening variables that describes the link between independent and dependent variables are identified to include system quality, GIS management activities and Monitoring and evaluation activities.
The independent variable of GIS components comprises data, software/hardware, people and methods. Aspects of data quality within a GIS implementation are characterized as fitness for use that facilitates quality of service for excellent GIS performance (Bielecka & Burek, 2019). People represents GIS competency that demonstrates knowledge, skills and abilities essential for a
![]()
Figure 1. Conceptual model (Source: Owner).
successful GIS performance (DiBiase et al., 2010). GIS software enables execution of GIS functions that support decision making during various GIS applications. These functions support capturing, displaying, editing, manipulation, analysis, modelling and presentation of GIS datasets (Miller et al., 2018). The performance of GIS is dependent on how well the GIS software is utilized to execute these functions.
The methods component represents GIS operating procedures that must be put in place for a successful GIS performance. GIS operate according to well-designed plans and rules unique to each organization and with respect to application at hand (Ali, 2020).
Organization culture within an enterprise GIS influences the aspects of GIS system management and support activities such as resource coordination; mechanisms of quality control; sound, responsible and well defined organizational structure (Ahmed, 2021) among others. The ability of an Organization culture to align to the laid down structures and strategies influences GIS performance.
The GIS management and support activities are intervening variables as they influence decision during implementation hence affecting GIS performance. The quality of GIS system and realistic monitoring and evaluation of GIS activities, data and technology influences overall performance of GIS. System quality and monitoring and evaluation activities are influenced by organizational culture which determines how well they are executed.
The conceptual model describes the concept of an enterprise GIS whose functionality is shaped by dependent, independent and intervening variables that support its implementation. The inter-relations between these variables have been investigated through theories and existing literature. They have provided a direction in formulating basic guiding questions in seeking to provide GIS audit parameters that lead to improved quality of service for an established GIS program which includes:
1) What are the parameters of a functional GIS system? In order to come up with appropriate GIS audit parameters, it is important to understand basic GIS concepts and theories that constitutes and integrates a working/functional GIS
2) What are the most likely failures of a GIS system? Both human and organizational factors may increase the likelihood of system failure (Madni & Jackson, 2009). A resilient GIS System facilitates successful performance.
3) What are the most likely challenges in GIS development? Research shows that complex systems development, such as GIS pose greater safety and risk management challenges (Madni & Jackson, 2009). Recognition of adverse conditions that attribute to these challenges enables identification of appropriate audit parameters.
3. Audit Parameter Identification
3.1. Method
Desktop review through theories and existing literature has been applied as the main method of understanding GIS context to help in identifying optimum GIS audit parameters. GIS system contextual factors such as components, inputs, outputs and interactions have been investigated to reveal parameters that contribute quality audits. These have been investigates with the help of conceptual model as demonstrated in Figure 1. The following have been identified to reveal and present directions on optimal GIS audit parameters required:
1) Data quality aspects for a successful GIS applications and performance
2) GIS software functionality aspects that guides optimal utilization of tools for performing GIS functions
3) GIS competency aspects for thorough application of spatial reasoning
4) GIS system procedures for quality checks and quality assurance for successful implementation and lasting GIS program
These aspects present major expectations of a functional GIS system. In view of identifying appropriate GIS audit parameters based on these aspects, the conceptual model variables and their interactions have been reviewed. The review was based on GIS concept appropriate for all aspects of its implementation and management as presented in conceptual model Figure 1. This concepts is elaborated in Figure 2 consisting three layers; C1, C2 and C3. Identification of appropriate GIS audit parameters is reviewed in Layer C1. This layer is core and is intended to examine vital elements for a conducive GIS environment as presented in Layer C2. This is an environment that integrates GIS components and ensure GIS functions are executed in observance of GIS industry dynamics, technological trends, integration with other information systems or disciplines, organizational management and performance areas as described by (Shastri,
![]()
Figure 2. Categories of audit parameters (Source: Owner).
2020). For a sustainable GIS implementation, these features need to be properly monitored and evaluated in terms of quality, utilization and performance. Moreover, GIS industry is dynamic and the ever changing technological trends (Walter, 2020) points to a regular monitoring and evaluation of emerging technologies to keep updated. In this respect, four main categories of GIS audit parameters deemed optimal for a sustainable GIS implementation have been identified as indicated by Layer C3; Data Quality, Software Utilization, GIS Competency and GIS Procedures. For each main category, associating minor audit parameters were identified as presented in Table 1. Associating audit parameters present general characteristics considered relevant in defining each category. They comprise specific audit aspects of each category depending on their effect on various GIS applications and impact to overall GIS system setup.
The four categories of parameters are generally related to basic GIS components and demonstrate essential audit themes that must be put in place to ensure GIS activities maintain high quality and generally meet the needs of the organization. They form core infrastructure of a GIS system (Esri, 2021), hence their evaluation cannot be ignored. During implementation, a fault caused by any of the identified category may render obsolete results for entire GIS application. In this regard, it is important to evaluate each category in relation to associated
![]()
Table 1. Categorized audit parameters.
audit parameters. No ranking of parameters is done whatsoever. However, data quality parameters are emphasized as primary elements (Gabriel & Holanda, 2019) in which initial audit efforts should be directed. This is because there exist different techniques of data collection with different formats and integration methods (Song & Wu, 2021), hence need to be carefully evaluated to ensure fitness for intended application.
3.2. Results and Discussions
Four main categories of GIS audit parameters were identified. Each category is represented by various associating minor parameters which provides insights on how, where and what to look for during GIS audit. GIS application revolves around best utilization of its components (Mierzejowska & Zogała, 2018). The main categories of parameters are identified with respect to basic GIS components; data, people, software and procedures. The parameters reflect monitoring and evaluation elements within the concept of GIS establishment, implementation and sustainability (Mathot, 2020). Associated parameters are meant to evaluate quality of GIS system and evade contradicting decisions which are sometimes subjected to legal issues (Verrax, 2017). Parameters associated with data quality are of essence bearing in mind that access to geospatial data within an enterprise GIS involves many users and this may bring data quality related challenges (Gabriel & Holanda, 2019). As demonstrated by Urisa (2012), personnel competency is key in an enterprise GIS and it is termed as a primary assets and factors that determines the success of GIS application. In this case, parameters associated with GIS competency forms elements of evaluating GIS competency within a specialized process that safeguards quality of outputs in GIS work functions (DiBiase et al., 2010).
A GIS software is designed to provide unlimited analytical capabilities for various GIS applications (Mierzejowska & Zogała, 2018). Associated software parameters provide elements of evaluating software tools for effective GIS application analytical capability (U.S. Department of Homeland Security, 2013). Due to existence of both open source and commercial software, evaluation is crucial for proper decision on the best software in line with organizational goals (Giannelli, Sánchez, & Agugiaro, 2022).
GIS operations and procedures should be well observed in order to maintain a GIS which is healthy and sustainable (Scott & Rajabifard, 2017). The associating parameters provide crucial audit elements for a sustainable GIS.
3.3. GIS Audit Parameters
The parameters are presented in Table 1. They constitute categories and associating parameters that that determines an established GIS fitness for use. The parameter identification results are discussed as follows:
1) Data Quality Parameters presents basic elements that define quality components in a GIS datasets. The parameters describe a functioning GIS which is based on relevant data in terms of quality (Mierzejowska & Zogała, 2018). The parameters take into consideration evaluation of data quality with respect to international standard ISO 19157 “Geographic Information Data Quality” as pointed out by (Kin & Lazorenko-Hevel, 2021). Monitoring and evaluation using these parameters safeguards the quality of an established GIS system and builds confidence in data integrity (Yvan, Eric, & Sonia, 2015). It ensures that decisions are made based on complete knowledge of capabilities and limitations of each dataset depending on the context in which GIS applications are done.
2) GIS Software Parameters takes into consideration evaluation and monitoring of a GIS software to improve operations. The parameters are intended to unveil approaches that make it easier to extend software capabilities to the reach of GIS across the organization (Esri, 2021). Audit priority is given to evaluation of software functionality and percentage utilization of a GIS software. This is intended to provide knowledge on percentage at which GIS software tools are utilized by GIS users. It also provides insight on various capabilities as supported by a GIS software, hence guide in identification of appropriate functionalities of the developed GIS system. This forms appropriate software tools that support activities of a functional and efficient system in line with organizational goals (Mierzejowska & Zogała, 2018).
3) Personnel Competency Parameters outlines critical GIS performance subjects that guides in assignment of appropriate staff who meets system needs and desired output quality (Urisa, 2012). Personnel competency audit parameters focus on a GIS system that results in a successful performance during each implementation process. The parameters provide an avenue in which existing GIS workforce is evaluated to reveal their capability to apply their skills and knowledge to perform required tasks or functions within an established GIS.
4) Procedures provide quality management that fulfills quality requirements of the established GIS. They provide a framework for GIS system quality management with respect to:
a) Performance of a GIS system at regular intervals
b) Monitoring and evaluating GIS resources and components
c) Organizations need to keep trends on the upcoming technological changes especially on geospatial (big) data and GIS software
d) Improved framework for data update frequency
e) Overall established GIS sustainability
This is meant to overcome challenges faced during operations and maintenance.
Considering the broad nature of GIS, the identified parameters may not be exhaustive. However, the identification was done based on what is most important for a functional GIS system audit. Table 2 shows the number of associating parameters for each category. The parameters are reasonably adequate in representing each category for an informed GIS audit. The parameters are almost constant for other categories except for software utilization. While the
main aim of software audit in this study is to determine its percentage utilization, the other three parameters focus on full compliance of technical quality and processes within a GIS system. This explains why their associated parameters are more as compared to software utilization audit.
4. Conclusion
The main objective of this paper was to identify and provide a conceptual framework for evaluating an established GIS in an organizational setting. Four main categories of GIS audit parameters have been identified and form basis for coherent GIS audit in the context of GIS components vital to its development. In each category, minor parameters have been identified that constitute aspects of monitoring and evaluating an effective, efficient and sustainable GIS system. The identified parameters may not be exhaustive and future additions can be done where needed, especially due to technological dynamics of GIS.
Although some GIS audit frameworks and tools such as Functional Requirement Study/Studies (FRS) and GIS Capability Maturity Models (CMM) exist, gaps have been identified that warrant a different method of GIS auditing. With these issues in mind, the identified parameters have provided a foundation in which a new GIS audit framework is developed.
During parameter identification, various challenges that would be encountered while attempting to audit GIS within an organization, for example large data holding, scalable nature of an enterprise GIS and GIS technological trends among others have been brought to attention As a result, this study has identified possible solutions to these challenges as provided here after.
1) Identified audit parameters are simple and clear for both GIS expert and non-expert to understand. They are derived from basic GIS components and provide best GIS establishment requirements and underlying factors.
2) Audit process will be designed with flexibility to accommodate additional audit needs that may arise due to GIS technological changes.
3) Audit scope is limited to internal GIS processes and procedures, covering full portfolio of a GIS system.
Next step will include the design of the framework. In the proposed framework design, each category of audit parameters will be uniquely described based on the identified parameters. This will be done by considering key features and contribution of each parameter to the GIS system applications. The framework design will be done based on these parameters and listing of main questions to be answered. The parameters will be visually organized into simple tables whose contents will be well designed for the auditor to quickly understand the degrees of audit necessity.