AHP and CA Based Evaluation of Website Information Service Quality : An Empirical Study on High-Tech Industry Information Center Web Portals

Web portal is a useful way for high-tech information center providing information services for high-tech enterprises. The importance of evaluating the information service quality of the web portal has been recognized by both information service researchers and practitioners. By applying for an analytic hierarchy process (AHP) approach, a hierarchical structure evaluation criteria model has been constructed, and the global priority weight of each criterion in the model has also been determined in this study. According to the model, two different methods—comprehensive score method and correspondence analysis (CA) have been used to analyze and evaluate the information service quality of ten primary information center web portals in China high-tech industry from different aspects. The findings of this study provide a useful instrument for researchers who wish to measure the information service quality of web portals, as well as for high-tech information center managers who want to enhance their web portals service performance.


Introduction
High-tech industry is a knowledge and technology intensive industry, which needs high quality and diversified information services.In China, information services for high-tech industry are usually provided by special information centers, such as China National Chemical Information Center, State Food and Drug Administration Information Center etc.All of them mainly provide information services through their web portals.A high-tech industry information center web portal (HIIC web portal) is a website that provides high-tech enterprises with online information and information related services for their decision-making, such as information retrieval, information publication, business consulting, and it is also a good channel for enterprises to communicate with their business partners.
With the rapid development of high-tech industry in China, the information quality requirements of high-tech industry have become higher and higher.As a result, analyzing and evaluating the web portals information service quality and then improving it become a very important task for information centers.In recent years, many evaluations of website information service quality have been concerned, and relevant researches [1][2][3] have also been carried out around this issue.According to the findings of these researches, different scholars have identified different evaluation criteria and proposed some evaluation methods and models, such as website flux index statistical method [2], expert assessment, questionnaire investigation [1], and the comprehensive evaluation method [3].But most of these studies only introduce the evaluation model constructing process or application fields without getting specific evaluation results for the websites.In addition, as to the types of evaluation websites, the majority were concentrated on the e-commerce websites [4] and library websites [5], with little focus on the information center web portals.Therefore, there is an urgent need to provide an effective method to evaluate HIIC web portals information service quality and help the owners improve their web portals' quality.
This study firstly identifies the information service quality evaluation criteria and sub-criteria of HIIC web portals according to their characteristics.Then by applying an analytic hierarchy process (AHP) approach [6], the study constructs a hierarchical structure evaluation criteria model and determines the global priority weight of each criterion.Secondly, according to the evaluation criteria model, two different methods-comprehensive score method and correspondence analysis (CA) have been used to analyze and evaluate the information service quality of 10 samples from different aspects.To be specific, the comprehensive score method is used to compare the information service quality by calculating the final scores of that 10 different HIIC web portals, while CA is used to clearly reflect their advantages and disadvantages in each factor.Finally, according to the results of evaluation, this study proposes the suggestions for high-tech industry information center owners to improve their web portals quality and provide better information services for users.

Research on Information Service Quality Evaluation
In many studies, information service quality is generally regarded as an aspect of service quality; few researches specially focus on this issue.Levjakangas [7] has defined information service in his study as "a useful set of refined data provided to the user of information that supports the user's decision making as planning and execution of efficient operations."This definition included the right content, right timing, right formatting and right channeling of information, which can be used as indicators for information service quality evaluation.S.Feindt [8] thought that in the e-business context, website information service quality insinuated delivering relevant, updated, and easy-to-understand information to significantly influence online users' attitude, satisfaction, and purchases.Delone and McLean [9] have established a well-known model to measure information service quality; they highlighted the importance of relevance, timeliness, and accuracy of information.In addition to these researches, the information service quality evaluation is usually studied within service evaluation like some other related issues, such as website design quality, website security quality.For example, the most well-known service quality instrument, SERVQUAL, was used to measure customer's expectation and perception of service quality [10,11], and has been adopted successfully in the information system field to measure information system service quality.SERVQUAL consists of reliability, responsiveness, empathy, assurance, and tangibility, among which, reliability, responsiveness, and empathy are con-sidered as the factors relating to information service quality evaluation.The .comQ (dotcom service quality) scale established by Wolfinbarger and Gilly [12] also considers information service quality which includes four major factors: website design, information reliability, privacy/security, and customer service.In order to measure e-business success and develop a high service quality website, Lee and Kozar [4] proposed a comprehensive research model from information quality and service quality two aspects respectively.

Research on Web Portal Information Service Quality Evaluation
With the rapid development of internet, evaluation of website information service has become a key issue, more and more organizations and researchers have developed different methods for various websites information service quality evaluations.Gomez.comdeveloped different scorecard indices to evaluate websites information service quality in different industries such as banking, mortgage, insurance, and retail [13].Their evaluation of a website combines results from both consumer surveys and expert judgments based on factors such as usage, web resources, information reliability, response speed and personalized service.BisRate.comestablished an index system for the information service evaluation of ecommerce website based on dimensions such as product representation, product information, on-time delivery, and customer support [14].Web portal information service is more diversified than general website information service-a web portal presents information from diverse sources in a unified way.So it is more complicated and difficult to evaluate their qualities.to extant literature by establishing a basis for further theoretical advances on information service quality of a web portal.
In sum, prior researches from different perspectives developed some fundamental knowledge about information service quality evaluation, but most of them are for general websites and not for web portals, especially not for HIIC web portals.HIIC web portals help high-tech information centers to provide a consistent look and feel with access control and procedures for multiple applications, which otherwise would have been different entities altogether.Compared with general websites, HIIC web portals are more authoritative and information-oriented. Therefore, the general information service quality evaluation index system or criteria model may not be suitable for HIIC web portals.It needs to identify related evaluation criteria (or factors) according to the characteristics of the HIIC web portals.Each of the criteria should have a significant impact on overall information service quality of a web portal.So as to be used to formulate a model to dealing with the problem of HIIC web portal information service quality evaluation.

Identifying the Criteria and Sub-Criteria
There is currently no established evaluation criteria model for measuring the information service quality of HIIC web portals.Thus, we integrated several conceptual methods to identity important service quality dimensions related to HIIC web portals for our study.Before we identify the criteria we should determine that a user must have a reason to adopt the website as an information and communication channel.The well-known technology adoption model (TAM) is thus embraced.Then, an HIIC web portal essentially is also an Information System, consisting of digital information and an information delivery infrastructure (browsers, search engines, encryption, networking systems, etc.).Accordingly, information quality and system quality are of importance for HIIC web portal users.Based on these foundations, by analyzing previous models of general websites and considering our research feasibility, we make a reference to the model proposed by Yang et al. [2] to identity information service quality criteria related to the HIIC web portals for our study.Considering the particularity of the HIIC web portals, two criteria and wording of some items have been changed.Finally, this study proposes a modified evaluation model consists of five criteria to measuring the information service quality of HIIC web portals, including usefulness of content, adequacy of information, specialization, easy of use and interaction.Each of the five criteria had a significant impact on overall information service quality.Figure 1 lists the foundations and their relationships to the proposed information service quality criteria.

Usefulness of Content
The content of a HIIC web portal is of great importance since it directly influences the user's perception of the destination [16].Usefulness of content examines whether a HIIC web portal can provide reliable, up-to-date, relevant and accurate information.Specifically, content reliable refers to its dependability and consistency.Content up-to-date is concerned with information timeliness and continuous update.Content relevance includes relevant depth and scope, and completeness of the information [4].Content accuracy describes the degree to which the web portal information is free of error [2].

Adequacy of Information
High-tech enterprises usually need HIIC web portals to provide information as complete as possible.Adequacy refers to the extent of completeness of information.It can be measured using complete product/service description, complete content, detailed contact information and rich variety of information.Complete product/service description facilitates users understanding of the products and personalized services.Detailed contact information refers to whether a HIIC web portal contains such information as contact address, telephone numbers, email and relevant high-tech enterprises Yellow Pages.In addition, web portals should provide complete and a great variety of information to satisfy the demands of different department users [15,17,18].

Specialization
Compared with other industries, high-tech industry has its own features.For example, it is knowledge-intensive, market-oriented and highly innovative.It needs special information and services in its development process.Specialization is a critical factor, which can make a HIIC web portal distinguish itself from general information center websites.It can be measured by examining whether a HIIC web portal has special databases, industry research reports and supply-demand information.Special databases can provide high-tech enterprises with rich products information, technical standards and front knowledge.Industry research reports can help enterprises better understand the development status and trends of the industry.In addition, a main purpose of the high-tech enterprises is to put their products into market quickly, so they also need lots of supply-demand information.

Easy of Use
In the websites context, easy of use has been regarded as the most frequently used factor in measuring information quality or user satisfaction [19,20].It involves five aspects: search function, navigation, personalization, hyperlinks structure and speed of page loading.Search function refers to the website's capability to provide diversity retrieval methods, and helps users to quickly find and select the products or services they need [21].Good navigation allows users to stay oriented during their visits and easily locate the information or products they need.Personalization services can provide online users an individualized interface, effective one-to-one service and customized information.Effective hyperlinks structure and high speed of page loading of a web portal can enhance the user experience.

Interaction
Interaction is a kind of action that occurs as two or more objects have an effect upon one another [22].The inter-action about a HIIC web portal can be divided into three categories: enterprise-information centre interaction, enterprise-enterprise interaction and enterprise-website interaction.Although using a HIIC web portal is primarily a self-served process, high-tech enterprise staff may still expect websites quick response to them and provide professional follow-up services by the knowledgeable and caring contact persons working in information centers.HIIC web portal is a platform for information communication between users and service providers, therefore, it should provide contact channels for business-tobusiness/information centre such as email, message boards, char room and discussion forum.

Model Construction
In order to enable persons engage in the evaluation work to visualize the problem systematically in terms of relevant criteria and sub-criteria, we adopt the AHP to construct an evaluation criteria model.The AHP, developed by Saaty (1980) is designed to solve complex multi-criteria decision problems [23].The hierarchical structure used in formulating the AHP model can show the relationship between different factors more clearly.Furthermore, AHP has inherent capability to handle qualitative and quantitative criteria used in information service quality evaluation problems.The criteria and sub-criteria we have identified have both qualitative components such as usefulness of content and quantitative ones such as number of special database.Therefore, AHP is the best method for evaluation criteria model construction in this study.
The common AHP process involves three phases: construction of a hierarchical structure of the AHP model to present the problem, perform pair wise comparison of the criteria at the same level and determine their weights and synthesis to obtain the global weights for the criteria [24].The consistency of the results is measured using a consistency ratio (CR).A CR of less than 0.1 is considered adequate to interpret the results [25].Using this three-phase approach, this study first formulates a threelevel hierarchy criteria model for HIIC web portal evaluation, as show in Figure 2. The goal of our problem is to evaluate the information service quality of HIIC web portals, which is placed on the first level of the hierarchy.Five factors described in the above section are identified to achieve the goal, which form the second level of the hierarchy.The third level occupies the criteria defining the five factors of the second level.
In addition, using the AHP, we can also determine the weights of each criterion and sub-criterion, which will be  Based on the pair wise comparison judgment matrices obtained from five experts, we collected the valid data, calculated the geometric mean and checked the integrated CR.The software Matlab is used to determine the normalized priority weights for each criterion.The results of CRs are all less than 0.1, so the matrices satisfy the consistency condition.Based on these results, the global weight of each sub-criterion of level three has also been calculating by Matlab, the results are show in Table 1.

Method
The questionnaire consisting of 19 items, as showed in Appendix B, was used in the study.All the items were in one-to-one correspondence with the sub-criteria in evaluation model and measured by using a 5-point Likert scales ranging from low to high.Then the questionnaires were sent to evaluate work participants by emails.

Evaluation Samples
According to the Organization of Economic Cooperation and Development (OECD)'s industry classification standard, high-tech industry has been divided into 4 major categories which including aerospace manufacturing, computer and office equipment manufacturing, electronics and telecommunications equipment manufacturing and pharmaceutical manufacturing [26].At present, China is basically in accordance with the OECD's high-tech classification standards to define the statistic scope of high-tech industry.But considering the National Economic Industry Classification Standard (GB /T4745-2002) of China, another four categories have been added to the OECD's categories in our country, they are nuclear fuel processing, information chemicals manufacturing, public software services and medical equipment and instrument manufacturing.In order to accurately reflect the level and current situation of in

Participant
Thirty CIOs from different high-tech enterprises agreed to our cooperation request and then participated in this study.The enterprises were selected from "China Hightech Enterprise Directory" and all of them have the specialized information technology department.Generally speaking, CIO is the person in an enterprise who is responsible for the information technology and computer systems; so he could understand the enterprise's information services demands better than general employees.75% of the participants were male and 25% were female.All participants were experienced HIIC web portal users: 40% of them had visited more than 8 HIIC web portals and 80% had visited more than 5. Regarding their usage frequency, over 80% of them used services provided by HIIC web portals for supporting enterprise goals several times a day while the others several times a week.
Every CIO must fill in 10 questionnaires because they should score all the 10 samples respectively.Before filling in the questionnaires, all the participants were asked to visit the samples and access the information service experiences.Every one should score all items of each HIIC web portal according to their experiences.In order to ensure the effectiveness of response results, follow-up calls were made to these participants.In the end, all the questionnaires responded with complete data.

Data Analysis Methods
The data collected from the 300 (30×10) questionnaires were coded and analyzed statistically, and then we calculated the average score of each sample in each item.In order to analyze and compare the information service quality of the samples, two data analysis methodscomprehensive score method and correspondence analysis (CA) were used in this study.They evaluated the information service quality of HIIC web portals from different aspects respectively.

Comprehensive Score Method
In this study, the evaluation samples (alternatives) have not been placed in a separate level like usual AHP approach, an improved method-Comprehensive Score Method was instead of assessing pair wise comparisons among the evaluation objects.In Subsection 4.2 we have already asked thirty CIOs of different enterprises scored the each criteria of each HIIC web portal sample with their experiences by using 5-point Likert scales, and then we calculated the average score of each HIIC web portal sample in each criterion.The Comprehensive Score Method is used to calculate the final score of each sample by multiplying the global weights of criteria in the third level with the 5-point Likert scales average scores of the corresponding criteria obtained from questionnaires.So, assume the average score of each sample in each item (sub-criterion) is ( , , the final score S i (i=1,2,…,10) of each sample can be calculated by comprehensive score method as shown in following equation.is the global priority weight of each sub-criterion.Then we can compare the information service qualities of 10 HIIC web portals by their final scores.It should be noted in this study that we didn't calculate the weights of evaluation samples as traditional AHP method which usually placed the evaluation samples (in traditional AHP me-thod it is called alternatives) in the lowest level in an AHP hierarchical structure and then got their global weights by a pair wise comparison judgment matrix.0.577,0.0428,, 0.0452,0.0417( , , , , ) ( ) This study only calculated the global priority weights of sub-criteria, and then used them to get final scores of 10 samples.The reason is that if we adopt the traditional AHP method to compare 10 samples (alternatives), the number of pair wise comparisons required for each of the 19 sub-criteria would be equal to n(n-1)/2=45, and it becomes computationally difficult and sometimes infeasible.The major advantage of comprehensive score method is to overcome the explosion in the number of required comparisons when the number of samples is large, and shorten the evaluation time.

Correspondence Analysis
In order to further explore the relationship between the 10 HIIC web portals in different criteria and then classify them according to their similarities, this study adopts the correspondence analysis to deal with the collected data.Correspondence analysis is a powerful method for the multivariate exploration of large-scale data, which reveals a correlation between variables through analyzing interactive tables constituting qualitative variables, thus presenting differences between different categories of the same variable, and those between different categories of different variables.The primary idea of correspondence analysis is to present the proportion structure of elements in the columns and rows of a contingency table in the form of points in a space with lower dimensionality [5].Mathematically, correspondence analysis can be regarded as either: 1) A method for decomposing the chi-squared statistic for a contingency table into components corresponding to different dimensions of the heterogeneity between its rows and columns, or 2) A method for simultaneously assigning a scale to rows and a separate scale to columns so as to maximize the correlation between the resulting pair of variables.
Quintessentially, however, correspondence analysis is a technique for displaying multivariate categorical data graphically, by providing coordinates to represent the categories of the variables involved; these may then be plotted to provide a "picture" of the data.Therefore, we can present the evaluation samples and criteria in a single figure with CA, directly and clearly observe categories and attributes of the samples.

Comprehensive Score Method Result
According to the equation explained above, we calculated Compared to the total score of each HIIC web portal as shown in Figure 3, we find that W8 has the highest score 3.8296, which means its integrated information service quality is the best, especially at adequacy of information, specialization and easy of use.This web portal has a clear navigation system.When users visit it, they can easily find what they need and got special on-line services and consultation.W2 and W5 respectively got the second and third highest score, which are 3.8254 and 3.6956, so their service qualities are satisfactory.Comparatively speaking, W7, W1 and W10 are the web portals whose comprehensive scores are lower than other ones.They have some common problems such as slow information update, less information content, imperfect search function, and bad interaction.So they should make great efforts to improve their information services so as to better satisfy the needs of high-tech enterprises.
In addition, other HIIC web portals are in the general level of information service quality.
Figure 4 has further shown the comparison results of information service quality from five factors: usefulness of content, adequacy of information, specialization, easy of use and interaction.On one hand, we find that W4 has the highest usefulness of content, because it can usually provide latest official policy about aerospace manufacturer and update information content everyday.On the other hand, W2 has the highest adequacy of information and specialization; it has a quotations center which can help users understand chemical products price trend and market development in time.In addition, it has detailed industry statistics and special reports which provide users with abandon useful information.W8 has the highest easy of use, users can search information what they need by keywords or full text.The navigation of W8 is also very good, which divides the website content into fourteen sections and the most important information are put on the striking location, so users can find it quickly.W10 has the highest interaction which provides online consultation services and responses users' needs quickly, and it also provides a chart room for business negotiations between different enterprises.But, we also find that W10 has the lowest usefulness of content and adequacy of information, W7 has the lowest easy of use and interaction, while W1 has the lowest easy of use.Based on these comparison results we can clearly find out the advantages and disadvantages of each HIIC web portal and then propose feasible suggestions to improve its information service qualities

CA Result
In order to further analyze the advantages and disadvan-tages of these HIIC web portals in which criterion, the average score of each HIIC web portal sample in each criterion was entered into the SPSS-the data analysis tool, and analyzed.The results of correspondence analysis appear in Figure 5.The association graph was analyzed further, and the points of criteria and HIIC web portals on Figure 5 were classified into four group.
The first grouping includes W1 and W4.The common ground of these two HIIC web portals are reliable information (C 11 ), rapid information updating (C 12 ), complete product/service description (C 21 ), complete content (C 22 ), detailed contact information (C 23 ) and good personalized services (C 43 ).They get high scores in these criteria.For example, W1 updates news of Nuclear Fuel Processing industry everyday and provides detailed contact information of these kinds of companies.But it is also presented in Figure 5 that search function (C 41 ) and supply-demand information (C 33 ) of the two web portals are not very good, there is a long distance between them.Therefore, much attention must be paid to the search function and supply-demand information in the future to improve their information service qualities.
The second grouping includes W2, W3, W5 and W8.The advantages of these web portals lay in the relevant information (C 13 ), rich variety of information (C 24 ) and industry research reports (C 32 ).However, among these four web portals, W2 is not near the other three points and far from many criteria.It is seen from average scores made by participants that W2 got similar high scores in these criteria, which indicates that W2 performs better in these aspects than other web portals.
The third grouping includes W6 and W7.Their advantages are accurate information (C 14 ) and high speed of page loading (C 45 ).But they are not very successful, with little contact information and specialized information such as special databases, industry research reports, etc, revealing that the information service qualities are at a lower level.Therefore, such web portals should make great efforts to improve their information abundance and specialization.
The fourth grouping includes W9 and W10.They can provide users with rich supply-demand information (C 33 ) and good interactive services such as quick responsiveness (C 51 ) and follow-up services to enterprises (C 52 ), contact channels for business-to-business/information centre (C 53 ), etc.They also have good navigation (C 42 ) and effective hyperlinks structure (C 44 ).This indicates that overall performance of such web portals is in normal order.Therefore, such web portals should find their proper positions in the high-tech industry according to their own situation and advantages.
There are still another two criteria points-special databases (C 31 ) and strong search functions (C 41 ) don't be-long to any of the above groupings, which shows that compared with other criteria, they are not the unique attributes for a particular web portal, but they make great influence for the information service qualities of some HIIC web portals such W1, W4, W6, W7, W9 and W10.These web portals don't get high scores in the two criteria, so they should attach much importance to these two aspects.

Discussion and Conclusions
Information service quality evaluation of a HIIC web portal is an important problem to information centre itself and high-tech enterprises.By improving the model proposed by Zhilin Yang et al., this study identified five information service quality criteria including usefulness of content, adequacy of information, specialization, easy of use and interaction, which included 19 sub-criteria.Then by using an AHP approach, this study constructed a hierarchical structure evaluation criteria model and determined the global priority weight of each criterion and sub-criterion.Based on this evaluation criteria modal, the study comprehensively applied the comprehensive score method and CA to analyze the information service quality of ten typical information center web portals in high-tech industry and got several useful results.From the research results we can find: First, using the AHP approach proposed in this study, the criteria for information service quality evaluation were clearly defined and the problem was structured systematically.This enabled executives of HIIC to examine the strengths and weaknesses of their web portals by comparing them with respect to appropriate criteria.The weights of five criteria showed that specialization and easy of use carry more weight on the overall evaluation of a HIIC web portal than other factors.Thus, executives should expend more effort to make their web portals more professional and easy to use.
Second, the study found that the comprehensive score of each HIIC web portal calculated by equation proposed in this paper all are not very high (less than 4), especially at two aspects-easy of use and interaction.It indicated that the overall information service qualities of these web portals were at a lower level and the main shortages are at easy of use and interaction.To a high-tech enterprise, it was important for it to be able to locate the needed information without difficulties.So the information center should design a user-friendly website, with strong search functions, good navigation, personalized services, effective hyperlinks structure and high speed of page loading.At the same time, the information center should strengthen the interaction between web portal and users by using some facilities such as user chat room, message board, reputation system and providing online consulta- Finally, according to the CA results, the 10 samples have been divided into four groupings by their similarities in different criteria, which can help such web portals to position themselves properly and provider extra specially services for enterprise users.It can also urge the web portals to compare themselves with those that have the best performance in different criteria, to enable them to make improvements on information services.
At the same time, this study still has several limitations that should be revisited in the future studies.First, the study only chose CIOs in the high-tech enterprises as the participants, which could not represent various kinds of users' views.Second, this study was conducted with relatively small samples, 10 official HIIC web portals might not give a comprehensive reflection of the overall situation of HIIC web portals in China.Finally, evaluation criteria were selected based on the previous model of general websites, which could have excluded some criteria that might strongly influence website information service quality.Future research needs to take into consideration more kinds of users such as managers, staff in the high-tech enterprises participate in the evaluation process.Furthermore, it also should increase the number of website samples and collect data to examine the validity of the proposed model better.

Figure 1 .
Figure 1.Research foundations and proposed information service quality criteria of HIIC web portals

Figure 2 .
Figure 2. The evaluation criteria model Table1.Global weights of the sub-criteria Criteria Local weights Sub-criteria Local weights Global weights

Figure 5 .
Figure 5. Positioning maps tion and training services.
This work was supported by grants from the China National Social Science Foundation Major Project "Study of Information Services in China National Innovation System" (No.06&ZD031) and Key Project of Philosophy and Social Sciences Research "Education of Ministry of Knowledge Information Service System of Innovation-Oriented Country" ( No. 06JZD0032).

B
to calculate the total scores of each HIIC web portal for their information service quality evaluation in our following study.Based on the AHP hierarchy model, 5 experts were invited to assign pair wise comparisons to the criteria and sub-criteria in each level of the hierarchy by scoring the comparison matrices.The nine-point scale as suggested by Satty is used by them to assign their pair wise comparisons of all elements.Of these five experts, one is senior engineer from the website design department, two are information consultants of famous hightech enterprises and the other two are professors of information management.All of these experts have sufficient experience in web information service quality evaluation and are qualified to assign pair wise comparison judgment matrices for the proposed AHP model, as shown in Appendix A. used