Research on the Evaluation of Urban Open Data

Urban open data is the key to the construction of smart city. Through the research on evaluation of urban open data, the concept, types, characteristics and other basic problems of urban open data are systematically summarized. From perspective of “quality”, “opening” and “acquisition”, a complete urban open data evaluation framework and index system is built. And the corresponding weights of evaluation indexes and score and overall rating methods are determined, so as to objectively evaluate the conditions of urban open data, and describe, monitor, guide and promote the construction and development of urban open data.


Introduction
In many areas and links of the construction of smart cities, urban open data has been considered as a prerequisite and basic work.The four main features of a "smart city"-comprehensive and thorough perception, ubiquitous interconnection of broadband, application of intelligent convergence and human-based sustainable innovation"-are based on open urban data.It can be said that urban open data is the key to "smart city".Through urban open data, we can maximize the use of data by the whole society, and realize added value of data; more importantly, we can establish an ecological system operating around data, so that individuals, organizations, enterprises and government in the society are not only data producers, but also data analyzers, explorers and application users, so as to inspire infinite creativity and make the city really smart.Urban open data is a complex problem involving multiple disciplines.This In terms of open access (OA), J. Beall established a number of criteria to judge the poor journals; Jiang Jing pointed out the comprehensive evaluation index system of open access journals, including academic information content, inclusion, quality of information released, copyright policy, academic influence, etc.
[1]; Gu Liping, et al. discussed about the evaluation and the selection of open access journals from the perspective of quality level, and the degree of opening service ability [2]; and Chen Ming constructed the evaluation model of open access journals, including 16 evaluation indexes such as the number of articles published, time delay of article publishing, the total citation frequency, impact factor, journal h index and visits [3].
In terms of open repository resources, through project planning, satisfaction of academic goals of institution, the allocation and usage patterns of funds, the relation with related digital project, platform interoperability, measurement of document use and other indexes, M. Westell built the evaluation model of repository resources of institution [4]; Y. H. Kim, et al. developed and strengthened the evaluation system of open repository resources from three aspects, including system index, content index and management index [5]; and Sun Tan, et al. evaluated [6] and studied repository resources of foreign institutions from the perspective of system construction, content organization, service management, etc.
In terms of open government data, T. Davies evaluated the open data portal of government.The main standards included allowing users to directly find their desired fact, data visualization, supporting more efficient work, supporting innovation and reuse, etc.; T. Berners-Lee established a five-star evaluation system of linked open data, and each star corresponded to the specific evaluation content [7]; and Open Data Institute pointed out that excellent open data could be provided in correlated and structured format, and had guaranteed availability, consistency and traceability, etc.
From the existing literatures, we can see that researches

Concept
In broad sense, urban data contains all the data related to a city.Due to the complexity of a city, the urban data under the concept is too broad.This paper defines urban data as a series of data directly produced by urban activities or which can directly affect urban activities; and not all urban data is open city.
Therefore, urban open data can be defined as data suitable for being opened directly produced by urban activities or which can directly affect urban activities.
In short, it can be understood as the intersection of urban data and open data (Figure 1).

Classification
The  Data which is produced by urban economic activities or can reflect economic law and situation, is usually calculated with formula and abstracted.It is general figures, such as CPI, GDP, GNP and financial prices.It also contains information of some enterprises which can influence economy.

Social activities
A large number of social dynamic data is contained, such as household registration, social behavior data and so on.Macro data and micro data can both be involved.The deep law of urban social behavior should be explored from them.

Public facilities
A large number of related data of public service facilities is contained, such as entertainment, science and education, sports, medical treatment, social security, etc.They are practical and have flavor of life.

Landscape environment
Various types of data which affect the quality of urban living environment, including data of landscape beautifying life, such as parks and green belt, and environmental data related to environmental safety, such as pollutants, air quality, and resource and energy consumption.
Tourism development These data promotes urban economic and construction, but they often start from the perspective of social services, and provide the public with service resources of tourist attractions and some tourism-related projects.

Road traffic
Related data resources of urban road traffic planning and construction, transportation, traffic management and others.Part of them has the characteristics of social services, but because they have a direct impact on the layout and shape of city, they are not distinguished between deliberately.

Municipal facilities
These data includes planning, construction and management of municipal facilities except roads and squares, such as hydropower, gas and communication.They are professional.

Land use
Data related to the use of cities and surrounding land, such as land transfer, land parcel trading and geological survey, which is featured by strong professional attributes.
Planning management Data resources involving planning, design, construction, management and other aspects of the whole city, areas, sites, building monomer, etc., such as park planning, site selection of construction project, etc.They have strong characteristics of entity space carrier and obvious professional attributes and performance.data is clearly goal-oriented.Data is often recorded according to certain rules [9], so the data has strong practicality and stability, and a high value density.

Framework of the Evaluation System
The framework of evaluation is foundation and support for the establishment of  The quality of urban open data is mainly embodied in three aspects, including authenticity, integrity and spatiality.Among them, it is difficult to verify authenticity in the actual operation process of evaluation, so it is not directly considered as an evaluation index, and it can be replaced from an authoritative point of view.To sum up, Level 2 Evaluation Indexes of "quality" can be decomposed into integrity, space and authority.The content evaluated by authoritative index mainly is mainly whether sources of data are authoritative and reliable, and have evidences.On the one hand, data provided by an authoritative and professional organization or department usually can guarantee a certain degree of quality; on the other hand, if the source or link of data is provided, the data is well documented, and it is also a guarantee of the quality of data.Seen from representation of authority, data provider is the primary basis.Metadata that can be acquired (i.e., data that describes data) is also included.Therefore, Level 3 Evaluation Indexes corresponding to authoritative index are set as data provider and metadata.

b) Decomposition of "public" evaluation index
The openness of urban open data can be mainly embodied in three aspects, including security, timeliness and relevance.Among them, timeliness is essentially the control of the timeliness of data.Therefore, "open" Level 2 Evaluation Indexes are decomposed into security, timeliness and relevance.
Security has always been one of the unavoidable problems of urban open data.
It mainly can be judged by the sensitivity of information provided by the data, and whether the data itself contains risk factors.For processing of sensitive data, there are many technical means, which can effectively protect sensitive information from being disclosed.Whether the data itself is dangerous can be evaluated by whether the download link of test data contains virus or phishing links and other means.To sum up, Level 3 Evaluation Indexes corresponding to security are set as sensitive information and dangerous link.
The timeliness index mainly focuses on the validity of data at the time level.
The timeliness of data can be judged from three aspects.The acquisition of urban open data mainly includes machine reading, convenience and degree of interaction.Therefore, Level 2 Evaluation Indexes of "acquisition" are decomposed into machine reading, convenience and interactivity.
The machine reading index mainly includes two aspects.First, data format is correct and complete, and has no missing or damage.It will not hinder or have adverse effects on data reading and reuse; second, the diversity of data format, which means that the data format provided shall ensure the common general format, and provide different kinds of data format as many as possible, in order to meet different needs of users.Therefore, Level 3 Evaluation Indexes corresponding to machine reading are set as format integrity and format diversity.
The convenience index mainly considers the efficiency level in the first phase of data acquisition behavior, and it is the service function performance based on the user experience.On the one hand, the platform carrying data is one of the most important factors, and high-quality interface design allows users to quickly and accurately find the required data resources; on the other hand, restrictions on user identity may also affect the convenience to some extent, such as the need for registration, filling out the questionnaire, etc.In summary, Level 3 Indexes corresponding to the convenience index can be set as user restriction and interface design.
The interaction index mainly refers to feedback and evaluation of users' data utilization.
For any urban open data, the feedback given by user after using the data is important data evaluation basis.The common form of feedback includes comment system and scoring (star) system; in addition, by providing APP software and other means, interest and enthusiasm for user feedback, upload and reuse results or other related data resources can be stimulated and improved.To sum up, Level 3 Evaluation Indexes corresponding to the interaction index can be set as comment system, scoring system and data application.
The summary of evaluation indexes of urban open data at different levels are shown in Table 2.

Weight of Evaluation Indexes
Determining the specific weight value of each evaluation index is an important  method, 30 experts of city planning and other professional fields are consulted, and the judgment matrix of any two indexes at each level of the evaluation system of urban open data is obtained.With the help of matlab software, the normalization processing and consistency test of the judgment matrix are completed, and finally the weight value of each evaluation index and the overall ranking result of different levels are obtained (Table 3).

Scoring and General Comment of Evaluation Indexes
After the text edit has been completed, the paper is ready for the template.Duplicate the template file by using the Save As command, and use the naming convention prescribed by your journal for the name of your paper.In this newly created file, highlight all of the contents and import your prepared text file.You are now ready to style your paper.

Scoring of Evaluation Indexes
Scoring of evaluation indexes refers to scoring of the final level of evaluation indexes according to certain standards, so as to derive the score of the upper level of indexes, and complete the overall scoring of urban open data.In the evaluation system in this paper, all three levels of evaluation indexes will be scored.
The scoring process follows the 5-level scoring system.Each level has corresponding judgment criteria and scores.The corresponding score will be assigned to Level 2 Index that meets a certain level of judgment criteria.Specific scoring settings are shown in Table 4.

Overall Comment
paper will systematically summarize the concept, types and features of urban open data and other basic problems through research on evaluation of urban open data, and put forward evaluation factors and the evaluation system of urban open data, so as to have a comprehensive and clear understanding of urban open data.From the practical point of view, quality and levels of urban open data at home and abroad are uneven.The research on evaluation of urban open data can standardize and deepen the content of urban open data, and comprehensively and objectively evaluate the quality and value of urban open data, so as to effectively describe, monitor, guide and promote the urban open data work.In existing literatures, there are few researches on the evaluation of urban open data.We can mainly borrow from researches on open access (OA), open repository resources and open government data.
Urban open data inherits the common features of open data, and it also has its own features.As open data, it has originality, readability, interactivity and relevance; as urban data, it has the following features.It is characterized by time and space.Different from other types of data, urban data often has space and time attributes.The data are based on composition and distribution of urban space.At the same time, changes of urban activities in different periods of time give data more meanings.It is comprehensive.City is a complex comprehensive system.The factors cross and superpose in the city, and form different attribute characteristic of the city.City, a comprehensive system, can be divided into several small systems, such as ecosystem, hydrogeology, pipe network system, power system and so on.Therefore, urban data is bound to be complex and diversified.Urban open data is structured.Acquisition and management of urban open Y. Liu et al.DOI: 10.4236/wjet.2017.53B014126 World Journal of Engineering and Technology the evaluation system.The main idea of the construction of the evaluation framework of urban open data is based on the analytic hierarchy process (AHP).The hierarchical index evaluation structure model is used for multi-level and multi-angle control of evaluation indexes, and the objective and comprehensive evaluation behaviors are ensured.All the evaluation indexes of urban open data are divided into Level 1 Evaluation Indexes, Level 2 Evaluation Indexes and Level 3 Evaluation Indexes.All levels of indexes are linked together and progressive level by level.Different levels correspond to content of indexes with different precision and depth.Level 1 Evaluation Indexes are highly general, programmatic and concise; Level 2 Evaluation Indexes decompose Level 1 Evaluation Indexes, and form more specific types of indexes; and Level 3 Evaluation Indexes are further subdivision of Level 2 Evaluation Indexes, and complete the final index selection.They are rich in content and operational (Figure 2).
Integrity can be judged by the breadth, depth, and the degree of repetition of data.The breadth of data is reflected in the coverage of data types, and can be evaluated from the data types involved.Depth aims at accurateness and level of detail of the same type of data information provided.The degree of repetition of data can be direct evaluated by the number of data repeated.Therefore, Level 3 Evaluation Indexes corresponding to integrity index are set as data type, infor-Y.Liu et al.DOI: 10.4236/wjet.2017.53B014128 World Journal of Engineering and Technology mation depth and number of data repeated.Spatial indexes mainly focus on the spatial attribute of urban open data.Spatiality is an important feature of urban open data.The spatiality of data can be mainly judged from two aspects.One aspect is the accurate degree of spatial orientation of data.The other aspect is the performance of spatial orientation of data (such as just text description or spatial data).Therefore, Level 3 Indexes corresponding to spatial index are set as spatial granularity and spatial orientation.
First, the release time of urban open data can reflect the time background and the significance of data well; second, the update time of urban open data can reflect whether the data is latest and timely; third, different urban open data has different statistical cycle requirements, so rationality and stability of the update cycle is also one of the important aspects of timeliness.Therefore, Level 3 Evaluation Indexes corresponding to timeliness are set as release time, update time and update cycle.The main content investigated by linkage index is the degree of interoperability between different data with reference to Linked Data.Linkage requires that local data (sets) be other external data (sets) are linked together.Therefore, it can be judged by observing whether urban open data has related external links or whether external links contain return link pointing to the data.Therefore, Y. Liu et al.DOI: 10.4236/wjet.2017.53B014129 World Journal of Engineering and Technology Level 3 Evaluation Indexes corresponding to linkage are set as being linked to data and being pointed to by data.c) Decomposition of "acquisition" evaluation index link in the establishment of the evaluation system of urban open data.The weight value reflects the position and influence of the index in the overall evaluation, and directly decides the evaluation results of urban open data.Therefore, the weight value is an important manifestation of rationality and scientificity of the evaluation system.This paper adopts the analytic hierarchy process, combines with meaning and connotation of each index, and considers city planning science, statistics, information science, library science, science of public management and other disciplines of knowledge; at the same time, with the expert Y. Liu et al.DOI: 10.4236/wjet.2017.53B014130 World Journal of Engineering and Technology

After scoring, Level 3
Index is multiplied by its weight to obtain the final score of the index.The final scores of all Level 3 Evaluation Indexes are added to obtain the final score of Level 2 Evaluation Indexes.In this way, finally the total score of urban open data is calculated.The total score is essentially the weighted average of urban open city.It still complies with the classification rules of the 5-level scoring system.After calculation of the total score of urban open data, according to different Y. Liu et al.

data in the research is summarized. It is divided into three categories, including social and economic data, public service data and urban construction data. On this basis, it is further subdivided. There are a total of nine categories, including urban
economy, social activities, public facilities, landscape environment, tourism development, road traffic, municipal facilities, land use and planning management (Table1

Table 1 .
Classification and content description of urban open data.
). Figure 1.Relationship between urban open data, open data and urban data.Y. Liu et al.DOI: 10.4236/wjet.2017.53B014125 World Journal of Engineering and Technology

Table 2 .
Selection of each level of evaluation indexes of urban open data.

Table 3 .
The overall ranking result of each level of evaluation indexes.