Research Status and Hotspot Analysis of Chinese Product Design Evaluation Based on CiteSpace ()
1. Introduction
Modern product design is a series of activities that seek a design solution from requirements, and it has the characteristics of creativity, complexity, and uncertainty (Xie, 2004). In these activities, an essential link in the product system design analysis and evaluation is necessary for decision-making in the product design process (Vinodh, Jayakrishna, Kumar, & Dutta, 2014). “The so-called evaluation generally refers to measuring the object’s attributes according to a clear goal and turning it into the subjective utility (the degree meets the subject’s requirements), that is, the process of clarifying the value. In this process, we have to compare the evaluation object with a certain object to determine its value” (McDonagh, Bruseberg, & Haslam, 2002).
In recent years, the important role of product design evaluation has attracted the attention of many scholars in China. In this context, a large number of research results on product design evaluation have been produced. However, through literature collection and combing, the author found that there are few studies in the academic circle to summarize these research results. This hinders the deepening of the understanding of the academic circle and the promotion of future research. Therefore, this study will use CiteSpace software to visually analyze the literature in the field of product design evaluation to understand the research status in the field of product design evaluation and summarize the research hotspots and frontiers in the field of product design evaluation, which will help promote subsequent product design evaluation theories development.
2. Research Tools
This study’s literature data was processed using the CiteSpace information visualization software developed by Professor Chaomei Chen, an internationally renowned expert in information visualization at Drexel University, USA. The software was produced based on a JAVA application that enables visual analysis of scientific literature, tracking research hotspots in the field, and detecting research trends in the area (Chen & Song, 2019). Its distinguishing feature is its ability to bring together the vast literature of a field of knowledge in a multifaceted, time-phased, and dynamic visual language and present its evolution in a citation network knowledge map through a clever spatial layout (Chen, 2017). The software’s keyword co-occurrence, clustering analysis, cooperative network analysis, time zone view, and other functions help explore knowledge evolution in specialist areas and identify research hotspots and frontiers (Chen, 2014). However, given that the software can only outline an overview of the research field, it cannot provide more in-depth details of the literature (Fang, Yin, & Wu, 2018). Therefore, this study will rely on the results of CiteSpace’s analysis, combined with a critical reading of the literature, to further sort out the research literature to analyze the research hotspots and research trends in the topic.
3. Data Collection
To understand the research status of published literature related to product design evaluation, CNKI (China National Knowledge Infrastructure) was used as the primary search database for the Chinese literature collection. CNKI is China’s largest knowledge resource database, including journals, doctoral and master’s theses, conference papers, yearbooks, statistical data, books, standards, patents, and other resources.
In the Chinese literature collection, the search formula was selected as “Topic = Product Design Evaluation OR Topic = Product Evaluation”, the source categories were chosen as “SCI Source Journals” “EI Source Journals” “Peking University Core” “CSSCI” and “CSCD” (Chen, 2014). The search date was selected as 20 January 2021, and a total of 1475 pieces of literature related to product design evaluation were retrieved. The following were excluded from the retrieved literature to ensure data collection accuracy: 1) product design works; 2) information on magazine calls for submissions; 3) reportage articles, such as interviews with people, exhibition inquiries, conference news, congress speeches, etc. A total of 1414 pieces of literature were retained at the end. In CNKI, data collection items are the title, abstract, keywords, author, research institution, source journal, and publication time of each article (Chen, 2014).
4. Analysis of Annual Publication Volume
The annual number of publications in a knowledge area is an essential indicator of scientific research development. To a certain extent, it reflects the increase in knowledge in the area and researchers’ progress (He & Chen, 2018). Since 2021 has not yet ended, when discussing the volume of publications, 2021 has no analytical value. Figure 1 shows the annual number of articles published in
![]()
Figure 1. Distribution of Chinese literature in the field of product design evaluation research, 1992-2020.
Chinese in product design evaluation from 1992 to 2020. Overall, the number of articles in product design evaluation has shown an increasing trend, especially after 1996, which indicates that product design evaluation research has gradually attracted the attention and importance of Chinese scholars. In terms of development stages, Chinese product design evaluation research has roughly gone through three phases: a slow growth phase from 1992 to 1995, a rapid growth phase from 1996 to 2009, and a stable growth phase from 2010 to 2020. Among them, in the third stable growth stage, 2010, 2012, 2013, 2017, 2019 have all declined compared with the previous year, but basically, all have growth in the next year, especially in 2018. The development status of the annual publication volume of Chinese product design literature reflects that there is still a lot of research space in product design evaluation.
5. Keyword Co-Occurrence Analysis and Cluster Analysis
Keywords as a refined expression of an academic article’s research topic, the information contained in it include the title, abstract, and innovation of the paper, etc., which is a highly condensed version of an essay (He & Chen, 2018). From the perspective of bibliometrics, keywords with high betweenness centrality and high frequency represent researchers’ shared concerns over time: research hotspots (Chen, 2006). As a measure of a node’s power, betweenness centrality reflects this point’s importance in the network. The higher the co-occurrence frequency of keywords, the higher the point centrality, which indicates that the node is more important in this field (Chen, 2014). Figure 2 below shows the keyword co-occurrence visualization map obtained by CiteSpace’s calculation of
![]()
Figure 2. A visual map of co-occurrence of keywords in the research field of product design evaluation.
the collected data with a time slice of one year. The larger the keyword node, the higher the frequency of occurrence (Chen, 2005). Table 1 shows the core keywords with betweenness centrality above 0.02, a total of 28, and the co-occurrence frequency of the selected 28 keywords is also high. It can be found that “product evaluation” “evaluation index system” “product design” “analytic hierarchy process” “fuzzy comprehensive evaluation” “industrial design” “design evaluation” “Kansei Engineering” “life cycle evaluation” “product innovation” and “green
![]()
Table 1. Keyword co-occurrence Information.
product” are the research hotspots in the product design evaluation research field at different stages, combining the results of Figure 2 and Table 1.
At the same time, this research continues to cluster keywords through the LLR (Log-Likelihood Ratio) method to refine research hotspots in this field (Chen, 2014). Figure 3 shows that CiteSpace clusters keywords distributed in different periods and then merges the clusters into a visual map. It can be seen from Figure 3 that there are 17 cluster labels in the visualization map. These labels mainly include “life cycle evaluation” “evaluation” “program evaluation” “Kansei engineering” “product innovation” “usability evaluation” “index system (evaluation index system)” “analytic hierarchy process” “user evaluation” “quality evaluation” “design evaluation” “new product development” “comprehensive evaluation” “fuzzy comprehensive evaluation”, etc. Besides, in the cluster map, the Modularity Q value is 0.3392 (higher than the test value of 0.3 (Chen, 2006), and the Mean Silhouette S value is 0.7786 (higher than the test value of 0.5 (Chen, 2006), indicating that the clustering structure of this study is significant and the clustering results are reasonable.
Based on keyword co-occurrence and clustering results, and combined with the reading of specific articles, this study summarizes the hot topics of product design evaluation research into three aspects. The detailed discussion is as follows:
5.1. Product Design Evaluation Research Base on Evaluation Methods or Technical Perspectives
As can be seen in Table 1, Figure 2, and Figure 3, “analytic hierarchy process” “fuzzy comprehensive evaluation” “artificial neural network” “topics method” “fuzzy analytic hierarchy process” “fuzzy evaluation” “factor analysis” “data envelopment
![]()
Figure 3. Visual map of keyword clustering in the research field of product design evaluation.
analysis” and “fuzzy triangular numbers” are all methods and techniques that are used in the product design evaluation process in nature. In product design evaluation research, the discussion of evaluation methods is an important evaluation system research content. Product design evaluation methods have been developed and accumulated through trial and error in design practice and knowledge from management, operations research, mathematics, and other related disciplines (Yoon, Kim, & Rhee, 2012). To date, dozens of design evaluation methods have been proposed from different perspectives in product design theory, which can be summarized into three categories: subjective evaluation methods, objective evaluation methods, and comprehensive evaluation methods (Wang & Liu, 2021).
The first type of subjective evaluation method is judging product design according to the evaluator’s subjective feelings and rules such as aesthetic experience, use, safety, and comfort. The initial result is usually a partial or overall score of the product design, which is then processed by software tools such as Stata, SPSS, and MATLAB to obtain a qualitative conclusion (Vermeeren et al., 2010). The common methods include Delphi, analytic hierarchy process, investigation and research, fuzzy analytic hierarchy process, ring rating method, judgment matrix method; the second objective evaluation method refers to using the professional testing equipment to record the respondents’ physiological characteristics during the experiment. After data processing, the experimental results are obtained (Vermeeren et al., 2010). Still, this method is limited by the number of samples, testing environment, etc., and can be divided into experimental evaluation methods, simulation evaluation methods, etc., according to the specific situation (He, Xiao, & Deng, 2018). This type’s standard methods include eye-tracking systems, motion capture systems, surface electromyography systems, pressure distribution measurement systems, etc.; this third type of comprehensive evaluation class technique is also widely used in product design evaluation research. When a product design involves too many evaluation factors, due to the complexity of the evaluation object, it is necessary to establish a comprehensive evaluation model to help designers make decisions (Vermeeren et al., 2010). In recent research, different scholars have chosen to combine various methods in their studies to highlight the strengths of a particular method and, through the combination of methods, to compensate for the lack of the alone method itself.
Finally, based on indicators such as keyword frequency and mediated centrality, we can also identify two main characteristics of the current research on product design evaluation methods in Chinese articles: First, of the three different types of evaluation methods used, two methods, the subjective evaluation method, and the comprehensive evaluation method are still currently predominant, a feature that is also evident in the keyword co-occurrence and clustering mapping. The reason may be that objective evaluation methods, represented by eye-tracking systems, motion capture systems, etc., often require expensive equipment and personnel costs in practice, limiting many researchers. Some objective evaluation techniques do not evaluate the price and potential market demand and do not reflect direct market attitudes towards new products. So the most practical evaluation activities for companies do not occur in the laboratory but in the marketplace. Second, in the keyword co-occurrence map, the color of the lines represents the flow of knowledge. The dark area is the focus of early research, indicating that this field was once a hot topic, and then knowledge has flowed to other fields. The lighter colors represent the areas being streamed into, which are recent or current research hotspots. It can be seen from Figure 3 that compared with methods such as analytic hierarchy process, fuzzy evaluation, and TOPSIS, the fuzzy analytic hierarchy process is located in the light-colored area, indicating that the method technology is a recent research hotspot, which further verifies the discussion above the development trend of comprehensive application of research technology. In actual research, the analytic hierarchy process can use a simple structure to analyze and explore complex decision-making problems with high practical value (Vaidya & Kumar, 2006). However, this quantitative method does not fully consider the subjective nature of human thinking and the vagueness that is not easy to quantify. Design decision-making is multi-criteria decision management, and the commonly used spoken evaluation vocabulary of design is subjective, uncertain, and fuzzy. Therefore many scholars have begun to use fuzzy set theory to improve the traditional analytic hierarchy process’s relevant deficiencies as a basis for design evaluation or decision making (Chen & Chu, 2012).
5.2. Research on Product Design Evaluation Based on the Perspective of the Evaluation Index System
The evaluation index system is a comprehensive description of the product based on the main aspects of the product’s aesthetic characteristics, technical characteristics, economic characteristics, human-machine characteristics, and environmental characteristics, requiring designers and decision evaluators to consider all factors of the product design comprehensively and to objectively and fairly consider the contribution and impact of each factor on the product (Zuo & Wang, 2020). The establishment of the evaluation index system allows for the further development method of scientific evaluation and systematic comparison of product design in theory; in practice, it allows for a systematic and comprehensive assessment of product design levels and the improvement of horizontal comparisons between product design levels (Qiu et al., 2019). Based on the above keyword co-occurrence and clustering map, combined with the reading and sorting of specific articles, the author found that the related research fields of Chinese product design evaluation mainly researched the product design evaluation index system from qualitative and quantitative perspectives.
The first type of qualitative product design evaluation index system pays attention to the definition of product design measurement dimensions, measurement indicators, the various concepts of product design-related elements, and the relationships between these elements. Qualitative evaluation index systems mainly include two types of general and specific product evaluation index systems (Qiu et al., 2019). The general product design evaluation index system can be suitable for certain products or more types of products. However, due to the significant differences between different products, the general product design evaluation index system has certain limitations. Therefore, some researchers have constructed design evaluation index systems for specific products; the second type of quantitative product design evaluation index system focuses more on the quantitative integration of some product design evaluation elements. The quantitative product design evaluation index system includes three parts, measurement tools, data integration, and measurement indicators from the system’s content (Qiu et al., 2019). The existing quantitative evaluation index system also mainly includes two types of general and specific product evaluation systems (Zheng et al., 2014). The general product evaluation index system will be suitable for more products; due to this system’s high requirements’ external validity, the system is more challenging to construct, so the existing research is less involved. In contrast, the specific product evaluation index system is less applicable, only suitable for a specific product or a particular type of product; most of the existing quantitative research belongs to this category (Zheng et al., 2014).
Finally, based on the visualized knowledge map results and the analysis of specific articles, two main trends can be identified in the research on product design evaluation index systems in Chinese: First, from the perspective of product types, the prominence of keywords such as cultural and creative products, artificial intelligence, green products, children’s early education machines, led lights and industrial design in Figure 1, to a certain extent verifies that the evaluation index system of specific products is one of the trends in the field. In actual research, further case studies of evaluation index systems for various products are carried out, and quantitative studies of evaluation index systems are carried out in as many different products as possible, based on which quantitative product design evaluation index for similar products or products, in general, can be refined. In this way, it can effectively make up for the lack of research results of the general product quantitative evaluation index system due to high external validity requirements. Second, qualitative and quantitative are two perspectives for the current research on product design evaluation index system. Through combing the literature in the past two years, the author found that quantitative product design evaluation index system research is an important trend (Xia, 2018; Peng & Bian, 2021). The quantitative product design evaluation index system emphasizes the quantitative integration between elements, making a quantitative evaluation of its design level. This integrated systematic evaluation can evaluate the product design level more effectively than the single evaluation and complete the evaluation between similar products or even different products, which is of decisive significance for improving product design (Zheng et al., 2014). However, it should be noted that the current quantitative evaluation system is often aimed at a specific single or one type of product, which limits the promotion and application of the system. Therefore, it is necessary to combine the first point, carry out many empirical studies on different products, and then refine similar products’ design evaluation index.
5.3. Product Design Evaluation Theory Research Base on the Perspective of Multiple Consumer Demands
The prominence of keywords such as Kansei evaluation, Kansei engineering, green design, quality evaluation, usability evaluation, performance evaluation, comprehensive evaluation, product innovation, product cultural symbols, and user evaluation, as derived from the keyword co-occurrence map and clustering map, reflects the current diverse product consumption needs. From the above analysis, it is clear that by evaluation, we generally mean the act of determining the properties of an object according to a clear objective and turning it into a subjective utility (the degree to which it satisfies the subject’s requirements), i.e., the process of specifying the value of the thing to be evaluated, in which we compare it with a certain object to determine the value of that thing (McDonagh, Bruseberg, & Haslam, 2002). In the actual product evaluation process, consumer needs or user psychology play a fundamental role in guiding the construction of product design evaluation criteria, indicator weights, and evaluation models and is a fundamental core issue in product design evaluation research (Stanton & Young, 1997). By sorting through high-frequency keywords and specific articles, the author summarizes and concludes the research results in Chinese product design evaluation research that focuses on multiple consumer needs, divided into product design evaluation of single-dimensional needs and product design evaluation of multi-dimensional needs.
The single-dimension of product design evaluation refers only to the product dimension and can be divided into four categories: evaluation of product engineering performance, evaluation of product quality, evaluation of product emotional qualities, and evaluation of product comprehensiveness (Yamamoto & Lambert, 1994; Khalid & Helander, 2006; Peng, Huang, & Peng, 2021). Among them, the first type of product engineering performance or essential attributes evaluation research. In product design, engineering refers to the use of sophisticated technology to produce products that are functional and easy to use. The engineering is related to the product’s structure and material, which belongs to its physical aspect, which meets the consumer’s basic needs and is one of the product’s necessary attributes (Kind, 1999). The keywords concerning usability evaluation, safety evaluation, suitability evaluation, comfort evaluation, efficacy evaluation, greenness evaluation, nutritional value evaluation, stability evaluation, and performance evaluation refer to the product’s engineering properties in the visualization maps and specific articles; The second type of product quality characteristics’ evaluation research. Product quality refers to the product’s completeness and detailed functionality that can give consumers a high-quality impression. The product’s quality is related to its details, which need to be highlighted through detailed design and a high level of craftsmanship. It is also a physical attribute of the product, but it can intuitively influence consumers’ emotions and satisfaction (Yin, Liu, & Hou, 2016). In visual knowledge maps and specific articles, quality evaluation, premium quality evaluation, precision evaluation, and performance evaluation are descriptions of product quality; the third type of product evaluation research on emotional characteristics. The emotional characteristics of products are derived from the ability to arouse feelings or emotional states. When the product can change the feelings or emotions of consumers, it produces so-called emotional value (Ho & Siu, 2012). In this study, keywords related to Kansei evaluation, Kansei image evaluation, emotional evaluation, sensory evaluation, product innovation evaluation, comprehensive evaluation of beauty, aesthetic evaluation, and others in the article are descriptions of products’ emotional characteristics. Product aesthetics is also included as one of the product’s emotional characteristics because consumers can use the product shape as a transmission medium to convey information. Through the product shape elements, allowing them to form a visual perception of beauty, which in turn gives them a sense of inner pleasure (Yen, 2018); Comprehensive evaluation research on the fourth type of products. The comprehensive product evaluation study contains comprehensive product evaluation, product multiple evaluations, multi-level product evaluation, etc. It is a type of study that includes any combination of all of the product’s engineering, quality, and perceptual characteristics in the evaluation study (Wen & Yang, 2019).
The product design evaluation of multi-dimensional demand refers to the product dimension and the brand dimension, market dimension, country dimension, and even cultural dimension (Xing, Wang, & Qian, 2013). For example, Li et al. (2017) take the new Chinese living room combination furniture as an example and establish a multi-level multi-index design evaluation model based on sensory, interaction, brand, and technology from the user experience. Then, determine each index’s weight, apply the fuzzy comprehensive evaluation method to evaluate the design, etc. Based on the purpose of environmental protection and economic development, Wang (2017) provides a comprehensive examination of sports products in terms of environmental dimensions, resource dimensions, technological dimensions, economic dimensions, etc.
6. Keyword Burst Analysis
Burst keywords indicate that a research topic has received a high level of attention over some time (Chen, 2006). It is a research direction that is emerging and has excellent potential for development and value. By exploring the evolution of burst keywords, it is possible to reveal to a certain extent the development trend of a topic in a research area and to explore potential valuable research frontiers (Chen, 2014). This study’s keyword burst analysis resulted in a product design evaluation research burst keyword map, as shown in Figure 4.
![]()
Figure 4. The burst of keywords in the research field of product design evaluation.
Based on Figure 4, we can intuitively see two stages of product design evaluation research: divided by the beginning year in which the research hotspots burst. The first stage is the initial stage of research before 1993. In this stage, the overall perspective of product design evaluation theory research is relatively single. The number of research hotspots in the field is relatively small. Still, the duration of each research hotspot is long, among which “children’s early education machine” “artificial intelligence” “product design specifications” and “product cultural symbols” have all lasted for more than ten years. Regarding the hotspot burst intensity, the starting field research product objects are mainly concentrated in computer-based industrial products, agricultural products, medical products, banks, etc. From a design perspective, it also focuses on the discussion of topics such as “product design specifications” “decision-making thinking mode” “innovation risk” and “product cultural symbols”.
The second stage is the rapid development stage after 1997. In this phase, product design evaluation research began to show a rich and diverse situation. More and more cross-disciplinary research results from design, economics, management, operations research and mathematics, and related theoretical research deepened. The specific manifestation is that there are many emerging research hotspots, but each research hotspot’s duration is relatively short, and the hotspot update speed is faster than before. In terms of hot content, “green products” “cultural and creative products” and “product-service systems” have become the critical subject objects of evaluation; “fuzzy comprehensive evaluation” “fuzzy analytic hierarchy process” “fuzzy evaluation” “Neural network” and “data envelopment analysis”, etc. became the classical evaluation methods; “quality evaluation” “performance evaluation” “comprehensive evaluation” “program evaluation” “product manufacturability”, etc. have become essential product evaluation items.
Among, it can also be seen from the keyword burst map that “cultural and creative products” “program optimization” “user experience” and “product-service system” have become the current research hotspots in the research of product design evaluation theory, and will be the focus of scholars’ attention for some time in the future.
7. Conclusion
In this study, the following conclusions can be drawn by analyzing the annual publication volume, co-occurrence analysis, clustering analysis, and keyword burst analysis of 1414 papers in the field of Chinese product design evaluation research:
1) In general, the number of articles on product design evaluation is on the rise, which proves that the research on product design evaluation has received the attention of Chinese scholars.
2) Based on keyword co-occurrence and clustering results, and combined with the reading of specific articles, this research divides the hot areas of product design evaluation research into three regions: evaluation methods and technologies, evaluation index construction, and multiple consumer needs.
3) Through keyword burst analysis, the study concludes that “cultural and creative products” “program optimization” “user experience” and “product-service system” have become the current research frontiers in the research of product design evaluation theory.
The above research results in this article can provide references for subsequent theoretical research.