Performance Assessment Based on T-Indicator

A novel indicator named after Tianjin University (TJU)-T-indicator-was investigated as an effective supplement of established Article Assessment System of Tianjin University, aiming to correct differences among fields. Based on normalized citation counts, T-indicator could give the order of research performance of researchers or groups in different disciplines. Weighted citation analysis was also introduced in this method to judge the contribution of researchers to their research outcomes. A given example was used to thoroughly discuss this evaluation method, via the application of derivative indices, including Tyear, Taverage, Ttotal and weighted-Ttotal.


Introduction
Research performance assessment (RPA) plays important roles in universities and research institutions, especially in the process of recruitment, academic promotion, offering tenure, granting, etc.The general indices of RPA include publications, patents, awards, and grants.It is hard to evaluate the quality level of patents, awards, and grants among different institutions and countries as there is no same standard.However, journal publication, mostly published after peer reviews, is a good and unique index for internal and external comparison.Nowadays, journal publication has been widely used officially or subconsciously in the process of RPA.An article assessment system has been successfully established based on both Tianjin University and nine key Chinese Universities' academic disciplinary benchmarks [1].With this scientific benchmarking system, the quality of a researcher's papers could be easily located in a percentile scale in corresponding field and within certain groups.Several factors, including total number of papers, order of authors, impact factor of journals, citation count, h-index [2], e-index [3], a-index [4], m-quotient [2], as well as weighted citation analysis [5], were also utilized for both quantity and quality analysis.This article assessment system has played a significant role as an important part of RPA in Tianjin University.However, with unique advantages in comparing researchers or groups in a same field, it is hard to tell their RPA in different fields.To improve this article assessment system, citation counts were normalized for correcting differences among fields.Breaking the boundary of disciplines, this modified citation-based article as-sessment system could easily give the order of research performance of researchers or groups even in different disciplines.

Methods
The average number of citation count of all TJU publications from Scopus citation database are obtained for each discipline and for each year from the year of 2001 to the year of 2009, based on the accumulation of citations from the year of publication to the current year (Equation (1)).
, 1 where , i j are the citations received by the i th paper in the year j, and j n is number of papers published in the year j.On the left hand of Equation (1), j AC represents the average number of citations received in the period from year j to 2009 by papers published in the year j.
To obtain the total T-indicator (T total ), annual T-indicator (T year ) are required to be calculated firstly: the sum of a researcher or group's actual number of citations of all publications is divided by the above average number for each year in the same discipline (Equation (2)).
where j m is the number of papers published by an in- dividual researcher or a group of researchers in the year j, and y j TJ is the ratio of the average citations received for an individual researcher or a group of researchers in the year j, over the average number of citations received in the year j of the whole university, both in the same discipline.
The average number of T year is the T-indicator (Equation (3)), and the standard deviation (SD) is also calculated to show the stability of research performance (Equation ( 4)).
  where y 1 is the first year of the period in which the research performance of an individual researcher or a group of researchers are required to be analyzed, and y 2 is last year of this period required to be analyzed.The following example is taken to discuss the application of T-indicator.Tianjin University announced the competition for a 3-level award funding for research performance, and there are 8 candidates entered the last round.In the process of research publication assessment, as shown in Table 2, all of them are excellent in their research fields, and some of them have similar number of publications (Candidate 3 and Candidate 5), total citation count (Candidate 3 and Candidate 8), and average citation count (Candidate 1 and Candidate 5) as well.Furthermore, considering the property of citation frequency in different research areas, it is very hard to simply compare them via the common indices, including citation count, h-index, e-index, etc., as mentioned above.However, T-indicator, based on normalized citation count, could be conveniently used here to give the order of research performance as a helpful reference to the award funding committee.In Scopus citation database, collected journals are categorized into 25 disciplines; however, due to the relativity among certain fields, publications of some journals are subjected to 2 or even more disciplines.In such case, the average of T-indicators of different disciplines could be used instead, due to the normalized native of T-indicator.For example, Candidate 1 has published 197 articles, which are categorized to "Physics & Astronomy" (140) as well as "Material Science" (102).Apparently some of the publications are classified to both disciplines by Scopus.

Results and Discussion
As shown in Table 3, in Discipline 1-the category of "Physics & Astronomy", averages of citation count of different year were calculated firstly (Row 3), which were then divided by the corresponding average number of citation count of all TJU publications for each year in Table 1, and the quotients obtained (Row 4) were T yearindicator.T total (1.53) and SD (0.91) were then subsequently calculated.The same method was also been used to calculate the T total (0.83) and SD (0.97) of publications in Discipline 2 of "Material Science".Finally T average-year and T total (1.18) were achieved by simply computing the mean value of them in different subjects.SDs indicated the consistency of research performance of Candidate 1 in the same discipline.
As shown in Figure 1, T average-year could also show an individual annual research performance.For Candidate 1, his T average-year hit the peak (2.83) in the year of 2002, and reached the bottom (0.22) in the year of 2008, presenting a decreasing research performance.However, the T aver- age-year of Candidate 2 has gradually climbed up since the year of 2001, and a sudden jump to the maximum of 7.14 appeared in the Year of 2009, demonstrating an increasing research performance.A conclusion could be drawn that both Candidate 1 and 2 are very excellent in their own research field as their Ts average-year are almost over 1, and Candidate 2 showed higher potential in research.
When comparing the research performance among more scholars in different disciplines, T total displays unique advantages.As shown in Table 4, T total of each candidate was calculated, and from these data, Candidate 8 showed the best research performance with the highest T total of 5.47, followed by Candidate 5 and Candidate 4, with 3.19 and 2.69, respectively, and the poorest performance in this group is Candidate 6, showing the lowest T total of 1.06.
For further analysis when considering candidates' contributions to publications, weighted T-indicator is introduced based on weighted citation analysis.The use of weighted citation analysis has been thoroughly discussed elsewhere (Zhang 2009b, Zhang 2010), which is a Copyright © 2012 SciRes.OJS  As described above, the research performance of these 8 candidates was quantitatively analyzed via this assessment method, which could give helpful reference to the award funding committee but still need the comprehensive qualitative evaluation via peer reviews, to get a final reasonable evaluation result of research performance of

Conclusion
This new article assessment method, via the application of T-indicators, was established successfully for correcting differences among disciplines.An example was given to describe this whole assessment procedure which could not only give the research performance curve with year of candidate each, but also could provide the order of their research performance.Last but not least, because of the increasing citation times with time, the

Table of
Mean of Citation Count of all TJU Publicationsis prepared (Table1) for 25 disciplines from the year of 2001 to the year of 2009.Total number of TJU publications over 9 years and of each year, as well as the annual mean citation count were all included for every category in this table.For example, in category of "Agricultural and Biological", total number of TJU publication is 388; the number of publications in the year of 2001 and the mean citation count is 11 and 14.18, respectively.