Content, network, and density analysis of the global and Kazakhstani literature on scientometrics

Abstract

Object: The current study offers a review of the scientometric literature from Kazakhstan.

Methods: Using two literature review approaches (content analysis and scientometric analysis), we conducted content, network, and density analysises. For the content analysis we read the full text of some influential papers in the field. For the network and density analyses, we used the VOSViewer tool.

Results: The obtained results showed that there were 17,804 articles in the scientometric literature on the global scale and 22 articles by authors affiliated with Kazakhstan. We used the VOSViewer tool to visualize the trends in the literature. This work's novelty consists of conducting a critical review of the literature on scientometrics in the Scopus database, the use of analysis methods that did not have been used in scientometric research in Kazakhstan. Secondly, we determined the development level of the scientometric literature in Kazakhstan.

Conclusions: The major findings can be used to assist researchers and policy makers in science management in determining the evolutionary scientometric characteristics of the global literature, in general, and the Kazakhstani literature, in particular, as well as to identify patterns in the development of science.

Introduction

The science sector of a country is crucial in contributing to its economic and social wealth. One of the integral parts of this sector is the scientific potential and technological development, which determine the nation’s place in the global economy. For Kazakhstan, the effective utilization of scientific potential and its globalization are the critical actions pursued by the government to improve the national science system (The Law of the Republic of Kazakhstan «On Science», 2011).

In turn, the improved science system can contribute to implementing national strategic goals in the education and science sectors. In particular, one of these goals, «the increase in the contribution of science to the socio-economic development of the country», is explicitly declared in the State Program for the Development of Education and Science of the Republic of Kazakhstan for 2020–2025 (State program of education and science development in Kazakhstan for 2020–2025, 2019). In the long-term, the realization of this program contributes to the implementation of the national strategic initiatives such as the Kazakhstan-2050 Strategy, the Strategic Development Plan of the Republic of Kazakhstan until 2025, and the Nation Plan «100 concrete steps to implement the five institutional reforms» (The Law of the Republic of Kazakhstan «On Science» (2011), State program of education and science development in Kazakhstan for 2020–2025 (2019), Strategic Development Plan of the Republic of Kazakhstan till 2025 (2018). Therefore, the assessment of the science sector by analyzing its output in international journals is of paramount importance to science management in Kazakhstan.

The current study offers a review of the scientometric literature from Kazakhstan. Also, we compare our assessment of the local scientific productivity with one on the global scale. Accordingly, we attempt to answer the following questions:

1. What are the main characteristics of the scientometric literature in Kazakhstan?
2. What do network and density analyses suggest?
3. What are the implications of our findings for improving scientific productivity in Kazakhstan?

In the next section, we present our methodology, which includes the data collection process from Scopus and our analysis methods. Then, we report our main results and significant findings in two directions: those derived from analyzing the scientometric literature on the global scale, and from Kazakhstan. As a result of this, comparing the Kazakhstani scientometric literature with the one on the global scale is performed. Finally, the paper summarizes the study's important findings and highlights the benefits for researchers and policymakers in science management.

Methodology

Data collection

We used a search field «title/abstract/keyword» in Scopus to collect the relevant publications. The search terms were «bibliometric», «science mapping», and «scientometric». We conducted two types of analyses. First, the content analysis, was targeted to understand the scientometric situation globally and find research gaps in the literature. Second, the scientometric analysis, was targeted to visualize the current publication pattern using the VOSViewer tool. To visualize the state of scientometrics in Kazakhstan, the VOSViewer tool was used, and the relationship between frequently encountered keywords was displayed. As a result, a map of the network was created. A comparative analysis was carried out for a broader presentation between the Kazakhstani and global networks in VOSViewer.

The search returned 17,804 articles related to the scientometrics literature globally, and 22 of them were from Kazakhstan. We aimed to identify papers that had one of the highest annual citation rates. Therefore, 17,804 articles were further ranked by citation rate. As a result, 15 articles were identified with an effective annual citation rate of over 71 times. All articles were found by using keywords to identify relevant and most frequently occurring words, which are «bibliometrics», «science mapping», and «scientometric». Finally, for the content analysis we read the content of 8 global and 7 local articles, for the scientometric assessment with the network and density analyses all 17, 804 articles were uploaded into VOSViewer (Table 1).

Table 1. Outline of the research approach

Methods

Over the past decade an assessment of research productivity and impact using various performance indicators has become common (Glanzel et al., 2016; King 2004). One of such widely used approaches is scientometrics, part of science and technology policy in many countries, with dedicated international associations, journals, and conferences (Sandoval-Romero et al., 2019; Guskov et al., 2016; Heilig et al., 2015). The term «scientometrics» (naukometriya) was created by the Russian scientist Nalimov in 1969, and means «measuring the science». On the one hand, science mapping is helpful because it can assist a researcher in investigating international research collaborations by building visualized networks, cluster maps, evolution timelines, and trend patterns. Such an analysis reveals research relationships across various publication attributes such as countries, research areas, journals, and authors. On the other hand, scientometrics is used to evaluate scientific contributions inductively by analyzing the corpus of literature with quantitative methods such as co-author analysis, co-word analysis, co-citation analysis, and cluster analysis.

Even though publications are organized in databases like Scopus, the rapid growth of global research collaboration has made networks of co-authors and citations complex. From the scientometric viewpoint, this resulted in «inflationary values» (Persson et al., 2004; Shelton, 2020), and it is almost impossible to analyze and measure them with traditional approaches like content analysis, systematic literature review, and expert judgment. Thus, the review, analysis, visualization, and density analysis of such a massive amount of world literature is a burden on traditional labor, and VOSViewer software packages and mathematical models have become available to shift such burden to computer algorithms. Such tools and models were used in similar studies by Coile et al. (1977), Bensman and Smolinsky (2017), as well as by the current study’s authors in previous related studies by Narbaev et al. (2020) and Narbaev (2015). We performed the content analysis by screening and analyzing abstracts and keywords to identify the most relevant articles in scientometrics.

Main results and findings

Following our research approach introduced in Table 1, we present our main results and findings in the following four subsections. They reflect the content analysis of the research on scientometrics globally and in Kazakhstan (Step 2 of our approach in Table 1) and network and density analyses (Step 3 and 4 of our approach in Table 1).

1. The global research in scientometrics

Glanzel and Zhang (2018) found 6 paradigms that explain the power of the scientometrics approach in analyzing the body of knowledge. This included the need for advanced information systems, monitoring and evaluating performance in research and technology, monitoring dynamics and evaluating research results and their impact, and reasoning globally and regionally. Their methodology included collecting publications from Web of Science, the count of bibliometric indicators, and the creation of networks of emerging topics. The authors studied 16 developing countries and, using citation indicators, evaluated and compared the development of science, mainly in the Latin American and Asian countries. Also, using the example of Nigeria, they concluded that purely relying on the scientific productivity of local publications gives slow development. On the contrary, the co-authorship of local scientists with scientists from more developed countries ensures the rapid growth of local scientific productivity.

The following work by Pillai Sudhier (2013) assessed the classical law in scientometrics and science mapping, known as Lotka’s law. The author used the law to assess the publication productivity of researchers in the physics field. Based on the productivity of doctoral dissertations of the Kerala University, India, the author applied the Chi-test and Kolmogorov-Smirnov test in order to assess the validity of the Lotka’s law in the field. The author found that the publication productivity in this field per researcher is higher than in the other fields of science.

Shelton (2020) introduced a different approach compared to previous studies. The author proposed two alternative scientometric laws, which were able to analyze the relationship between the national output of scientific papers and the national funding of these papers. The first model was based on regression modeling, while the second on mathematical modeling of the publication trends. The respective data on publications were collected from 93 countries from the UNESCO database and 43 industries from the OECD database. Shelton found that China would increase the state funding into the research publications by 2.5 % by 2020, increasing China’s publicity. The two laws also helped determine the leader counties in publication productivity, including the USA, the European Union, and China. However, while China was far behind its two predecessors, it grew fast with an annual increase of 19 % in its publications. The model also found that China would continue to develop just as quickly, while the U.S. and the European Union had very slow growth dynamics. As a result, at some point, China may become the leader in terms of the number of journal papers contributed to the overall global literature. Overall, the relevance of this study is high since it helped determine the state of scientific development of a country and the world.

Glanzel et al. (2006) examined the state of science in Brazil and compared it with the ones of other countries around the world. In their methodology, the analysis took a 3-year citation window for articles published from 1991 to 2003 and covered 12 fields of science. They applied numerous bibliometric indicators such as Activity Index, Advantage Index, and Citation Index. The comparison revealed that Brazil was the leader in Latin America regarding the number of publications, but their scientists preferred to publish in local journals and in the local language, which made it difficult to access with the productivity on the global scale. Overall, we see this work as an excellent example for analyzing the state of scientometrics research in Kazakhstan. Some techniques can be applied to Kazakhstan too. The situation in Kazakhstan is similar to Brazil in that local scientists are more likely to publish in local journals, which are in Russian and Kazakh. This, in turn, limits the general availability of Kazakhstani articles and the country’s productivity to the global scale, which communicate and publish mainly in English.

Padalkar and Gopinath (2016) analyzed existing and future research trends in the field of project management. Their two-stepped methodology used the search engines of Google Scholar, ProQuest, Emerald Insight, Scopus, and ABI/Informs. In the first part, they studied the existing literature and synthesized them, while in the second part, they classified the collected literature. Analyzing about six hundred publications, they determined the existence of deterministic research, while there was a lack of thematic analysis, theory, and non-deterministic research in this field. The authors insisted on investing in writing non-deterministic papers to create more insightful topics in the project management field.

Henriksen (2016) analyzed the global author collaboration patterns in the social sciences (economics, political sciences, psychology, sociology and anthropology, public administration, management, and planning and geography) for the past 34 years. The author found that the social sciences fields have become more centric in certain areas of research, which required more tight author cooperation and financial costs. Also, working with large amounts of data and quantitative research methods was common in the field. The study also analyzed the productivity of doctoral students and their collaboration with more senior researchers. The increased collaboration and co-authorship in the long term affects the maturity of a particular scientific field.

Rosetto et al. (2018), in their study of innovation research, used citation analysis to determine the size and composition of research networks and the connections of authors in a particular field. For this, they clearly defined the boundaries of a research field and revealed the dynamism of the research field. This is also important for Kazakhstani researchers. Often, the Kazakh authors publish in collaboration with the Russian scientists due to the close location of countries, economic and political ties.

In Kalachikhin’s (2018) work, a scientometric classification was proposed as a tool for revealing the existing problems in Russian science. They used expert judgment on the local research productivity and analyzed using a set of bibliometric indicators. The author claims that scientometrics should analyze the opportunity to move science from purely knowledge creation into technology creation. This was crucial because the science and the real industry in Russia are separated.

2. The local research in scientometrics

The interest in scientometrics research has been growing in recent years in Kazakhstan. Suluimanov et al. (2009) assessed the activity of Kazakhstani scientists in 1991–2008 based on analysis of citations of published articles using data derived from the Scopus database in CIS countries. The findings revealed cooperation between scientists in CIS, specifically Kazakhstan and Russia, in physics and chemistry. The dynamic of science activity of local authors is essential to understand potential growth areas of research and, more specifically, cooperation.

Narbaev (2015) applied co-word analysis to study 826 articles published in project management in Kazakhstan in order to identify the project management areas, their relationship with other management areas, and the co-word analysis of the most occurring terms. The co-word analysis of local papers is crucial in understanding the direction of a particular research area. Moreover, it contributes to potentially growing areas.

Scientometrics can be widely applied in the university context. Abdramanova et al. (2019) used scientometrics and science mapping methods to assess entrepreneurial education in higher education in Kazakhstan. This paper applied scientometrics to demonstrate university from the educational side. Also, Kuzhabekova and Lee (2018) used scientometrics systems to study research and publication activity of universities in Kazakhstan. The research methods to analyze Kazakhstani studies based on 361 peer-reviewed articles from the Web of Science database were bibliometric analysis, link analysis, and publication content. The study revealed the essential nature of collaborations to grow research ties that are highly dependent on time. Moreover, international researchers employed in Kazakhstan positively affect the expansion of collaboration of local universities and their entry into international research ties.

A few studies had contributed to the research of local scientific activities by using various methods of scientometrics, comparative analysis, and identified current trends in scientometrics research related to Kazakhstan. Suleymenov et al. (2011) analyzed bibliometric indicators such as publication activity, citations by subject area in Kazakhstan and other CIS countries in 1996–2006. The study revealed the leading role of Armenia in citation analysis compared to other studied countries in the European part of the CIS (Russia, Armenia, Azerbaijan, Belarus, Moldova, and Ukraine). Kazakhstan and Uzbekistan have a stable increase in publication activity in Central Asia, although Turkmenistan and Kyrgyzstan had reached a level compatible

on an international scale at some point. Then, an analysis of publication activity in 2009–2017 by Smagulov et al. (2018) had revealed an increased number of Kazakhstani papers in the Scopus database and, at the same time, an increased number of publications in low-ranking journals. Recently, Zhanabayev et al. (2020) used several methods (forecasting, various models and methods of bibliometrics, scientometrics analysis) to study the publication activity of Kazakhstani researchers and identified that 60 % of research takes place in the natural sciences, engineering and technology. The bibliographic indicators for 2017 revealed that the largest number of papers were published in physics and astronomy.

The review of the global and local literature in science mapping and scientometrics research revealed the effectiveness of research in the global and local context, trending topics in research, and methodologies used. The analysis demonstrated that scientometrics research in Kazakhstan had not reached a level comparable with the world.

3. Network analysis of the global and local research in scientometrics

As per Step 3 of our research approach (Table 1), we conducted the network analysis on the collected articles. Figure 1 shows a visualization of the global network of keywords in the scientometrics literature. The network has 24,372 keywords, out of which 1,742 items meet the requirement of 5 occurrences that represent the keywords that have links among them and have a pairing relationship. Global keyword distribution designates the rapid growth of scientometrics on a global scale. The figure demonstrates links that are strong have been selected by the VosViewer. It is visually shown that scientometrics has been comprehensively studied and has a uniform distribution over clusters. Words as «bibliometrics», «publications», «scientometrics», «research evaluation», and «Scopus» have a higher weight and have the most strong and frequent links. The higher the weight of a link, the more influential the item is in the cluster. 1,742 keywords create 24 clusters that have 31,294 links.

The uneven distribution of scientometrics and bibliometrics in Kazakhstan literature (Figure 2) is identified. The network has only 104 keywords out of each 61 items are connected. Major developing areas are education, medicine, biotechnology, higher education, public administration. 61 keywords create 8 clusters that have 200 links. The relationship between keywords and clusters is rare and has the strength of a single relationship in a given network. This suggests that the development of scientometrics in Kazakhstan is at its early stages. Therefore, investments of government and other organizations are critically important to support scientists and increase publications in this area.

89

4. Density analysis of the global and local research in scientometrics

The density color is an indicator of the density of areas — the greater the number of elements in one area, the closer the color to red. The fewer elements, the closer the color is to blue. Figure 3 shows the density view of the global network of the scientometric literature. There are primarily green colored spots, representing clusters with a medium number of items and medium weight in a neighboring space. Consequently, the clusters with green color are h-index, research evaluation, scientific impact, clustering, co-citation, clustering, knowledge mapping. At the same time, there are yellow areas with high weight and a high number of items in a neighboring space like bibliometrics, publications, Scopus, and machine learning. Finally, the rest of the items have a blue color because they have a low density in their clusters. As a result, yellow areas are supposed to be developed and studied actively, while green-colored items have medium popularity in science, whereas blue-spotted items have a few activities in global research.

The density network of the global scientometric literature is different from the Kazakhstani scientometric literature, as presented in Figure 4. The keywords such as bibliometrics, machine learning, CiteSpace, and h-index were the most occurring in the global density network. While on the Kazakhstani density network, such keywords as bibliometrics, VOSViewer, child welfare, biotechnology, education, science mapping, and bibliometric review are the most addressed by the local researchers studies in Kazakhstan. This can be explained by the fact that today these areas are significant for Kazakhstan, since the most advanced approaches in science, such as scientometrics, mapping, and bibliometrics, are growing. However, this means there are prerequisites for growth in other related areas.

Discussion

Our review revealed the growing interest in the research using scientometric methods, both globally and in Kazakhstan. On the global scale, multiple methods were used in the past studies to evaluate the performance of researchers and publication activity in certain areas, including such methods as the Lotka’s law, Chi-square test, and Kolmogorov-Smirnov test. Moreover, we found out that the maturity of a given scientific field is highly dependent on collaboration activity and the ability of authors to collaborate on an international level. However, on the local scale, we revealed the growing trends in science mapping and scientometrics research. The local research is at the early stages of its development, therefore, there is a significant research gap. Yet, positive trends and interest by the local researchers demonstrate the opportunity to analyze the local research using a variety of bibliometric methods in the future.

The network and density analyses demonstrated trends and potential areas for further developing the research in scientometrics. They revealed the focus of the global and local studies in scientometrics. In particular, the density analysis shows clusters with growth potential, although there are no clusters with a large number of keywords. This implies that scientometric literature in Kazakhstan has the potential to grow extensively in the coming years. The difference between the density analysis of the global literature and Kazakhstan literature shows several trending research themes related to bibliometric approaches and VOSviewer, the focus on education, and others. Improvement in the science management for Kazakhstan is aligned with the government's strategic goals to support research and encourage publication activity among scientists. The government’s focus on science development and recent initiatives to boost the international publication activity of researchers is critical, and our analysis confirms this. The scope of research, areas for potential collaboration on a local and global level, are identified in this study and can be used in a decisionmaking process related to the development of science and research productivity of local authors. Overall, as the findings show, assessing the scientific potential and its literature output in international outlets is critical to scientific management in Kazakhstan.

Conclusion

This paper proposed an approach to comprehensively review the scientometrics studies of the global literature and Kazakhstan, in particular. Moreover, we used the scientometrics methods that were not used in Kazakhstan to analyze the literature. The following analysis was used: citations, research productivity, keywords, network, and density. The analysis revealed the early stages of the development of literature in Kazakhstan. However, the positive trends over the recent years have revealed the main directions of development, research relationships, and their density. Also, the funding by the government into science is Directly

Reflected in the increase of scientific publications, mainly in education, medicine, biotechnology, public administration subject areas.

We note that this study is part of the broader research project on the evaluation of the scientific productivity of Kazakhstan and, therefore, the results and findings from this specific portion of the research offer a theoretical basis for further planned research in this project. In particular, the findings from the current study are recommended to be considered in identifying the trends in the development of the scientific potential in and an objective assessment of the current state of the science development in Kazakhstan. In addition, this study offers a tool to identify the evolutionary scientometric characteristics of the global and Kazakhstan literature.

Acknowledgments

This research was funded by the Science Committee of the Ministry of Education and Science of the Republic of Kazakhstan (Grant No. AP08856031).

References

1. Abdramanova, N., Algozhaeva, N., Huseyin, H., Mambetova, A. (2019). The modernization processes of the service quality of entrepreneurial education of Kazakhstan universities: expert analysis. Journal of Entrepreneurship Education, 22(3), 1–10.
2. Bensman, S.J., Smolinsky, L.J. (2017). Lotka’s inverse square law of scientific productivity: Its methods and statistics. Journal of the Association for Information Science and Technology, 68(7), 1786–1791.
3. Coile, R.C. (1977). Lotka’s frequency distribution of scientific productivity. Journal of the American Society for Information Science, 28(6), 366–370.
4. Glanzel, W., Leta, J., Thijs, B. (2006). Science in Brazil. Part 1: A macro-level comparative study. Scientometrics, 67(1), 67–86.
5. Glanzel, W., Zhang, L. (2018). Scientometric research assessment in the developing world: a tribute to Michael J. Moravcsik from the perspective of the twenty-first century. Scientometrics,115, 1517–1532.
6. Guskov, A., Kosyakov, D., Selivanova, I. (2016). Scientometric research in Russia: Impact of science policy changes. Scientometrics,107, 287–303.
7. Heilig, L., Vob, S.A. (2015). Scientometric Analysis of Public Transport Research. Journal of Public Transportation, 18 (2), 111–141.
8. Henriksen, D. (2016). The rise in co-authorship in the social sciences (1980–2013). Scientometrics,107, 455–476.
9. King, D. A. (2004). The scientific impact of nations: What different countries get for their research spending. Nature, 430, 311–316.
10. Kalachikhin, P.A.(2018). Funding of basic research based on scientometric criteria. Scientific and Technical Information Processing, 45 (3), 121–127.
11. Kuzhabekova, A, Lee, J.T. (2018). International Faculty Contribution to Local Research Capacity Building: A View from Publication Data. Higher Education Policy, 31(3), 423–446.
12. Narbaev, T., De Marco, A., Orazalin, N. (2020). A multi-disciplinary meta-review of the public-private partnerships research. Construction Management and Economics, 38 (2), 109–125.
13. Narbaev, T. (2015). Project Management Knowledge Discovery in Kazakhstan: Co-Word Analysis of the Field. Proceedings of the 12th International Conference on Intellectual Capital, Knowledge Management and Organisational Learning IKI-SEA of Bangkok University. Thailand, 5–6 November 2015. (69–175).
14. Padalkar M., Gopinath, S. (2016). Six decades of project management research: thematic trends and future opportunities. International Journal of Project Management, 34, 1305–1321.
15. Persson, O., Glanzel, W., Danell, R. (2004). Inflationary bibliometric values: The role of scientific collaboration and the need for relative indicators in evaluative studies. Scientometrics, 60 (3), 421–432.
16. Pillai Sudhier, K.G. (2013). Lotka’s law and pattern of author productivity in the area of physics research. Journal of Library & Information Technology, 33 (6), 457–464.
17. Pravitelstvo RK. Gosudarstvennaia programma razvitiia obrazovaniia i nauki Respubliki Kazakhstan na 2020–2025 gody. Utverzhdena Postanovleniem Pravitelstva Respubliki Kazakhstan ot 27 dekabria 2019 goda № 988 — NurSultan, 2019 [State program of education and science development in Kazakhstan for 2020–2025] [in Russian].
18. Pravitelstvo RK. Strategicheskii plan razvitiia Respubliki Kazakhstan do 2025 goda. Utverzhdena Ukazom Prezidenta Respubliki Kazakhstan № 636 ot 15 fevralia 2018 goda — Astana, 2018 [Strategic development plan of the Republic of Kazakhstan till 2025] [in Russian].
19. Rosetto, D.E., Bernardes, R.C., Borini, F.M., Gattaz, C.C. (2018). Structure and evolution of innovation research in the last 60 years: review and future trends in the field of business through the citations and co-citations analysis. Scientometrics, 115, 1329–1363.
20. Sandoval-Romero, V., Larivière, V. (2019). The national system of researchers in Mexico: implications of publication incentives for researchers in social sciences. Scientometrics, 122, 99–126.
21. Shelton, R.D. (2020). Scientometric laws connecting publication counts to national research funding. Scientometrics, 123, 181–206.
22. Suleymenov, E.Z., Ponomareva, N.I., Dzhumabekov, A.K., Kubieva, T.S., Kozbagarova, G.A. (2011). An Assessment of the Contributions of Kazakhstan and Other CIS Countries to Global Science: the Scopus Database. Scientific and Technical Information Processing, 38(3), 159–165.
23. Suluimanov, E.Z., Frolova, V.A., Khasenova, S.K. (2009). The scientometric analysis of the activity of Kazakh scientists based on the materials of the Scopud DB (Netherlands). Scientific and Technical Information Processing, 36(5), 290–297.
24. Zhanabayev, R., Sagintayeva, S., Abildina, A., Nazarov, A. (2020). The use of the foresight methods in developing an algorithm for conducting qualitative examination of the research activities results on the example of the Republic of Kazakhstan. Mathematics, 8, 1–19.
25. Smagulov, K.E., Makanova, A.U., & Burshukova, G.A. (2018). Analiz naukometricheskikh pokazatelei publikatsionnoi aktivnosti kazakhstanskikh avtorov v izdaniiakh, vkhodiashchikh v bazu dannykh Scopus / [Analysis of scientometric indicators of publications of Kazakh authors in publications included in the Scopus database]. / Vestnik KazNU imeni Al-Farabi. Seriia yekonomicheskaia — Bulletin of KazNU named after Al-Farabi. Economic series, 1 (123), 233–253 [in Russian].
26. Zakon Respubliki Kazahstan «O nauke». Ot 18 fevralia 2011 goda № 407-IV (s izmeneniiami i dopolneniiami po sostoianiiu na 28 oktiabria 2019 goda) // Kazakhstanskaia pravda. — № 66–67 (26487–26488). — 2011. 23 fevralia [The Law of the Republic of Kazakhstan «On Science»] [in Russian].