Transformation of the state planning system — from state programs to national projects: the experience of Russia

Abstract:

Object: Today, almost all countries of the world use a program-target approach to the development of the country's economy. Regional, industrial and government programs, national projects have become an integral part of the country's socio-economic activity, a tool for solving topical problems and achieving specific goals in various sectors of the economy. The transition from the format of state programs to national projects means changing the approach to solving topical issues. National projects are a direct state response to current system requests from society. The new state management format also requires the revision of the mechanism of selecting monitoring and updating of development projects. The purpose of this article is to study the practice of managing current national projects in Russia as a development tool.

Methods: During the research, the methods of comparative analysis of the practice of selection and evaluation of the effectiveness of national projects in the regions of Russia; analysis of the expert opinions on the experience of monitoring and implementing existing national projects of Russia have been applied.

Results: In the result of research the mechanisms of selection, monitoring and re-enabling projects into the system of national projects of Russia; the problems of coordination and linking the mutual interests of the beneficiarys and administrators of projects and ways to develop the mechanisms of coordination of interest have been identified.

Conclusions: Summarizing the studied experience of Russia in the transition from state programs to national projects, a new approach to the mechanism of dynamic monitoring and update of projects can be highlighted, when projects with low implementation efficiency are excluded from the composition of national projects. In their place, from the projects that have passed the examination and have a high assessment of the quality of their development come from “basement of projects”. Despite this positive practice in the management of national projects, the following problem areas can be identified: systemic management problems, poor planning quality in the formation of a system of project indicators, motivational problems.

Introduction

For thirty years of independence more than 400 regional, sectoral and government programs have been developed and implemented, which demanded significant financial resources from the country's budget. Evaluation of the effectiveness and productivity of the implementation of sectoral and government programs was conducted by both administrators and third-party experts over a number of years. Thus, in 2007–2009, RSE Institute for Economic Research of the Ministry of Economy and Budget Planning of the Republic of Kazakhstan together with the experts made monitoring and an evolution of the development and implementation of the 300 government, sectoral and budget programs. At that time, it was the first experience in the CIS countries to evaluate all the programs of Kazakhstan.

The objects of the assessment were the definition as follows:

• feasibility, timeliness, relevance of the tasks and their implementation;
• efficiency and efficiency of development;
• impact of program and sustainability of the results.

The result of the assessment is useful for processing both the content of programs and the methods of their implementation. The subsequent generation of state and sectoral programs in Kazakhstan has illustrated much higher efficiency and effectiveness, which have directly affected the economic growth and increase in the well-being of Kazakhstan citizens.

Currently improving the program-target approach in the state management is also expressed as a change in form of the target managing. In this regard, Kazakhstan, as many states moved to the design method of management. President of Kazakhstan, K.-J. Tokayev, approved ten national projects aimed at obtaining public benefits, which concern each citizen’s education, health, ecology, state services, safety, quality of life. The transition from the format of state programs to national projects means changing the approach to solving topical issues. Therefore, the issue of studying the experience of Russia on selection, monitoring, and evaluation of the effectiveness of national projects is relevant.

Literature Review

The twenty-first century can rightfully be called the time of global projects. In its first decade, international institutions were built that determine the socio-economic development of individual territories and countries. The organization of regional entities and even the administrative division faded into the background. Erasure of borders, total transparency became the main pillar of economic growth and equalization of social space, when residents of less developed countries could easily find themselves in more favorable conditions while migrating within the framework of one state (union of states). The abolition of visa barriers, the unification of management processes, and the standardization of products have largely levelled regional barriers. However, the intensification of the crisis in both the economic and geopolitical spheres led to a sharp change in paradigms. Over the past five years, the role of international institutions has significantly decreased, the coronavirus pandemic and political controversy have returned protectionism to the agenda. The socio-economic space began to fragment.

Gills B.K. (Gills B.K., 2001) and Conteh C. (Conteh C., 2013) emphasize that the high instability of the economic environment creates problems for countries and individual regions, which should be taken on the responsibility due to the decrease in the effectiveness of universal solutions to socio-economic problems. The question remains, in what direction and in what form to implement strategic initiatives, looking back on past successful approaches or shaping the agenda based on global trends.

The most effective form of such public administration has become state programs and projects aimed at achieving the development goals of various industries, spheres of economic activity, and territories. At the end of the 20th century, the project approach in the state administration of various countries of the world moves from the military-industrial and space spheres to the sphere of socio-economic development of society. Many development institutions such as the World Bank, OECD, UNDG have developed recommendations for planning, monitoring and evaluating the implementation of state and public projects and programs (OECD, 1991; World Bank, 2007; UNDG, 2007; OCHA, 2007; Feinstein O. et al., 2003). Also, over the past ten years, there has been an interactive resource center for evaluation (https://erc.undp.org/) that allows to get acquainted with programs and projects of different countries, the practice of their development, monitoring, and evaluation.

Sprague R.H. et al. (Sprague, R. H. et al., 1982) consider the project approach as a decision-making system tool for managers at various levels in the implementation of changes.

Van Gundy et al. (Van Gundy et al., 1988) describe the project approach as a method of solving the structural problem in practice of an unstable environment.

A. Walton considered project planning as a form of information development and communication in society (OECD, 1995).

Sander C. (1997) examines the effectiveness of government and community programs and projects. He concludes that the effectiveness and efficiency of programs and projects directly depends on the mechanisms for monitoring and evaluating programs and projects that are designed and used at each stage.

Vetrov G.Yu. et al. (Vetrov G.Yu. et al., 2003) develop approaches to assessing municipal programs and projects using quantitative and qualitative research methods.

Alexandrova A.L. et al. (Aleksandrova A.L. et al., 2005), using practical examples of social projects in various cities and regions of Russia, demonstrate the mechanisms for introducing a project monitoring system to increase their efficiency and effectiveness.

Soderlund J. (Soderlund, J., 2004) in his study summarized thirty years of project management practice in the public sphere, and revealed new patterns in connection with the increase in the mobility of people, knowledge and technologies.

The works of Vechkinzova E. (Vechkinzova E, 2007, 2008, 2009) highlight approaches to the formation of a methodology for assessing the effectiveness of evaluating state programs and projects in Kazakhstan. The main problems of effective and efficient implementation of a number of programs were identified:

• incompletely and vaguely defined goals, insufficient developments of tasks and event of programs;
• in some programs, set tasks do not correspond to their content;
• weak analysis of the external environment, foreign experience, legislative base or their absence;
• ambiguous identification of the target audience and stakeholders of programs;
• undeveloped monitoring system of the implementation of the program or its absence;
• the lack of a clear system of indicators of efficiency and effectiveness for each task;
• lack of a system of calculating indicators of the value of implementation in some programs.

Bopieva Zh.K. et al. (Bopieva Zh.K. et al., 2009) provide an example of the major mistakes in the development of programs and projects in Kazakhstan, which did not allow achieving their goals and reduced the socio-economic effect of the implementation of state programs and projects. The authors have developed recommendations of improving the tools for managing programs and projects at the state level.

Pūlmanis E. (Pūlmanis E., 2013) examined in detail the current problems of the efficiency of public project management in Latvia. The main recommendations of the study are: to increase the capacity and level of professional skills of local municipal personnel in project management, to determine the appropriate organizational structure for the development and implementation of the project, the tools, and methods of project management (they should be in a mandatory requirement in large-scale public sector projects).

Gianelle C. et al. (Gianelle C. et al., 2020) explores the implementation practice and results of a large European smart specialization project. At the time of the formation of the problem of smart specialization, it was a direction focused on the concentration and specialization of innovation and research activities in such a way as to complement and develop processes in the innovation and entrepreneurial spheres. In the process of its development, the concept of smart specialization began to spread to more and more in industrial market.

Under the European Union's Innovation Union (RIS3) program, policymakers had to focus on supporting the identified priority activities throughout the study period. At the end of such a period, further support is received by those who have demonstrated sufficient potential for development. As on example, the Russian system of National Projects and the “basement of projects” (a pool of projects that can be accepted for implementation if certain conditions change) can be represented.

Barzotto M. et al. (Barzotto M. et al., 2019), analyzing the results of the project, suggest that smart specialization (smart specialization strategies were at the heart of the RIS3 program) can prevent any downturns in the development of the region. This is due to the smoothing of the trajectory of the region's development through many potential growth paths across a range of priority activities.

Zozulya A.V. et al. (Zozulya A.V. et al., 2019) consider the evolution of national programs and management projects in Russia. The authors proposed recommendations for improving the process of managing national projects and concluded that it is necessary to train management personnel with knowledge in the field of project management.

Methods

For studying the experience of managing current national projects in Russia, the authors used such methods as:

• study of the legislative framework for the development and implementation of national projects;
• methods of comparative analysis of the mechanism of selection and monitoring the implementation of national projects in the context of the regions of Russia;
• analysis of publications in the media;
• analysis of assessment judgments of experts on the experience of monitoring and implementing existing national projects of Russia.

Results

During the development of the Russian Federation after the collapse of the Soviet Union, the topic of national projects took on various outlines and organizational forms of management. The state constantly tried to find the best forms of investment in large-scale projects and attract private investors to these projects:

• since 2002 the preparation and adoption of federal target programs have begun;
• in 2005, the launch of four priority national projects “Affordable and Comfortable Housing and Utility Services for Russian Citizens”, “Development of the Agro-Industrial Complex”, “Health” and “Education” were announced;
• in 2010, Federal Target Programs were transformed into state programs due to the failure to meet the target indicators;
• in 2012, 11 “May” decrees of the President of Russia were signed, aimed at priority areas of state development and defining targets that were required to be achieved by 2018 or 2020;
• in 2016, there was a reorganization in the field of strategic project management in the form of the creation of the Council for Strategic Development and Priority Projects, the Department of Project Activities of the Government Office of the Russian Federation and the abolition of two other advisory bodies. The formed list of 11 main directions of the strategic development of the Russian Federation for the period up to 2018 and 2025 served as the basis for the preparation of 29 priority projects. A number of previously adopted state programs were also transferred to project management.

Currently in the Russian Federation, a system of national projects, which was formulated by the President of V.V. Putin in 2018, is being used. The main approaches to the development of project activities and the implementation of national projects as follows:

• Consolidation. Immersing priority projects in national projects (that is, national projects are larger in scale of priority projects);
• Saving the governance structure: Council, Council Presidium, Project Committees, and other mechanisms that have shown efficiency;
• Planning. Clarifying target indicators for years, flexible 3-year plan (rolling planning, synchronized with the budget), ensuring the decomposition and communication of all management objects;
• Simplification and de-bureaucratization. Simplifying regulatory procedures and use of an electronic reporting format, development of a monitoring and decision-making system, change management;
• Projects in ministries. Including project activities in the current work of ministries, ensure the personal responsibility of ministers;
• Training and motivation. Organizing training for the main participants, form a bonus fund and an incentive system.

The main transformations of state and sectoral programs and projects into a portfolio of national projects:

1. Formation of a system of goal-setting of national projects — focus on ensuring breakthrough results.
2. Definition of national projects as a portfolio of federal (priority) projects.
3. Transformation of state programs affecting the implementation of national projects (programs) into “pilot” ones.
4. Ensuring the segregation of expenditures for national projects in the budget classification.
5. Formation of a single information resource, allowing to ensure, in compliance with the principle of one-time data entry, the availability of complete information on the parameters of national projects, federal projects, and government programs in real time.

In particular, the work on national projects involves the creation of integrated action plans in 13 main areas (see Fig. 1). To date, the assessment of all the projects is submitted and the most significant projects entered the system of national projects. The public selection methodic allows to evaluate and rank projects that are not included in the National Projects System. So, a queue is formed from projects (or the so-called “basement of projects”), which can be implemented in the future.

The most important property of the National Projects System is its dynamism. To do this, monitoring will be regularly used and in case of deviation or disaccomplishment of the planned indicators, some projects can be excluded from the National Project System (the process marked in Fig. 1 — red arrow directional down). Next step, the projects out of “basement” turn into National Projects System. This is possible, both in the case of the exclusion of some projects until 2024, and continuation of development through the National Projects after 2024 (the process marked in Fig. 1 — green arrow directed up).

The collection into a single assessment and monitoring system of a large number of various projects that are in the queue to enter the system of National Projects makes it possible to launch the rule of a scarce resource, which is used more efficiently with less resource. Federal budgetary funds act as a scarce resource. This is the exact opposite of the “spend by all means” principle, when a guaranteed amount of federal funding is allocated, which must be spent in full, otherwise funding will be cut. The main negative consequence of the development of money “at any cost” and the state order system based on this principle are “unfinished (incomplete objects)”, the purchase and creation “reserve” of what, in essence, is not needed. Therefore, the key principle for the selection of projects in the system of National Projects can be emphasized, which was formulated by the Head of the Udmurt Republic A. Brechalov — to understand “why and for whom” this project is being formed.

Many researchers note that in order to achieve the great goals of the National Projects, non-standard approaches are needed. However, an analysis of practice illustrates that there are problems of coordinating mutual interests and developing mechanisms with the help of which this could be done. Indeed, in the 2000s in Russia, the priority of national goals has developed, when any interests of lower levels of government (regional and municipal) were subordinated to solve problems at the national level. On the one hand, this helped to overcome the crisis in Russia, to quickly build up the economic potential of the state. On the other hand, it led to a weakening of incentives and the leveling of factors of regional and local development. The consequence was an increase in intra- and interregional imbalances in socio-economic development.

To solve the problems of extremely rapid growth and development, it is necessary to bring together the efforts of the entire society. For this, it is essential to learn how to coordinate and take into account the interests of all parties. Thus, there are two possible ways to solve the tasks of the National Projects. The first is large-scale and rapid growth due to the large projects based on national industry players, but without considering regional development. That is, a continuation of the already established practice. The second is painstaking work that includes all citizens and territories in the process of socio-economic development through National Projects. The consequence of this will be the distribution of all positive effects and the real delivery of growth and prosperity to all regions. Therefore, the following question arises: how to involve broad strata

of the population, small and medium-sized businesses in the process of implementing National Projects? In this regard, the answer to the question “why and for whom the project is being implemented” facilitates to refract the goals of national projects to a lower level of management and link them with the potential of the region.

It may seem that considering the interests of the population and wide public involvement is unprofitable, ineffective since this leads to a reduction in the profitability of the largest projects. However, development at the level of the population and local small economies is necessary as a basis that can provide large- scale impulses for development. In the absence of such development at a lower level, all major projects will turn out to be unrealizable and limited in their growth. Therefore, the development and achievement of the goals of the National Projects with the wide involvement of the population and all territories of the country is the most effective and sustainable.

To introduce the interests of local communities into the National Projects, the supporting work of the regional authorities is mandatory. The main tasks of the region in this case are to complete work on regional legislation, which will allow to implement quickly and legally the initiatives of local and regional communities, as well as work with the federal authorities to ensure support for the implementation of projects. The essence of the work of regional administrations is the coordination of mutual tasks and interests of the region, the federal government and local beneficiaries. As an example of such work, we can cite a project from the Novgorod region, when a cooperative of the highest-level seed potato producers were created. Сoopera- tive, with the support of regional authorities, was able to obtain funding from the Ministry of Agriculture for the construction of a processing complex. This complex would never have arisen if the regional authorities did not seek to find a solution and defend local interests in the federal ministry.

To include all regions in the implementation of National Projects, the federal authorities abandoned the declarative principle of receiving investments for the regions, which can be expressed by the statement: “Give our region federal investment, and we will attract private investment”. The example of the Novgorod region shows that the principles have changed. If there are responsible executors and interested parties, regional or private investments, it is possible to attract additional federal investment. That is, it is possible for the regions to form the perspective of their development through the system of National Projects. For this, all ministries are conducting extensive consultations. An open methodology for evaluating projects allows regions to assess the position of the project and it is easy to understand on what aspects needs to strengthen the project. The initiative of the regional authorities in attracting extra-budgetary funds is also stimulated through the mandatory criterion of the ratio of attracted and budgetary funds, which is an important element of the project evaluation methodology. Thus, the regions can withdraw their projects, which have not yet been included in the National Projects System, first in line in the “basement of projects” and ensure their implementation even before 2024 (if other projects leave the system).

At the federal level, the basis for the adoption by the population of all National Projects is the linking of their goals and strategic plans of the regions (with the formation of development opportunities for all municipalities). It is important to coordinate the goals of the National Projects with the Spatial Development Strategy, the Program for the Comprehensive Development of the Transport Infrastructure of the Regions, and the Program for the Industrial and Technological Development of the Country and Regions.

The experience of exchange and distribution of successful projects from different regions is of great importance for the efficient and rapid achievement of the goals of the National Projects. In Russia, the Agency for Strategic Initiatives (ASI) acts as an accumulator and disseminator of best practices (there is a “Store of Right Decisions” or Smartek). For the interregional broadcasting of the best practices, the keys are the so- called “box solutions”, well-thought-out, legislatively grounded decisions that have been implemented in a certain region. Here, it should be paid attention to the fact that for the successful implementation of practices from another region, it is of great importance to take into account interregional differences in socioeconomic conditions. From this the following question arises: why and for whom will the project be implemented in the new region? For an adequate answer in the regions, serious information and analytical work with territories and municipalities are required. In this sense, we agree with the opinion that the box solution is a simplified solution. The paramount task that appears when deciding to implement a project from another region is to compare the conditions for implementation, the promising results of the project that may turn out in the new territory. For such work, ASI and other agents may be involved. However, structural subdivisions of the regional executive power — economic and forecasting departments, scientific organizations specializing in regional development — acquire a much greater role in this process. The structural subdivisions of the executive power of the regions have much more complete and specific information, systemic knowledge about the region. It is obvious that at the moment in Russia, information about the territories is poor and does not reflect the full completeness of socio-economic conditions. Therefore, for correct work towards achieving the goals of the National Projects, it is necessary to improve the system of regional and territorial information.

It is this kind of work on adapting projects to different regions that will allow the “Store of Right Decisions” to turn from a library into a part of the management system. With an understanding of the most likely consequences during the implementation of activities, it is possible to position and structure projects in different ways. In one case, an event can become a driver due to the benefits and profits of which it will be possible to solve other less profitable, but socially significant events. In another region, it may turn out that a driver is needed to implement the same measure. In order to understand which activities in which region can become a driver and ensure the implementation of other less profitable activities, a large costly preinvestment stage is required, which is associated with an understanding of the conditions in which projects and activities are being implemented. That is, a combination of various activities in one project can provide a flow of funds and thereby increase the feasibility of projects in various socio-economic conditions, ensuring the introduction of previously unavailable technologies and solutions.

This work is the basis for the development of mechanisms for the reuse (replication) of projects. It should primarily focus on considering regional characteristics that can greatly increase or decrease the benefits when transferring successful projects from other regions. This actualization is an important factor in scaling and replicating successful experiences.

Discussion

The importance of understanding the mechanisms of selection, monitoring and reuse of projects is dictated by the fact that now Russia faces the prospect of creating a large complex of digitalization infrastructure (systems of fiber-optic cables and cellular base stations for universal access to broadband Internet, systems of data storage and processing centers, for the operation of large-scale information systems such as State Services). This implies a colossal investment. The main fears are that huge investments will be paid off by economic growth and not become another element of the social environment that should be provided by the state, but, at the same time, it is not involved in economic and production-economic processes. That is, as in the situation with roads, which on the periphery of many regions are used only for the needs of the population (and not for business development), since there are no industries and economic activities near this population. Maintaining the required condition of roads is a social burden, but it does not allow stimulating economic growth. Awareness and consideration of the influence of the specific conditions of the regions in the implementation of any projects is a paramount condition for the involvement of large economic potential (population, infrastructure) in economic activities. On this basis, the implementation of a qualitative leap forward in development is the main goal of the National Projects.

Among the problems, it should be noted that there is a gap in opinions about what is social and economic development. The heads of the regions most often pay attention to the importance of the formation of the working environment as the most important element of socio-economic development. Such projects are found in many regions, thus, this issue is relevant for the population. On the other hand, the federal leadership sees the basis of socio-economic development more in social projects, in the sense that the priority for the population is high-quality services in education, housing and public utilities, and healthcare.

Conclusions

Summarizing the studied experience of Russia in the transition from state programs to national projects, we can single out a fundamentally new approach to the mechanism of dynamic monitoring and updating of projects, when projects with low implementation efficiency are excluded from the composition of national projects. In their place, from the “basement of projects” projects that have passed the examination and have a high assessment of the quality of their development came. Despite this good practice in the management of national projects, the following problem areas can be identified:

1. Systemic management problems.

The transition of the public administration system from a planned mechanism to a market model was spontaneous. Therefore, for the successful implementation of national projects, an understanding of how this institution should interact with other established institutions of management, how to “build” it into existing management ties, is required. A separate aspect of this problem is the unpreparedness of state management personnel at all levels to work on projects of great social importance.

Low-quality planning in the formation of the system of indicators of projects.

The problem of the low-quality development of indicators for national projects is of a methodological nature and, if there is a systematic approach, it can be easily corrected.

Motivational problems.

The success of the implementation of projects on a national scale is directly determined by how they are perceived by the main groups of actors: national elite, population, and business. The specificity of the situation in Russia is that the intellectual elite in the country is poorly represented, the business elite dominates, which does not associate itself and its interests with the country's development. This is precisely what the process of the brain and capital drain abroad demonstrates. In this regard, the motivation of business, as an integral and one of the key participants, which, in turn, depends on the investment attractiveness of national projects, is of great importance in the success of the implementation of national projects.

Thus, the considered Russian experience in the formation of a mechanism for selection, monitoring and replication of national projects will allow the experts and analysts of Kazakhstan to take into account the mistakes existing in practice and develop optimal tools for an effective Kazakhstani system for managing national projects.

References

1. Aleksandrova, A.L., Belyakov, I.V., Nikonova, L.S., & Chagin, K.G. (2005). Monitoring sotsialnykh programm: prak- ticheskie primery [Monitoring Social Programs: Case Studies]. — Moscow: Fond «Institut ekonomiki goroda» — Institute for Urban Economics Foundation, 112, ISBN: 5–8130–0092–5 [in Russian].
2. Barzotto, M., Corradini, C., M Fai, F., Labory, S., & R Tomlinson, P. (2019). Enhancing innovative capabilities in lagging regions: an extra-regional collaborative approach to RIS3. Cambridge Journal of Regions, Economy and Society, 12 (2), 213–232.
3. Bopieva, Zh.K., & Kazakova, T.P. (2009). Kazakhstanskii opyt otsenki effektivnosti gosudarstvennykh programm [Kazakhstan experience in assessing the effectiveness of government programs]. Vestnik Instituta zakonodatelstva Respubliki Kazakhstan — Bulletin of the Institute of Legislation of the Republic of Kazakhstan, 4 (12), 20–27 [in Russian].
4. Conteh, C. (2013). Changing Trends in Regional Economic Development Policy Governance: The Case of Northern Ontario, Canada. International Journal of Urban and Regional Research, 37, 4, 1419–1437.
5. Feinstein, O. & Ingram, G. (2003). ‘Lessons Learned from World Bank Experiences in Joint Evaluation’, Room Document submitted to the Evaluation Network, OECD, Paris, France, Retrieved from: http://www.oecd.org/dataoecd/secure/15/13/31736431.pdf.
6. Gills, B.K. (2001). Re-orienting the New (International) Political Economy. New Political Economy, 6 (2), 233–245.
7. Handbook on planning, monitoring and evaluating for development results (2009). United Nations Development Programme. One United Nations Plaza New York, NY 10017, USA Handbook. Retrieved from: http://web.undp.org/evaluation/handbook/documents/english/pme-handbook.pdf
8. Independent Evaluation Group (2007). Sourcebook for Evaluating Global and Regional Partnership Programs: Indicative Principles and Standards. Washington, DC: World Bank. Retrieved from: https://openknowledge.worldbank.org/handle/10986/6601 License: CC BY 3.0 IGO.”
9. Luchshie praktiki sotsialno-ekonomicheskogo razvitiia. Kompleksnye resheniia dlia regionov i gorodov [Best practices for socio-economic development. complex solutions for regions and cities]. Sait Rossiiskogo investitsionnogo fo- ruma – 2019, Roskongress – Website Russian Investment Forum 2019, Roscongress. www. rusinvestforum.org. Retrieved from https://rusinvestforum.org/programme) [in Russian].
10. Magazin vernykh reshenii [Store of the right decisions]. Sait Agentstva strategicheskikh initsiativ [Website Agency for Strategic Initiatives]. www. asi.ru. Retrieved from https://asi.ru/store/ [in Russian].
11. Natsionalnye infrastrukturnye prioritety – 2024: bystro opredelit i effektivno realizovat [National Infrastructure Priorities — 2024: Quickly Define and Effectively Implement]. Sait Rossiiskii investitsionnyi forum — 2019. Roskon- gress – Website Russian Investment Forum 2019, Roscongress. www. rusinvestforum.org. Retrieved from https://rusinvestforum.org/programme [in Russian].
12. OCHA (2007). ‘Guidelines: Results-Oriented Planning & Monitoring’, 11.
13. OECD (1991). ‘DAC Principles for Evaluation of Development Assistance’, Development Assistance Committee (DAC), Paris, France, Retrieved from: http://www.oecd.org/dataoecd/21/32/ 41029845.pdf.
14. OECD (1995). Governance in Transition: Public Management Reforms in OECD Countries. OECD, Paris.
15. Pūlmanis, E. (2013). Public sector project management efficiency problems, case of Latvia. Regional Formation and Development Studies: Journal of Social Science, 3, 177–188, ISSN 2029–9370.
16. Roslyakova, N.A. Voprosy stimulirovaniia territorialnogo razvitiia [N.A. Issues of stimulating territorial development]. Sait Roslyakova 24. ru [Website Roslyakova24.ru]. www.roslyakova24.ru. Retrieved from http://roslyakova24. ru/load/stati/voprosy-stimulirovanija-territorialnogo-razvitija/1–1-0–42 [in Russian].
17. Sander, C. (1997). Planning Monitoring and Evaluation of Programme Performance. A Resource Book. Ontario: IDRC Evaluation Unit. Retrieved from: http://www.idrc.ca/uploads/user-S/10504282450reach_e.pdf
18. Soderlund, J. (2004). Building theories of project management: past research, questions for the future. International Journal of Project Management, 22, 183–191.
19. Sprague, R.H. & Carlson, E.D. (1982). Building Effective Decision Support Systems, Englewood Cliffs, New Jersey: Prentice-Hall, p. 314.
20. UNDG (2007). ‘CCA/UNDAF Monitoring and Evaluation Guidelines’, Retrieved from: http://www.undg.org/ in- dex.cfm? P=259
21. Van Gundy, A.B.Jr. (1988). Techniques of Structured Problem Solving. 2nd ed. New York: Van Nostrand, R, 286.
22. Vechkinzova, E.A. (2007). Mezhdunarodnye podkhody k otsenke gosudarstvennogo upravleniia v Kazakhstane [International approaches to assessing public administration in Kazakhstan] // AlPari — AlPari, 3, 4 (51, 52), 9–18 [in Russian].
23. Vechkinzova, E.A. (2007). Otsenka yeffektivnosti gosudarstvennykh programm razvitiia: poniatie, printsipy, kriterii [Evaluation of the effectiveness of state development programs: concept, principles, criteria]. Vestnik regionalnogo razvitiia — Regional Development Bulletin, 4 (14), 138–146 [in Russian].
24. Vechkinzova, E.A. (2008). Opyt otsenki gosudarstvennykh programm razvitiia v Respublike Kazakhstan [Experience in assessing state development programs in the Republic of Kazakhstan]. Izvestiia Uralskogo gosudarstvennogo yekonomicheskogo universiteta — Bulletin of the Ural State University of Economics, 1 (20), 105–112 [in Russian].
25. Vechkinzova, E.A. (2009). Analiz mezhdunarodnoi praktiki otsenki gosudarstvennykh programm razvitiia [Analysis of international practice in assessing state development programs]. Vestnik regionalnogo razvitiia — Regional Development Bulletin, 1 (19), 90–98 [in Russian].
26. Vechkinzova, E.A. (2009). Institutsionalnye osnovy otsenki effektivnosti gosudarstvennykh, otraslevykh i regionalnykh programm v Respublike Kazakhstan [Institutional framework for assessing the effectiveness of state, sectoral and regional programs in the Republic of Kazakhstan]. Izvestiia Uralskogo gosudarstvennogo yekonomicheskogo uni- versiteta — Proceedings of the Ural State University of Economics, 4 (26),152–157 [in Russian].
27. Vetrov, G.Yu., Vizgalov, D.V., Pinegina, M.V., & Shevyrova, N.I. (2003). Otsenka munitsipalnykh programm [Evaluation of Municipal Programs]. — Moscow: Fond «Institut yekonomiki goroda» — Institute for Urban Economics Foundation, 2003, 88. ISBN: 5–8130–0067–4 [in Russian].
28. Zozulya, A.V., Zozulya, P.V., & Yeremina, T.N. (2019). Sovremennye problemy realizatsii prioritetnykh natsionalnykh proektov [Modern problems of implementation of priority national projects]. Vestnik Yevraziiskoi nauki — Bulletin of Eurasian Science, 1, Sait Vestnikа Yevraziiskoi nauki — Website Bulletin of Eurasian Science. Retrieved from https://esj.today/PDF/21ECVN119.pdf [in Russian].