The diagnostic assessment of 21st century skills acceptable for introduction in the educational process within the framework of teaching English

In conditions of a rapidly growing economy and the information society, it has become necessary to prepare specialists with broader professional competencies that are 21^st century skills. This article discusses the main peculiarities of 21^st century skills diagnostic assessment revealed as a result of the analysis of various authors' scientific works. Taking into consideration the distinguished peculiarities of the diagnostic assessment of these skills, the author of the article has developed a diagnostic test and assessment criteria to determine a level of the formation of the most important 21^st century skills that are acceptable for implementation in the educational process within the framework of teaching English at higher education institution. As a result of the experimental testing of the developed diagnostic test and assessment criteria with the participation of the students of M. Kozybaev NKSU, the main disadvantages of training future qualified English teachers have been uncovered. The results of this study are acceptable for use in further scientific works when developing 21^st century skills teaching methods and assessment criteria and integrating them into the higher education system of Kazakhstan that will contribute to providing the country with highly qualified workforce in the foreseeable future.

Introduction

Developments in society and economy, rapid production and technological growth are posing increasingly new and demanding challenges not only to young people, but also to human society on the whole. Educational systems are rethinking the knowledge, skills and competencies to equip young people with them that would allow students around the world to benefit from the emerging new forms of socialization and to actively contribute to economic growth under the system where the paramount investment is knowledge and skills. The 21^st century establishes new requirements for the future specialists and adds a new urgency to develop the knowledge and skills in which students need to attain success in conditions of the world globalization.

Nowadays, those skills, which enable students to generate and introduce new ideas and products, to use digital tools for scientific discoveries and communications, to apply the knowledge to solve pressing problems, are in large demand [1; 4]. Likewise, the current workforce needs to be flexible and adaptable in order to succeed in different professional fields. For that reason, an extended set of competencies must provide an opportunity to choose the direction of the personal and professional development. Educators and governments, employers and researchers name such set of competencies by various terms that are «21^st century skills», «higher-order thinking skills», «deeper learning outcomes» and «complex thinking and communication skills» [2; 2].

The need for defining such competencies and finding the methods for teaching deeper learning skills to prepare students for their future job was appeared many years ago. For more than 40 years, researchers at Harvard University's Project Zero have been studying how students learn and how to teach effectively these skills. In 1981, the National Commission on Excellence in Education was created by the US Secretary of Education to evaluate the quality of education in the United States. A discovery was that «educational reform should focus on the goal of creating a Learning Society» [3; 26]. To date, a set of international organizations specialized in research on 21^st century skills, such as Assessment and Teaching of 21^st Century Skills, Asia Society Partnership for Global Learning, Organization for Economic Co-operation and Development, The Partnership for 21^st Century Learning, have developed classifications, teaching methodology and key aspects of the 21^st century skills assessment. However, our analysis of theoretical sources showed that determining a level of the formation of 21^st century skills is still a poorly studied issue, especially in the process of teaching English as a foreign language, and requires more in-depth study and generalization.

Due to the fact that in Kazakhstan's educational system, there is actually no study of the world's best practices in integrating 21^st century skills into the educational process, a research in the field of determining a level of the formation of 21^st century skills that are acceptable for further instruction of students in the English language classroom seems innovative and relevant.

The purpose of this article is to develop a diagnostic test to identify a level of the formation of 21^st century skills and assessment criteria that would be acceptable for introduction in the educational process as a part of teaching English as a foreign language. To achieve the goal, it is necessary to complete the following tasks: to highlight those skills from the classification of 21^st century skills that are the most important and acceptable for teaching students in the English language classroom; to study and analyze various authors' approaches to assessing 21^st century skills; on the basis of the results of analysis and generalization of the presented principles and approaches, to develop a variant of the diagnostic assessment and criteria to determine a level of the formation of 21^st century skills that are acceptable for teaching students in the English language classroom; to conduct an experimental testing of the developed diagnostic test to determine students' level of the formation of 21^st century skills in the English language classroom at higher education institution; to analyze the results of the experiment and draw conclusions.

Experimental

In our article called «The classification of 21^st century skills which are acceptable for introduction in the educational process within the framework of teaching English», we have developed the classification of 21^st century skills, acceptable for introduction in the learning process in the English language classroom [4; 90]. The following 21^st century skills were included in this classification:

Learning and Innovation Skills:

• Creativity and Innovation
• Critical Thinking and Problem Solving
• Communication
• Collaboration

Digital literacy Skills:

• Information Literacy
• Media Literacy
• ICT (Information, Communications and Technology) Literacy

Life and Career Skills:

• Social and Cross-Cultural Skills
• Leadership and Responsibility

In the classification above, the most valuable skills for ensuring successful work in modern society are Learning and Innovation Skills, such as Creativity and Innovation, Critical Thinking and Problem Solving, Communication, and Collaboration become known as the «Four Cs». The significance of the «Four Cs» for students' development was confirmed by the survey conducted by American Management Association (AMA) in 2012. 768 managers and other executives were asked about the importance of the «Four Cs» to their organizations. According to this survey, managers and executives stated the «Four Cs» skills and competencies have been articulated within their organizations as priorities for employee development, talent management, and succession planning. The majority also agreed that their employees are measured in these skills during annual performance appraisals. Three out of four (74.6 %) managers and executives, who responded to the AMA survey, considered the skills and competencies of learning and innovation as ones of the most important to their organizations in the next three to five years [5; 3].

Due to the rather large number of the skills in our classification, we will consider only one group of 21^st century skills in our research work. The Learning and Innovation skills have a noticeable interrelationship and together contribute to more effective solution to a problem in the implementation of professional activities. For instance, developing the Critical Thinking and Problem Solving skill, students need to find more acceptable solution to the problem in given conditions, for which they have to resort to creative thinking. At the same time, in order to solve the problem in the most effective way, a set of creative ideas is needed that can be proposed by team members. In other words, collaborating with each other, students can identify one of the most relevant solutions out of all those proposed. Likewise, without developed communication skills, it is difficult to provide productive cooperation. Thus, the Learning and Innovation skills should be taught in the complex.

Based on the results of the AMA survey and our analysis above, one can conclude that the Learning and Innovation skills are the most acceptable for students' development in the English language classroom due to their importance to employers in the conditions of rapidly growing globalization.

Before teaching Learning and Innovation skills, it is important to carry out a diagnostic test that helps a teacher and learners identify problems that they have with the language and the development of 21^st century skills. Teachers can use a diagnostic assessment to design more rational lesson plans and provide differentiated instruction to meet students' needs and attain pedagogical goals, including the achievement of higher level of the Learning and Innovation skills formation. The analysis of what skills are more developed and how much students know about 21^st century skills and are able to apply their prior language knowledge and skills in real-world situations is one of the most indispensable steps to construct effective instruction. This test enables to identify areas and skills that need more attention to bridge gaps during further learning of the «Four Cs». As an example, at the beginning of teaching Learning and Innovation skills in the English language classroom, a teacher may give a pre-test to determine if the class has some of the skills developed better than other ones. The students' responses will help the teacher define how much time should be dedicated to training certain skills. The teacher may learn from this diagnostic test that many students already have sufficient level of Communication skills, but know little about how to think critically and tackle problems efficiently. From this, the teacher may adjust lesson plans to spend a bit more time on the improvement of students' Critical thinking and Problem solving skill and slightly less on their communicative skills.

The diagnostic assessment of 21^st century skills has a set of peculiarities that should be taken into consideration when devising a diagnostic test. Bernie Trilling and Charles Fadel, who have been researching the issue of teaching and assessing 21^st century skills for many years, highlighted two key tools that are «the most powerful learning tools ever devised: questions and the process to uncover their answers; problems and the inventing of their possible solutions» [6; 90]. Learning through solving problems and answering questions is still one of the most effective scientific method to develop and direct human thinking towards exploring and uncovering new knowledge in any area. It is noteworthy that the same authors also distinguish two approaches to finding solutions and answers to problems and questions respectively that are science and engineering. In their view, scientists make experiments to test a hypothesis and, in turn, engineering enables specialists to design prototypes to confirm or reject the effectiveness of the found solution. On the basis of both scientific and engineering methods, it was determined that there are two basic types of 21^st century approaches to learning: inquiry-based learning that is the learning method based on the power of questions and design-based learning that makes the design of solutions to problems possible. Both of these learning methods, combined with conventional ways of acquiring English language knowledge and skills, and leveraging digital learning tools, are the foundation for the effective learning of 21^st century skills, in particular, to determine students' level of the formation of Learning and Innovation skills [6; 92].

Another key point in building a diagnostic test to assess students' higher order thinking skills is a format of the diagnostic assessment. It has long been known that the most widespread and reliable format of an assessment is a standardized multiple-choice test. Most educators today are able to design high-quality multiple-choice tests. Even the international test administered by the Organization for Economic Co-operation and Development (OECD), called the Programme for International Student Assessment (PISA), relies in part on multiple-choice items to compare achievement in reading, mathematics, and science literacy across countries. Beyond large-scale accountability measures, multiple-choice tests are currently being applied for a variety of purposes, including assessing some 21^st century competences. For instance, the California Critical Thinking Skills Test (CCTST) has been translated into various languages and uses computer-based administration. Although there are many tasks on the analysis of reading passages, charts, pictures, paintings and drawing inferences from this, nevertheless, most of the questions included in the test are multiple-choice. As a result of the assessment of analysis, evaluation, inference, deduction, induction, and overall reasoning, each student is provided for appropriate scale scores [7; 18]. Another example of the use of the multiple-choice format to measure higher-order thinking skills is the Mission Skills Assessment (MSA) that tests students' collaboration skills by asking them about the most likely actions if they had problems with a deadline for an important group project and a group member who was not willing to contribute [7; 19]. These examples of 21^st century skills assessment demonstrate the attempt to refine the traditional principles of tests design by striving to place students in a more real-world environment where the right answer is not always evident.

Apart from the real-world environment approach to an assessment in the 21^st century, there is a need for more dynamic and less rote multiple-choice tests to enhance students' flexibility in their response to a question. In this direction, some of the international assessment organizations are already researching the issue how to modernize large-scale tests. Such scientific groups are the Partnership for Assessment of Readiness for College and Careers (PARCC) and the Smarter Balanced Assessment Consortium (Smarter Balanced) which are drawing to a large extent on formulating multiple-choice questions with highlighting students' critical thinking. By way of example, a test may include the task where test takers are asked to select two quotes from the suggested list that are most likely to be displays of students' contention about a reading passage [8; 235].

Despite the widespread use of multiple-choice assessment owing to inexpensive administration, beneficial time consumption, high degree of reliability, possibility to use for a variety of purposes, in particular to measure the Learning and Innovation skills, the lack of response flexibility does not enable to elicit complete picture of students' acquisition of 21^st century competences. Other types of assessments can assist to address such limitations, struggling with the inherent lower degree of reliability and validity.

The next format acceptable for the assessment of 21^st century skills is a subset of multiple-choice tests with self-report items. In the surveys with these items, test takers are asked to reflect themselves and rate their actions in certain cases selecting more likely answer along a scale, such as «very easy» to «very hard» and «strongly agree» to «strongly disagree». One of such multiple-choice assessments with self-report items is the College and Career Ready School Diagnostic developed by David Conley to measure several 21^st century competences. This diagnostic test comprises accurate statements like «I take complete, organized, and accurate notes during class» and «I talk to adults when I have concerns about school» which request students' attitude along the scale «strongly agree» to «strongly disagree» [7; 24]. Self-report items enable test takers to take into consideration past experience of tackling complex problems and working in a team that yields positive outcomes, such as self-reflection and, therefore, the growth of students' motivation to develop their higher-order thinking skills. Nonetheless, responses to self-report questions cannot reflect reality when students select answers that are more preferable to them but are deceptive in order to have a good grade or improve the teacher's attitude.

One more format in our consideration, that provide more flexibility and transparency in test takers' responses, is an open test which includes open-response questions. These questions elicit written products, generally in the form of a few written sentences, a paragraph, or even a full essay, which may be particularly well suited to assessments of non-routine problem solving or creativity. One example is the Formulating Hypotheses test, a measure of creativity developed by Educational Testing Services, where a question presents to students a prompt, for example a graph illustrating the declining rate of death from infectious illness in a developing country. Students, in turn, should list as many potential causes of the reduction as possible in a finite amount of time [7; 25]. However, as all assessment formats, an open-response test has its specific disadvantage, namely difficulty in generating reliable scores, that can be addressed by providing explicit scoring criteria, training raters on those criteria, and blending open-response and multiple-choice items.

On the basis of all considered above, it is important to point out that, in order to overcome the difficulties in building reliable and valid diagnostic test to measure students' level of the formation of 21^st century skills, it is indispensable to blend different testing items. To assess the Learning and Innovation skills, such as Creativity and Innovation, Critical thinking and Problem solving, Collaboration, Communication, it is rational to apply both multiple-choice and open-response questions that were operated while developing the diagnostic test for determining Kazakhstani students' level of the formation of the «Four Cs».

The diagnostic test suggested by us encompasses four sections that correspond to four 21^st century skills, such as Creativity and Innovation, Critical thinking and Problem solving, Collaboration, Communication. Each section includes three multiple-choice questions with self-report items and one open-response task. The first three questions are focused on students' past experience, self-reflection and rating their past actions along the following scales: «very easy» to «very hard», «often» to «rarely/extremely rare»,

As it can be noted, in all questions above, students are asked to select more accurate answer that is emphasized by the words «always» or «often» in the questions themselves. Furthermore, almost every variant of response comprises more concrete statement that helps the test takers reduce their hesitations while selecting most suitable answer to them. Overall, there are three options in each question, but the answer selected can be only one. More options could be an obstacle on the way of picking out the variant most suited to the students. In addition, an important point is that all these questions reflect the real-world situations with what any person faces in various cases. It provides definite proof that one of the main principles of 21^st century skills assessments was accommodated in the multiple-choice questions of the diagnostic test presented in this article.

The fourth question is a task which requests open response so that the assessment of the Learning and Innovation skills can be more objective in conjunction with the explicit criteria. These tasks in each section are directed to elicit a writing product as a result of the students' individual work that can assist to measure the appropriate 21^st century skill. In the Creativity and Innovation section, the task is based on the development of creative and imaginative thinking which can include drawing pictures as in our diagnostic test (Figure 1):

It is important to note that this task comprises several subtasks which are interrelated and interdependent. The complexity of the task lies in making a sentence with the names of all three pictures. In order to accomplish the last subtask, the student has to return to the first one, namely drawing pictures, and think about how to connect the different pictures and their names so that it can yield a sentence with a sense. The completion of this task requires the elaboration of every step and high level of creativity that proves the validity of the assessment, in other words, this task definitely measures students' level of the formation of Creativity and Innovation in the English language classroom.

The next example of the diagnostic assessment of the «Four Cs» under review is the task on the development of the Critical thinking and Problem solving skill from the section 4 of our diagnostic test (Figure 2):

As can be seen, in this task, there is a table with the data that should be analyzed before responding to the questions. Students should present several reasons for the problem revealed from the result of the analysis and find the most effective solution to the problem that includes in the Critical thinking and Problem solving competence. Thus, we can safely assume that the «Problems and Questions» principle of an assessment in the 21^st century is accomplished in the diagnostic test developed by us for English learners as well as the real-world environment approach.

In order to provide reliability of an assessment, it is vital to design explicit criteria. We have suggested the following criteria, for example, to the questions and the task in the Creativity and Innovation section (Table 2):

As can be noted from Table 2, all criteria are accurate and clear that contribute to the increase of the test reliability. The first three questions in each section have the criteria according to the answer selected by the test taker. If the student has chosen the variant A, B or C, the maximal score would be 10, 5 or 0 points respectively. As for the task, the criteria cover the quality and quantity evaluation of the students' responses. The quantity evaluation envisages the number of figures fully finished, pictures illustrating existing objects and named, the number of the names used in the sentence made by the student. The issue if the sentence makes sense is considered during the quality evaluation of the response. The accomplishment of each criterion for the task entails the appropriate score, maximum 14 points. Overall maximal score is 100 points that correspond to 100 %. Such percentage of the scoring makes the results of the Learning and Innovation skills assessment more evident.

On the basis of the results of our analysis of the diagnostic test and the criteria, one can draw an inference that the diagnostic assessment of 21^st century skills, such as Creativity and Innovation, Critical thinking and Problem solving, Collaboration, Communication, developed by us is reliable and valid in view of the integration of the main principles of the 21^st century skills assessment and the accuracy and explicitness of the criteria, and therefore, this diagnostic test is acceptable to introduce in the English language classroom at higher education institution.

The diagnostic survey was conducted at M. Kozybayev North Kazakhstan State University in 2019. The major function of this survey was to measure the students' level of the formation of 21^st century skills that would be acceptable for introduction in the educational process within the framework of teaching English at higher education institution. 34 fourth-year students, studying in 6B01701 «Foreign language: two foreign languages» specialty, were selected as a sample because of their high level of English language basic skills so that they could write clear responses to open questions in English. In addition, there were both Kazakh and Russian speaking students among the respondents. The survey worksheets were distributed to all students simultaneously. The respondents could spend 40 minutes to complete all sections of the test. Before the beginning of answering the test, the students were explained the purpose of the survey and presented the definition of 21^st century skills, the importance of these skills for their professional future, what skills would be assessed in that diagnostic test. The respondents voluntarily agreed to response to all questions asked in the survey.

Results and Discussion

For the convenience of analyzing the results of the survey, the maximal score 100 % was divided into five levels of the formation of the measured skill: very low level (0-20 %), low level (21-40 %), medium level (41-60 %), high level (61-80 %) and very high level (81-100 %). The data were analyzed by applying the quantitative interpretation that resulted in the number of students matching their scores and the five levels. The results of that analysis are presented in Table 3.

Table 3 Results of the experiment conducted at M. Kozybayev NKSU in 2019

In order to demonstrate the results of the survey graphically, the data were analyzed by the Excel chart data series software program (Figures 3, 4, 5 and 6).

Figure 4 indicates that the communicative skills of the students are amply developed, possibly due to the fact that, to date, the most widespread approach to teaching English is the communicative one. 27 % and 9 % of the respondents have high and very high levels respectively. Less than a quarter of the students (23 %) have low-level Communication skill. It is noteworthy that there is no respondent who has a very low level of this skill. The communicative skills of the remaining students (41 %) are at a medium level that proves the students' needs (64 %) in the development of this skill to a very high level.

From the analysis of Figure 6 it can be noted that the students' level of the formation of Critical thinking and Problem solving skill is mostly low (38 %) and very low (9 %). Only 9 % of the respondents have this skill at a high level. The medium level of Critical thinking and Problem solving skill was achieved by 44 % of the test takers. There is no student who reached very high level of the formation of this skill that indicates insufficient development of the higher-order thinking. Overall, at least 91 % of the students should improve their level of Critical thinking and Problem solving skill that is acceptable for teaching in the English language classroom.

Considering the entire analysis of the diagnostic test results, one can point out that the students have a lack of the development of Critical thinking and Problem solving and Collaboration skills. However, the students' level of other 21^st century skills, such as Creativity and Innovation and Communication, also requires the improvement and attainment of high-level creative thinking and communicative skills that is implementable in the English language classroom as well.

Conclusions

In conclusion, it should be noted that Learning and Innovation skills are the most important 21^st century skills to prepare future highly qualified workforce. One of the most indispensable procedures of teaching these skills is a diagnostic assessment that, regarding the determination of students' level of higher-order

thinking, has its own peculiarities which were taken into consideration in the design of the diagnostic test and assessment criteria presented in this article. As a result of the experimental testing of the developed diagnostic test, it was revealed that students' level of all Learning and Innovation skills requires improvement and development and, therefore, educators should pay attention to introduction of these 21^st century skills training in the educational process within the framework of teaching English at higher education institution. The results of this study will be useful in further researches of teaching methodology and assessment of 21^st century skills and ways of their integration into the higher education system of Kazakhstan that will contribute to providing the country with highly qualified workforce in the foreseeable future.

References

1. British Council. (2016). Core skills for learning, work and society: Summary. London: BC.
2. The Partnership for 21st Century Learning. (2012). P21 Framework Definitions: Summary. NY: P21.
3. National Commission on Excellence in Education. (1983). A Nation at Risk: The Imperative for Educational Reform: Report. Washington, D.C.: U.S. Government Printing Office.
4. Abdilmanova, D.R. (2019). Klassifikatsiia navykov XXI veka, priemlemykh dlia vnedreniia v uchebnyi protsess v ramkakh obucheniia anhliskomu yazyku [The classification of 21st century skills which are acceptable for introduction in the educational process within the framework of teaching English]. Vestnik SKHU imeni M. Kozybaeva — Bulettin of M. Kozybayev NKSU, 2, 83-91 [in Russian].
5. American Management Association. (2012). Critical Skills Survey: Executive Summary. Washington, D.C.: AMA.
6. Trilling, B., & Fadel, Ch. (2009). 21st century skills: learning for life in our times. San Francisco: Jossey-Bass.
7. Asia Society: Global Cities Education Network. (2013). Measuring 21^stcentury competencies: Guidance for educators. Hong Kong: Asia Society.
8. Griffin, P., & McGaw, B. (2012). Assessment and teaching of 21^st century skills. Springer.