A study on the construction of a data-driven civic assessment system for the course of composition and work analysis

The course of composition and work analysis is one of the basic theory courses for musicology and music performance majors in colleges and universities, and its nature has strong theoretical and practical guidance. Through the study of this course, students can master the structural principles, internal characteristics and other intellectual level theories of Chinese and Western musical works [14] and use these intellectual theories to analyze, understand and interpret specific musical works so as to improve the rational level of music aesthetics and educate people through music, and at the same time, provide a certain guiding basis for music practice [5–7]. With the continuous promotion of the concept of political thinking and human education, the task of strengthening the political thinking of professional courses has been gradually upgraded, and it is necessary to add the elements of political thinking in the professional courses of musicology and music performance [8–10].

“Curriculum Civics” is based on the educational concepts put forward by the reform of ideological and political education in colleges and universities, relying on, with the help of professional courses, general education courses and ideological and political education activities, so as to realize the new pattern of all-member, all-encompassing, all-around human education [11–14]. The teaching design and practice of the concept of ideological and political education in the implementation of the curriculum of the analysis of the composition and works has always been through the effective integration of the three levels of the perceptual experience of the sensibility of the musical works of the sensibility of the experience of the intellectual, technical analysis of the rational aesthetic transcendence [15–17]. This not only effectively avoids the emptiness of rational aesthetic transcendence due to the lack of support of ontological technical analysis, but also effectively avoids the avant-garde nature of perceptual associative experience due to the lack of rational connotation of aesthetic transcendence or the extension of music to educate people, as well as the boring nature of ontological technical analysis due to the lack of rational interpretation and aesthetic appreciation [18–21].

Taking the course of composition and work analysis of music majors in A university as the research object, this paper describes the application of the data-driven teaching evaluation method in the course of composition and work analysis. Using big data technology to mine policy texts, theoretical literature, etc., to extract high-frequency words, and taking the CIPP model, which fits the evaluation of the course’s civics, as the theoretical basis, the civic teaching evaluation system of the course on the analysis of compositions and works is constructed. Then, the combination of hierarchical analysis method and entropy weight method is used to assign weights to the evaluation indexes. Finally, the fuzzy comprehensive evaluation method is used to conduct empirical research to verify the reliability of the rating system.

2

Data-driven Evaluation of Civic Teaching and Learning in the Course on Analysis of Songs and Works

2.1

Teaching Civics in the Course of Composition and Work Analysis

Curriculum Civics refers to the organic integration of ideological and political elements in the teaching process of professional courses so that all kinds of professional courses and ideological and political courses work together to educate people and ultimately build a big pattern of all-embracing, all-round educating people.

Composition and Work Analysis is a basic course for music undergraduate majors in colleges and universities that provides strong theoretical and practical guidance. Through the study of this course, students can not only master the intellectual levels theories, such as the structural principles and structural types of Chinese and Western major and minor modal systems, but also learn to use these intellectual theories to analyse, understand and interpret musical works, so as to improve the rational level of music aesthetics and educate people with music, and at the same time provide a certain guiding basis for music practice.

According to the cultivation standard of undergraduate level, while teaching the basic knowledge of music and case studies of related music works, the educational concept of curriculum ideology should also be integrated so as to subconsciously cultivate the students’ attitude of striving for improvement, analytical and problem-solving ability at a higher level of educating people through music, improve the students’ aesthetic and humanistic cultivation of art, enhance the students’ vocational ethics and sense of social responsibility, and convey the truth, goodness and beauty, It also enhances students’ professional ethics and sense of social responsibility, conveys truth, goodness and beauty, cultivates sentiments, shapes personalities and enlightens wisdom.

2.2

Data-driven Evaluation of Civics Teaching in the Curriculum

Data-driven instructional assessment is an approach to evaluating instruction that relies on data collected during instructional activities for decision-support, using data analysis. The essence of this approach lies in the utilization of quantitative data to reveal student learning processes, outcomes, and needs in order to inform instructional improvement. Characteristics of data-driven teaching evaluation include real-time, objectivity, and dynamism. It requires teachers to have certain data-processing skills and sensitivity to teaching data.

The traditional course Civics teaching evaluation has the problem of information feedback lag; the evaluation lacks comprehensiveness and objectivity, and it is not easy to play the important role of evaluation for teaching and evaluation for learning. The rapid development of big data and artificial intelligence technology has transformed information technology from an adjunct to education to a complete integration with education and teaching. Teachers use data as the basis for measuring and evaluating students’ problems in the process of teaching practice, fueling teaching and pointing to teachers’ precise teaching and students’ precise learning. The gradual maturity of intelligent technology and software provides support for teachers’ evaluation on the basis of data, which evaluates teachers’ teaching of Civics and Politics of the curriculum to be visualised, precise, and comprehensive.

3

A model for evaluating the teaching of civics in the course of analysis of musical styles and compositions

This paper is based on the CIPP evaluation model and big data technology for the selection of evaluation indicators, the use of hierarchical analysis and entropy weighting method for the combination of the indicators to be assigned, and finally, the use of fuzzy hierarchical analysis to output the evaluation results, so as to realise the construction and application of the Civic and Political Evaluation System of the Curriculum for the Analysis of Songs and Works.

3.1

CIPP Evaluation Model

CIPP (Context-Input-Process-Product) is a model for evaluating educational activities and teaching effectiveness based on educational improvement [22]. The basic framework of the CIPP evaluation model is shown in Figure 1, which divides educational evaluation into four parts: background evaluation, input evaluation, process evaluation, and outcome evaluation.

Background evaluation aims to evaluate and judge the rationality of educational objectives based on social development and the needs of the evaluation target. Input evaluation is the identification and evaluation of alternative programmes, and the evaluation of the resources and conditions required to achieve the educational objectives to ensure that the programme can be implemented, which is essentially an evaluation of the feasibility and effectiveness of the programme. Process evaluation is mainly a monitoring and inspection of the programme implementation process, i.e., whether teachers implement the programme according to the plan, how the students’ feedback is during the teaching process, whether the existing resources are effectively used, and whether the programme implementation has been improved, and so on. Outcome evaluation is the evaluation of the results of programme implementation, i.e., measuring, judging and interpreting the educational outcomes and their impacts and confirming the extent to which students’ educational needs have been met.

The CIPP evaluation model is diagnostic, formative, and summative in nature, and is suitable for evaluating and promoting the continuous improvement of the Civics program. First, the CIPP evaluation model integrates diagnostic, formative, and summative evaluations and can provide decision-makers with more comprehensive information for judgement. Second, the CIPP evaluation model includes multiple subjects such as university management, teachers and students in the scope of the evaluation, which is in line with the concept of three-pronged cultivation of people in curriculum Civic Governance. Third, the CIPP evaluation model is committed to continuous improvement, which is in line with the dynamic development needs of curriculum civic politics. Therefore, this paper chooses this model for the selection of evaluation indexes and the construction of an evaluation system for the Civic Politics of Curriculum Civics of Song and Work Analysis.

3.2

Hierarchical analysis to determine indicator weights

Analytical Hierarchy Process (AHP) is a systematic approach to decision analysis, aiming to dismantle complex decision objectives layer by layer, compare elements within the same level two by two, and quantitatively assess the relative importance between them [23]. Its specific steps are as follows:

3.2.1

Modelling the hierarchy

Systematically sort and classify the decision objectives, factors involved (i.e., decision criteria) and decision objects, and build a clear hierarchical model based on the internal logical relationship between them.

3.2.2

Constructing judgement matrices

Elements at the same level were compared two by two, a comparison process aimed at constructing a judgment matrix based on their relative importance. This provides a quantitative basis for subsequent decision-making analysis. To achieve this goal, experts in the relevant fields were invited to meticulously score the indicators using a nine-level scale method. To wit: 1 $A = (\begin{matrix} a_{11} & a_{12} & \dots & a_{1 n} \\ a_{21} & a_{22} & \dots & a_{2 n} \\ ⋮ & ⋮ & ⋮ & ⋮ \\ a_{m 1} & a_{m 2} & \dots & a_{m n} \end{matrix}), among a_{n m} = \frac{1}{a_{m n}}$

Individual indicators can be compared on a nine-point scale, with a scale of 1 to 9, with indicator a_m increasing in importance compared to indicator a_n, and a scale of a_mn indicating that both are equally important.

3.2.3

Finding the Weight Vector and Maximum Eigenroot

In this paper, the square root method is chosen to calculate the weight vector and the largest characteristic root of the judgement matrix. The calculation steps of square root method are as follows: 1)

Calculate the product of the values in each row, and the result is recorded as M_i: 2 $M_{i} = \prod_{j = 1}^{n} a_{i j} (i = 1, 2, \dots, n)$

2)

Calculate the nth root of M_i. Record the result as ${\bar{W}}_{i}$ : 3 ${\bar{W}}_{i} = \sqrt[n]{M_{i}}$

3)

The vector $\bar{W} = {[{\bar{W}}_{1}, {\bar{W}}_{2}, \dots, {\bar{W}}_{n}]}^{T}$ is normalised and the result is written as W = [W₁,W₂,⋯,W_n]^T, W i.e. the eigenvectors of the matrix: 4 $W_{i} = \frac{\bar{W_{i}}}{\sum_{i = 1}^{n} \bar{W_{i}}}$

4)

Compute the largest characteristic root of the judgement matrix λ_max: 5 $λ_{\max} = \sum_{i = 1}^{n} \frac{{(A W)}_{i}}{n W_{i}}$

3.2.4

Consistency test

By calculating the matrix, the corresponding eigenvalues are derived and subsequently tested and ranked. 1)

Calculate the consistency index CI. The formula is: 6 $C I = \frac{λ_{\max} - n}{n - 1}$

Where: λ_max denotes the maximum eigenvalue, and n denotes the order of the matrix.

2)

Find the corresponding average stochastic consistency index RI. When the order n of the judgement matrix is 1~11, the corresponding RI values are 0, 0, 0.52, 0.89, 1.12, 1.26, 1.36, 1.41, 1.46, 0.49, 0.52 respectively.

3)

Calculation of Random Consistency Ratio CR. Due to some uncertainties there is a chance bias in consistency, and in order to determine the extent of this bias, the ratio of CI to the random consistency indicator RI, i.e., the test factor CR, is calculated with the formula: 7 $C R = \frac{C I}{R I}$

The judgement matrix is usually considered to pass the consistency test when CR < 0.1. Otherwise, the judgement matrix does not have proper consistency and needs to be corrected.

3.3

Entropy weighting method to determine the weights of indicators

The entropy weight method belongs to the objective assignment method, which is in accordance with the differences in the indicators, based on the entropy of information entropy to calculate the entropy of each indicator and then amend the weight of each indicator to achieve the effect of dynamic assignment [24]. The advantage of this method to determine the weights is that it is not interfered with by subjective factors, and the evaluation of the system is more fair and objective.

The steps to determine the weight of indicators by the entropy weighting method are as follows: 1)

Indicator forwarding. Due to the existence of indicator data of different qualities and dimensions, positive and negative indicators represent different meanings: the larger the positive indicators, the better. The lower the negative indicators, the better. Therefore homogenisation is needed to transform all indicators into very large ones. The forwarding matrix, consisting of n evaluation objects and m evaluation indicators, is as follows: 8 $X = [\begin{matrix} x_{11} & x_{12} & \dots & x_{1 n} \\ x_{21} & x_{22} & \dots & x_{2 n} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ x_{n 1} & x_{n 2} & \dots & x_{n n} \end{matrix}]$

2)

Data standardisation. Indicators still exist between the outline of the different and can not be mixed operation. The need for normalisation will be different indicators of the same scale, due to the above has been indicators of positive, here the standardization formula is: 9 $x_{i j}^{'} = \frac{x_{i j} - \min {x_{i j}, x_{2 j}, \dots, x_{n j}}}{\max {x_{1 j}, x_{2 j}, \dots, x_{n j}} - \min {x_{1 j}, x_{2 j}, \dots, x_{n j}}}$

3)

Calculate the weight occupied by each indicator. The weight occupied by the i nd object in the j st indicator is regarded as the probability value in the information entropy calculation. On this basis, the overall calculation results in the probability matrix P, and the formula for each indicator element in the matrix is as follows: 10 $p_{i j} = \frac{x_{i j}^{'}}{\sum_{i = 1}^{n} x_{i j}^{'}} (j = 1, 2, \dots, m)$

4)

Calculate information entropy. Calculate the corresponding information entropy of each indicator, i.e. uncertainty, and standardise to get the specific entropy weight value of each indicator. The calculation formula is as follows: 11 $e_{j} = - \frac{1}{ln n} \sum_{i = 1}^{n} p_{i j} ln (p_{i j})$

Where $k = \frac{1}{ln n} > 0, e_{j} > 0$ . e_j is larger, the larger the information entropy is, the less information is represented to be included.

5)

Calculate the weight coefficients. Introduce the information utility value d_j: 12 $d_{j} = 1 - e_{j}$

If the value of information utility is smaller, it indicates that the indicator is more important.

The overall information utility value is normalised to obtain the specific entropy weight of each indicator: 13 $w_{j} = \frac{d_{j}}{\sum_{j = 1}^{m} d_{j}}$

6)

Calculate the integrated score. The entropy weight size weight integration calculation formula is as follows: 14 $s_{i} = \sum_{j = 1}^{m} w_{j} \cdot p_{i j}$

3.4

Fuzzy integrated evaluation method

The fuzzy comprehensive evaluation method is a kind of evaluation method based on fuzzy mathematical theory, which is mainly used to deal with fuzzy and uncertain decision-making problems by quantifying the qualitative indexes so as to get comprehensive evaluation results [25]. In this paper, we use the fuzzy comprehensive evaluation method to calculate the evaluation results of the Civics and Politics of Curriculum Analysis of Songs and Works, and its specific process is as follows: 1)

Determine the evaluation index set U = (U₁,U₂…U_n).

2)

Determine the evaluation set V = (V₁,V₂⋯V_n).

3)

Establish the fuzzy evaluation matrix

Invite experts to evaluate and score different indicators according to the set evaluation level, get the vector of indicators’ affiliation according to the obtained data, and combine the affiliation degrees to get the fuzzy evaluation matrix: 15 $K = (\begin{matrix} μ_{1} \\ μ_{2} \\ ⋮ \\ μ_{n} \end{matrix}) = (\begin{matrix} k_{11} & k_{12} & \dots & k_{1 m} \\ k_{21} & k_{22} & \dots & k_{2 m} \\ \dots & \dots & \dots & \dots \\ k_{n 1} & k_{n 2} & \dots & k_{n m} \end{matrix})$

In the formula, k_ij is the degree of affiliation of the i nd indicator to the j rd evaluation level, and the value of i = 1,2⋯n,j = 1,2⋯m,k_ij is divided by the number of experts who made the j th evaluation level to the total number.

4)

Establishment of a fuzzy comprehensive evaluation matrix

Combine the obtained indicator weight set W with the fuzzy evaluation matrix K to obtain the fuzzy comprehensive evaluation matrix R: 16 $R = W \times K = (r_{1}, r_{2} \dots r_{m})$

Where m is the number of evaluation levels.

For multi-level indicators, it is necessary to establish the fuzzy evaluation matrix of the corresponding level, and then based on the weight set of indicators of each level, combine it with the fuzzy evaluation matrix of the corresponding level, and finally obtain the fuzzy evaluation matrix of the first-level indicators Z: 17 $Z = (\begin{matrix} z_{11} & z_{12} & \dots & z_{1 m} \\ z_{21} & z_{22} & \dots & z_{2 m} \\ \dots & \dots & \dots & \dots \\ z_{s 1} & z_{s 2} & \dots & z_{s m} \end{matrix})$

Where S is the number of first-level indicators and m is the number of evaluation levels.

The weights of the first-level indicators are combined with the fuzzy evaluation matrix Z of the first-level indicators to find the fuzzy comprehensive evaluation vector of the target L: 18 $L = (I_{1}, I_{2} \dots I_{n})$

4

Research on the application of a data-driven evaluation system for teaching civics in the curriculum

4.1

Construction of Evaluation Indicator System

4.1.1

Selection of evaluation indicators

1)

Initial construction of evaluation indexes for teaching Civics and Politics in courses based on the CIPP model

In this paper, we first use big data technology to mine and extract high-frequency words from policy texts and theoretical literature related to curriculum civics and politics, and then use artificial intelligence technology to optimise them, and then construct a preliminary evaluation index system for the evaluation of curriculum civics and politics teaching in the context of the CIPP evaluation model of curricula and works analysis as shown in Table 1.

Firstly, the CIPP model is set up with four first-level indicators of teaching background (A1), teaching input (A2), teaching process (A3), and teaching results (A4). Secondly, eight secondary indicators were constructed according to the categories to which the extracted high-frequency words belonged, namely, the background of CIPP teaching (B1), the teaching background of the course (B2), the input of the external environment (B3), the input of the teacher (B4), the design of CIPP teaching programme (B5), the implementation of CIPP teaching programme (B6), the objective evaluation (B7), and the subjective evaluation (B8). Finally, based on the high-frequency word association text analysis, 20 tertiary indicators were refined, which were civics teaching objectives (C1), civics teaching orientation (C2), civics teaching philosophy (C3), curriculum foundation (C4), teachers’ teaching ability (C5), financial support (C6), policy support (C7), preparation for classes (C8), civics teaching resources (C9), teachers’ civics theory reserve ( C10), Civics Teaching Design (C11), Civics Element Mining (C12), Reasonableness of Element Incorporation (C13), Programme Operability (C14), Classroom Student Participation (C15), Civics Teaching Goal Achievement (C16), Teaching Effectiveness (C17), Questionnaire Survey (C18), Student Feedback (C19), Teacher Self-Evaluation (C20).

2)

Questionnaire analysis method to determine the reliability and validity of evaluation indicators

According to Table 1, this paper designs a pre-survey questionnaire containing 20 question items and asks the subjects to score their importance. Finally, 150 valid questionnaires were recovered to optimize the index system through the reliability and validity test method. Reliability analysis is used to test the reliability of the question item design. When Cronbach’s alpha coefficient is between 0.8 and 1.0 and the corrected correlation is greater than 0.5, it indicates high reliability. The goal of validation analysis is to verify the accuracy of the question items and measurement content. When the KMO value is greater than 0.7 and the significance of Bartlett’s test of sphericity is less than 0.05, it indicates that the questionnaire design has good structural validity.

Table 1.

Preliminary evaluation index system

Primary indicator	Secondary indicator	Tertiary index
A1	B1	C1
		C2
		C3
	B2	C4
	B2	C5
A2	B3	C6
	B3	C7
	B4	C8
		C9
		C10
A3	B5	C11
		C12
		C13
	B6	C14
	B6	C15
A4	B7	C16
		C17
		C18
		C19
	B8	C20

In this paper, SPSS28.0 analysis software was used to obtain the analysis results. In this case, observation C10 was deleted because the corrected correlation was 0.451<0.5.

4.1.2

Determination of indicator weights

In this study, the weights of the indicators at all levels were first determined by using the hierarchical analysis method, and the management of Civic Education and first-line teachers were invited to make two-by-two comparisons through the Delphi method to get the judgement matrices of the indicators at all levels. Figure 2 shows the judgement matrix for the first-level indicators.

According to the judgement matrix in Fig. 2, the square root method was used to obtain the weights of the first-level indicators A1~A4 as 0.0521, 0.1199, 0.3373, 0.4907, respectively, and the largest characteristic root λmax=4.228. The consistency test was conducted, and it was obtained that: CI=0.076, RI=0.89, and CR=0.085<0.1, which indicated that the internal judgement matrix was consistently good and met the requirements.

Similarly, the weights of the secondary and tertiary indicators are determined, and the weights of the evaluation indicators based on the hierarchical analysis method are obtained, as shown in Table 2.

Table 2.

The evaluation index weight based on AHP

Primary indicator	W_ai	Secondary indicator	W_bi	Tertiary index	W_ci	Absolute weight
A1	0.0521	B1	0.5	C1	0.3333	0.0087
				C2	0.3333	0.0087
				C3	0.3334	0.0087
		B2	0.5	C4	0.3268	0.0085
		B2	0.5	C5	0.6732	0.0175
A2	0.1199	B3	0.4603	C6	0.5	0.0276
		B3	0.4603	C7	0.5	0.0276
		B4	0.5397	C8	0.3458	0.0224
		B4	0.5397	C9	0.6542	0.0423
A3	0.3373	B5	0.3642	C11	0.4256	0.0523
				C12	0.2569	0.0316
				C13	0.3175	0.0390
		B6	0.6358	C14	0.5635	0.1208
		B6	0.6358	C15	0.4365	0.0936
A4	0.4907	B7	0.6667	C16	0.2511	0.0821
				C17	0.4263	0.1395
				C18	0.1537	0.0503
				C19	0.1689	0.0553
		B8	0.3333	C20	1	0.1635

As can be seen from Table 2, the weights of the first-level indicators are, in descending order, Teaching Outcome A4 (0.4907), Teaching Process A3 (0.3373), Teaching Input A2 (0.1199) and Teaching Background A1 (0.0521). Teaching outcome is the ultimate goal of evaluating the CIPP of the course, including the comprehensive improvement of students’ knowledge level, ideological awareness and moral quality, which is the key indicator for evaluating the effectiveness of the CIPP of the course. Experts assign the highest importance to this indicator in the CIPP model. The teaching process affects the achievement of teaching results to a great extent, and a high-quality teaching process can effectively impart knowledge, guide students’ thinking and cultivate their abilities, and is therefore given high importance in the CIPP model. Teaching context and teaching input, though not highly weighted, play a guiding and supporting role and are the prerequisites and guarantees for the development of course-based Civics.

Since the hierarchical analysis method (AHP) has strong subjectivity in evaluating the indicators, and the entropy weight method (EWM) is mainly based on quantitative data, this paper then applies the entropy weight method to find the indicator weights, and couples the weights obtained by the two methods, in order to reduce the errors brought by the subjective and objective factors, and to make the final results of evaluation of the course civic politics more reasonable and objective. The combined weight formula is: 19 $λ_{j} = \frac{w_{j} v_{j}}{\sum_{j = 1}^{n} w_{j} v_{j}}$

In the formula, w_j is the weight of entropy value method, and v_j is the weight of the hierarchical analysis method.

Using AHP and the entropy method to assign weights to each index, the weight values of the evaluation indexes of the Civic Teaching Evaluation of the Curriculum for Analysis of Songs and Works are calculated by formula (19), as shown in Table 3.

Table 3.

Teaching evaluation index system weight

Primary indicator	EWM	Composite weight	Secondary indicator	EWM	Composite weight	Tertiary index	EWM	Composite weight	Absolute weight
A1	0.2746	0.0601	B1	0.5	0.5	C1	0.3057	0.3057	0.0092
						C2	0.3836	0.3836	0.0115
						C3	0.3107	0.3108	0.0093
			B2	0.5	0.5	C4	0.5123	0.3377	0.0102
			B2	0.5	0.5	C5	0.4877	0.6623	0.0199
A2	0.2473	0.1244	B3	0.5426	0.5029	C6	0.5	0.5	0.0313
			B3	0.5426	0.5029	C7	0.5	0.5	0.0313
			B4	0.4574	0.4971	C8	0.5528	0.3952	0.0244
			B4	0.4574	0.4971	C9	0.4472	0.6048	0.0374
A3	0.2623	0.3712	B5	0.4875	0.3527	C11	0.3561	0.4520	0.0592
						C12	0.3414	0.2616	0.0343
						C13	0.3025	0.2864	0.0375
			B6	0.5125	0.6473	C14	0.3768	0.4384	0.1053
			B6	0.5125	0.6473	C15	0.6232	0.5616	0.1349
A4	0.2158	0.4443	B7	0.4239	0.6954	C16	0.3143	0.2863	0.0885
						C17	0.3218	0.4977	0.1538
						C18	0.1285	0.0717	0.0221
						C19	0.2354	0.1443	0.0446
			B8	0.5761	0.3046	C20	1	1	0.1353

As can be seen from Table 3, experts pay more attention to the four dimensions of programme operability (C14), classroom student participation (C15), teaching effect (C17), and teacher self-assessment (C20), and the weights obtained are all greater than 0.1. 1)

The absolute weight of “teaching effect” among the three-level indicators is 0.1538, which is significantly higher than other indicators, indicating that “teaching effect” is the most important factor for evaluating the quality of Civic and Political Teaching in the course of analyzing music and works in the system, which is in line with the implementation of teaching. This is consistent with the teaching implementation.

2)

The final absolute weights of “teacher self-assessment”, “classroom student participation” and “program operability” are 0.1353, 0.1349 and 0.1053 respectively, ranking second, third and fourth respectively. The final absolute weights of “teacher self-evaluation and reflection”, “classroom student participation” and “program operability” are 0.1353, 0.1349, 0.1053, respectively, which are ranked the second, third and fourth respectively, indicating that teacher self-evaluation and reflection, student participation in the classroom, and program operability are the relatively important factors affecting the evaluation of the quality of teaching of civic education of the course of analyzing the music style and works in the system.

3)

Preparation for class (C8) and questionnaire (C18) only accounted for 0.0244 and 0.0221, respectively, indicating that these two items have less influence on the quality of Civic Teaching of Curriculum of Analysis of Pieces and Works. The main reason for this is that although teachers are advocated to carefully prepare their lessons and investigate the quality of the course through questionnaires, the teaching effect still depends on the teachers’ understanding and application of the course concepts as well as the rational design of the teaching program. Therefore, these two items play a supplementary role in the overall evaluation system.

4.2

Empirical analyses

4.2.1

Construction of evaluation factor set, rubric set and weight set

In order to verify the practical effect of the application of the evaluation system of the ideological and political teaching of the composition and work analysis course, this study applies the fuzzy comprehensive evaluation method and selects experts in the field of ideological and political education and the field of music education, peer teachers who are experienced in the teaching of the composition and work analysis course, and students of the music undergraduate majors of university A to evaluate the teaching of the ideological and political teaching of the composition and work analysis course of university A according to the evaluation system constructed.

Firstly, the factor set U is established, and the evaluation factor set of the first-level indicator is {U₁, U₂, U₃, U₄}, the evaluation factor set of the second-level indicator in the context of the first-level indicator teaching is {U₁₁, U₁₂, U₁₃, U₁₄}, the evaluation factor set of the third-level indicator in the context of the second-level indicator Civic Teaching is {U₁₁₁, U₁₁₂, U₁₁₃}, and so on to derive the evaluation factor set.

Secondly, the set of rubrics is established. V={V₁, V₂, V₃, V₄, V₅}={very good, better, average, poor, very poor}.

Finally, establish the weight set. According to the weights of indicators at all levels calculated in the previous section, the weight set is established in turn:

W =(0.0601,0.1244,0.3712,0.4443), W₁ =(0.5,0.5), W₂ =(0.5029,0.4971), W₃ =(0.3527,0.6473), W₄ =(0.6954,0.3046), W₁₁ =(0.3057,0.3846,0.3108), W₁₂ =(0.3377,0.6623), W₂₁ =(0.5,0.5), W₂₂ =(0.3952,0.6048), W₃₁ =(0.4520,0.2616,0.2864), W₃₂ =(0.4384,0.5616), W₄₁ =(0.2863,0.4977,0.0717,0.1443), W₄₂ =(1).

4.2.2

Fuzzy Comprehensive Evaluation Results and Analyses

By distributing the questionnaire, the evaluator scores each indicator of the indicator layer on a hierarchical scale, collates the number of scores for each indicator, and uses the number of scores/total number of scores to derive the degree of affiliation, and then establishes a single-factor fuzzy composite judgement matrix as shown in Table 4.

Table 4.

The evaluation membership of the tertiary index

Evaluation index		Evaluation grade
Evaluation index		V₁	V₂	V₃	V₄	V₅
B1	C1(0.3057)	0.1465	0.2999	0.4099	0.0612	0.0825
	C2(0.3836)	0.0575	0.4594	0.2829	0.1221	0.0781
	C3(0.3108)	0.0714	0.3842	0.3988	0.1427	0.0029
B2	C4(0.3377)	0.1382	0.5196	0.2279	0.1143	0.0000
B2	C5(0.6623)	0.1534	0.5144	0.2793	0.0529	0.0000
B3	C6(0.5)	0.0656	0.4249	0.4461	0.0065	0.0569
B3	C7(0.5)	0.1436	0.4871	0.3528	0.0165	0.0000
B4	C8(0.3952)	0.0233	0.2629	0.4321	0.1973	0.0844
B4	C9(0.6048)	0.0104	0.3776	0.3881	0.1638	0.0601
B5	C11(0.4520)	0.1117	0.4685	0.3394	0.0804	0.0000
	C12(0.2616)	0.0658	0.4568	0.2338	0.1227	0.1209
	C13(0.2864)	0.1756	0.4965	0.1939	0.0904	0.0436
B6	C14(0.43847)	0.0114	0.4565	0.4195	0.0682	0.0444
B6	C15(0.5616)	0.0689	0.3992	0.3343	0.1976	0.0000
B7	C16(0.2863)	0.1113	0.4896	0.3157	0.0834	0.0000
	C17(0.4977)	0.1177	0.2663	0.4811	0.1349	0.0000
	C18(0.0717)	0.1996	0.4107	0.3635	0.0081	0.0181
	C19(0.1443)	0.0553	0.4337	0.3737	0.0517	0.0856
B8	C20(1)	0.0567	0.5515	0.3182	0.0497	0.0239

Based on the affiliation of the three levels of indicators in Table 4, a fuzzy evaluation matrix of the context of B1 Civics teaching can be derived: $K_{11} = [\begin{matrix} \begin{array}{l} 0.1465 \\ 0.0575 \\ 0.0714 \end{array} & \begin{matrix} \begin{array}{l} 0.2999 \\ 0.4594 \\ 0.3842 \end{array} & \begin{array}{l} 0.4099 \\ 0.2829 \\ 0.3988 \end{array} & \begin{array}{l} 0.0612 \\ 0.1221 \\ 0.1427 \end{array} \end{matrix} & \begin{array}{l} 0.0825 \\ 0.0781 \\ 0.0029 \end{array} \end{matrix}]$

Then the fuzzy comprehensive evaluation vector of B1 is: 21 $\begin{array}{l} R_{11} = W_{11} ▯ K_{11} = (0.3057, 0.3846, 0.3108) ▯ [\begin{matrix} \begin{array}{l} 0.1465 \\ 0.0575 \\ 0.0714 \end{array} & \begin{matrix} \begin{array}{l} 0.2999 \\ 0.4594 \\ 0.3842 \end{array} & \begin{array}{l} 0.4099 \\ 0.2829 \\ 0.3988 \end{array} & \begin{array}{l} 0.0612 \\ 0.1221 \\ 0.1427 \end{array} \end{matrix} & \begin{array}{l} 0.0825 \\ 0.0781 \\ 0.0029 \end{array} \end{matrix}] \\ = (0.0890, 0.3873, 0.3577, 0.1099, 0.0561) \end{array}$

Similarly, the fuzzy comprehensive evaluation vectors for the secondary indicators B2~B8 are obtained, respectively: $R_{12} = W_{12} ▯ K_{12} = (0.1483, 0.5162, 0.2619, 0.0736, 0)$ $R_{21} = W_{21} ▯ K_{21} = (0.1046, 0.4560, 0.3995, 0.0115, 0.0284)$ $R_{22} = W_{22} ▯ K_{22} = (0.0155, 0.3323, 0.4055, 0.1770, 0.0697)$ $R_{31} = W_{31} ▯ K_{31} = (0.1180, 0.4735, 0.2701, 0.0943, 0.0441)$ $R_{32} = W_{32} ▯ K_{32} = (0.0437, 0.4243, 0.3716, 0.1409, 0.0195)$ $R_{41} = W_{41} ▯ K_{41} = (0.1127, 0.3647, 0.4098, 0.0991, 0.0137)$ $R_{42} = W_{42} ▯ K_{42} = (0.0567, 0.5515, 0.3182, 0.0497, 0.0239)$

Collating the results of the calculations leads to the evaluation affiliation of the secondary indicators as shown in Table 5.

Table 5.

The evaluation membership of the secondary indicator

Evaluation index		Evaluation grade
Evaluation index		V₁	V₂	V₃	V₄	V₅
A1 (0.0601)	B1(0.5)	0.0890	0.3873	0.3577	0.1099	0.0561
A1 (0.0601)	B2(0.5)	0.1483	0.5162	0.2619	0.0736	0.0000
A2 (0.1244)	B3(0.5029)	0.1046	0.4560	0.3995	0.0115	0.0284
A2 (0.1244)	B4(0.4971)	0.0155	0.3323	0.4055	0.1770	0.0697
A3 (0.3712)	B5(0.3527)	0.1180	0.4735	0.2701	0.0943	0.0441
A3 (0.3712)	B6(0.6473)	0.0437	0.4243	0.3716	0.1409	0.0195
A4 (0.4443)	B7(0.6954)	0.1127	0.3647	0.4098	0.0991	0.0137
A4 (0.4443)	B8(0.3046)	0.0567	0.5515	0.3182	0.0497	0.0239

According to Table 5, the fuzzy comprehensive evaluation vectors for level 1 indicators A1~A4 can be calculated as respectively: $R_{1} = W_{1} ▯ K_{1} = (0.1187, 0.4518, 0.3098, 0.0917, 0.0280)$ $R_{2} = W_{2} ▯ K_{2} = (0.0603, 0.3945, 0.4025, 0.0938, 0.0489)$ $R_{3} = W_{3} ▯ K_{3} = (0.0699, 0.4416, 0.3358, 0.1245, 0.0282)$ $R_{4} = W_{4} ▯ K_{4} = (0.0956, 0.4216, 0.3819, 0.0841, 0.0168)$ $= (0. 1025, 0. 3550, 0. 4154, 0.0 894, 0.0 375)$

Integrate the results of the judgments R₁, R₂, R₃, R₄, to get the comprehensive fuzzy evaluation matrix of the evaluation of the teaching of Civics and Politics in the course of the analysis of compositions and works in A university: $K = [\begin{matrix} \begin{array}{l} 0.1187 \\ 0.0603 \\ 0.0699 \\ 0.0956 \end{array} & \begin{matrix} \begin{array}{l} 0.4518 \\ 0.3945 \\ 0.4416 \\ 0.4216 \end{array} & \begin{array}{l} 0.3098 \\ 0.4025 \\ 0.3358 \\ 0.3819 \end{array} & \begin{array}{l} 0.0917 \\ 0.0938 \\ 0.1245 \\ 0.0841 \end{array} \end{matrix} & \begin{array}{l} 0.0280 \\ 0.0489 \\ 0.0282 \\ 0.0168 \end{array} \end{matrix}]$ ,

Then the comprehensive fuzzy evaluation vector of the evaluation of the Civic Teaching Evaluation of the Course of Analysis of Pieces and Works in University A is: $R = W ▯ K = (0.0830, 0.4275, 0.3630, 0.1008, 0.0257)$ .

The set of rubrics is assigned a value of 5 to 1 according to the different evaluation levels, i.e., V = {V₁, V₂, V₃, V₄, V₅} = {Very Good, Good, Fair, Poor, Very Poor} = {5, 4, 3, 2, 1}. Then the overall rating value can be derived as: $L = R ▯ V = [\begin{matrix} 5 & 4 & 3 & \begin{matrix} 2 & 1 \end{matrix} \end{matrix}] ▯ [\begin{array}{l} 0.0830 \\ 0.4275 \\ 0.3630 \\ 0.1008 \\ 0.0257 \end{array}] = 3.4413$ .

The final arithmetic result of the fuzzy comprehensive evaluation is 3.4413, which belongs to the level between average and better. By comparing the evaluation results of experts, peer teachers and students’ evaluation subjects with the analysis of the fuzzy comprehensive evaluation results, the evaluation results are more consistent, indicating that the evaluation system of the civic teaching of the song and work analysis course constructed in this study is reliable and reasonable.

5

Conclusion

The CIPP model is used to construct a data-driven, data-oriented evaluation system for civic and political teaching of composition and work analysis courses in this paper. It utilizes the hierarchical analysis method and entropy weight method for combined assignments, and employs the fuzzy comprehensive evaluation method for comprehensive assessments.

The evaluation system of Civic and Political Teaching of the Curriculum for Analysis of Songs and Works is based on the CIPP model, which sets four first-level indicators of teaching background, teaching input, teaching process and teaching results. It is divided into 8 secondary indicators and 19 tertiary indicators. The secondary indicators include a background in civics teaching, a course teaching background, external environment input, teacher input, Civics teaching program design, implementation, objective evaluation, and subjective evaluation. Tertiary indicators include Civics teaching objectives, Civics teaching orientation, Civics teaching philosophy, curriculum foundation, teachers’ teaching ability, financial support, policy support, class preparation, Civics teaching resources, Civics teaching design, Civics elements mining, the rationality of elements integration, programme operability, classroom students’ participation, Civics teaching objectives achievement, teaching effectiveness, questionnaire survey, students’ feedback, and teachers’ self-assessment.

The civics teaching of the song and work analysis course at University A was selected as the research object, and a fuzzy, comprehensive evaluation was conducted using the built-in evaluation index system. Its overall evaluation score was 3.4413, with the teaching level ranging between average and better. Comparing the fuzzy comprehensive evaluation results with its actual evaluation results, it can be seen that the two are more consistent, indicating that the evaluation system of the Civic Teaching of Song and Work Analysis Course constructed in this study has practicality.

Idioma:: Inglés

Calendario de la edición:: 1 veces al año
Temas de la revista:: Ciencias de la vida, Ciencias de la vida, otros, Matemáticas, Matemáticas aplicadas, Matemáticas generales, Física, Física, otros

RSS Feed de revista

Primary indicator	Secondary indicator	Tertiary index
A1	B1	C1
		C2
		C3
	B2	C4
	B2	C5
A2	B3	C6
	B3	C7
	B4	C8
		C9
		C10
A3	B5	C11
		C12
		C13
	B6	C14
	B6	C15
A4	B7	C16
		C17
		C18
		C19
	B8	C20

Primary indicator	Secondary indicator	Tertiary index
A1	B1	C1
		C2
		C3
	B2	C4
	B2	C5
A2	B3	C6
	B3	C7
	B4	C8
		C9
		C10
A3	B5	C11
		C12
		C13
	B6	C14
	B6	C15
A4	B7	C16
		C17
		C18
		C19
	B8	C20

A study on the construction of a data-driven civic assessment system for the course of composition and work analysis

Dan Wang

Yue Li

Haidong Wang

Publicado en línea: 03 feb 2025

Recibido: 26 sept 2024

Aceptado: 01 ene 2025

DOI: https://doi.org/10.2478/amns-2025-0037

Palabras claveCIPP evaluation model, Hierarchical analysis method, Entropy weight method, Fuzzy comprehensive evaluation, Course Civics and Politics

© 2025 Dan Wang et al., published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Palabras clave
CIPP evaluation model, Hierarchical analysis method, Entropy weight method, Fuzzy comprehensive evaluation, Course Civics and Politics

Primary indicator	Secondary indicator	Tertiary index
A1	B1	C1
		C2
		C3
	B2	C4
	B2	C5
A2	B3	C6
	B3	C7
	B4	C8
		C9
		C10
A3	B5	C11
		C12
		C13
	B6	C14
	B6	C15
A4	B7	C16
		C17
		C18
		C19
	B8	C20