A quantitative study of disruptive technology policy texts: An example of China’s artificial intelligence policy

Harold Lasswell (2017), a founder of policy science, introduced the concept of public policy evaluation, encompassing various dimensions such as goals, values, and strategies. Lambin et al. (2014) highlighted that to attain effective goals and efficiency, policies must harness their synergistic potential, enabling policy subsystems to collaborate and support one another, thus fostering policy synergy. Policy synergy primarily involves the collaborative integration of policy tools, cooperative efforts among policy actors, and the evolving cooperation of policy themes (van den Bergh et al., 2021). However, such research often lacks in-depth discussion and analysis at the policy theme level. Therefore, this article focuses on Al disruptive technology policy texts, establishing a three-dimensional analysis framework that considers policy tools, policy actors, and policy themes. This framework aims to explore and analyze the attributes and limitations of policies concerning AI disruptive technology, considering multiple dimensions and granularities, to furnish a foundation for the formulation of policies in the realm of AI to offer valuable reference and guidance. This framework is applicable not only to the analysis of policy texts concerning AI disruptive technology but also to the examination of policy texts across various domains. The specific framework is depicted in Figure 1.

Figure 1.

As a disruptive technology, AI is profoundly forward-thinking and transformative. It presents representative characteristics and challenges in policy formulation and implementation. Additionally, AI technology is intricate and diverse, spanning various fields including society, economy, and daily life. Hence, choosing AI as a case study for disruptive technology policy research can assist relevant government departments in formulating and enhancing pertinent policies.

To ensure policy comprehensiveness and accessibility, this study focuses on AI disruptive technology policies issued by the central government and national ministries and commissions. Keyword searches for “artificial intelligence” and “AI” are performed on the websites of national ministries and commissions, as well as on Pkulaw.cn, covering both subject and content. After excluding informal documents like approvals, letters, and announcements, a total of 142 AI-related policy texts consisting of opinions, plans, notices, approaches, guidelines, etc., are identified. Statistical analysis reveals that the majority of AI disruptive technology policies are enacted after the issuance of the Development Plan for New-Generation Artificial Intelligence by the General Office of the State Council in 2017. Therefore, this study groups a few policies predating 2017 into the year 2017 for analysis purposes. The distribution of policies across each year is illustrated in Figure 2.

Figure 2.

The content of 142 AI disruptive technology policy texts is coded using Nvivo12 qualitative analysis software. Initially, the policy texts are coded sequentially following the format of “policy number-chapter number-article number”. For instance, 1-1-1 denotes the first article of the first chapter of policy document number 1. Subsequently, the analytical units are classified into their respective policy tools, resulting in a total of 1,230 analytical units extracted from the policy texts and categorized into the relevant policy tool system. The definitive classification of policy tools is presented in Table 2.

Table 2 shows that environmental policy tools for China’s AI disruptive technology are utilized the most, comprising 41.22%, followed by supply-side policy tools at 39.02%. Conversely, demand-side policy tools are notably underutilized, making up only 19.76%. This discrepancy suggests an uneven distribution in China’s AI disruptive technology policy tool utilization, with greater emphasis on environmental aspects and lesser attention on supply and demand. Apart from the overall imbalance in AI disruptive technology policy tool usage, significant variation exists in the utilization of internal sub-tools. In terms of frequency, capital investment (18.54%), talent cultivation (23.54%), infrastructure (26.46%), and public services (20.21%) are utilized frequently and consistently. This highlights China’s emphasis on crucial factors like public infrastructure, basic guarantees, and talent development in its AI disruptive technology policy. However, the utilization of policy tools for technology project support (11.25%) is slightly inadequate, as a closer examination of policy texts reveals a lack of support in terms of specialized funds for scientific research projects and project subsidies. Regarding environmental policy tools, goal programming (23.87%), tax incentives (22.49%), policy incentives (19.53%), and finance (21.50%) are significantly utilized, reflecting China’s detailed approach to AI disruptive technology policy, particularly in target planning and consideration of preferences and incentives. However, regulatory control (12.62%) requires further enhancement, particularly in areas such as personal privacy and network security regulations. The New Generation Artificial Intelligence Development Plan, issued by the State Council, emphasizes AI pervasive influence across modern society, potentially impacting laws, regulations, economic security, and social stability. Therefore, it underscores the importance of enhancing AI regulatory control to ensure its safe, reliable, and manageable development. The utilization rate of demand-side policy tools is generally low, reflecting the pace of AI development in the country. Given its rapid advancement and wide-ranging applications, policies and regulations frequently lag behind technological progress. Analysis of specific measures and strategies reveals the stability of policy approaches such as pilot demonstration and open cooperation. This indicates government emphasis on establishing demonstration areas and fostering collaboration between AI industry, academia, and research. However, insufficient attention to government procurement, commercialization of scientific and research, and policy tools for scenario applications hinders policy adaptation to the diverse needs of AI applications in the market.

As a highly impactful disruptive technology, AI requires supply-side policy tools to drive essential technology research, development, and innovation. Simultaneously, environmental policy tools are necessary to foster the healthy growth of the AI industry. Additionally, demand-side policy tools are essential to stimulate market demand for AI products. However, the current AI disruptive technology policy in China exhibits an imbalance in tool utilization. While efforts are made to reinforce the use of supply-side and environmental policy tools, the utilization of demand-side policy tools remains notably insufficient. Against this backdrop, it becomes challenging to establish an innovation ecosystem for AI. Hence, there is a pressing need to enhance the utilization of policy tools in China’s AI disruptive technology policy.

The statistics indicate that 62 policy actors, including the Ministry of Industry and Information Technology, the Ministry of Science and Technology, and the Ministry of Education, are involved in the formulation of the 142 policies on disruptive AI technology. Among them, 52 policies involve joint issuance by two or more policy actors, with the highest number being 19. The multi-faceted nature of AI disruptive technology policies necessitates the collaborative engagement of numerous policy actors in their formulation. To delve deeper into the cooperative dynamics among policy actors, this study employs social network analysis to construct a network illustrating their collaboration. Social network analysis encompasses methodologies for examining the intricate social relationships and attributes among entities, offering insights into the relational structure between diverse subjects (Freeman, 2004). Initially, Python code is employed to generate the co-occurrence matrix of policy actors. Subsequently, Gephi social network analysis software is utilized to compute the density and degree centrality of the cooperative network. Finally, based on these calculations, the cooperative relationship network of policy actors is delineated. The specific network diagram is depicted in Figure 3.

Figure 3.

Figure 3 depicts the density of network connectivity as an indicator of the level of collaboration among policy actors, with nodes representing policy actors. Node size corresponds to the degree centrality of the policy actors, while the thickness of connections between nodes signifies the frequency of co-occurrence between subjects. Larger node size indicates higher degree centrality between policy actors, suggesting greater influence and control over the association with other actors. Upon calculation, the cooperative network density among AI disruptive technology policy actors in China is 0.353, suggesting that the cooperative relationship among these actors is not sufficiently close. Notably, nodes representing the Ministry of Industry and Information Technology, the Ministry of Science and Technology, and the Ministry of Finance are larger, indicating intensive cooperation and serving as central figures in the cooperation network, with degree centrality values of 157, 152, and 115, respectively. Conversely, nodes representing the Ministry of Justice, the National Mine Safety Administration, and the National Radio and Television Administration are smaller, indicating looser cooperation and marginal roles in the cooperation network, with degree centrality values of 12, 9, and 7, respectively. This implies that the influence and control over the association among AI disruptive technology policy actors in China are relatively centralized, with key actors such as the Ministry of Industry and Information Technology and the Ministry of Science and Technology serving as the core for establishing connections. However, the lack of effective coordination and cooperation among peripheral actors poses certain challenges in the policy formulation and implementation process.

An in-depth analysis reveals that the Ministry of Industry and Information Technology and the Ministry of Science and Technology are crucial national departments in AI. They promote scientific and technological innovation and the dissemination of achievements, necessitating leadership in AI-related policy formulation and implementation. Additionally, as an emerging industry in the country, the AI field necessitates adjustments in leveraging and talent training from the Ministry of Finance, Ministry of Education, and other relevant departments to offer adequate financial support and foster a large pool of scientific and technological talent for its development. Hence, central entities like the Ministry of Industry and Information Technology, the Ministry of Science and Technology, and the Ministry of Finance establish numerous connections with other entities, serving as intermediary bridges. Nevertheless, in the field of AI, marginalized entities like the National Working Commission on Aging often serve as “information receivers” or “policy implementers”, indicating their significant involvement in the policy formulation and implementation process. Excessive dependence on central entities impedes effective information transmission and resource allocation among other entities, leading to a sparse cooperation network among AI policy actors. Overall, the AI disruptive technology policy cooperation network in China exhibits a “centerperiphery” structure.

Policies like the Development Plan for New-Generation Artificial Intelligence, Artificial Intelligence Industry Innovation and Development Action Plan, Artificial Intelligence Standardization Innovation Action Plan emphasize the need for government departments to improve collaboration with diverse stakeholders, such as enterprises, academia, and social organizations. Collaborative endeavors and joint document issuance by multiple departments have become a new trend in shaping and enhancing AI disruptive technology policies in China. Therefore, there is an urgent requirement to further strengthen the collaborative cooperation mechanism among the policy actors engaged in AI disruptive technology.

By conducting policy theme analysis, a comprehensive understanding of policy content, goals, and tendencies can be achieved. LDA topic model analysis is employed to extract topics from policy texts, utilizing a Bayesian probabilistic model with a three-layer structure of words, topics, and documents, effectively uncovering latent themes within the analyzed texts (Gan & Qi, 2021). In the LDA topic model, the number of topics must be manually pre-set. To accurately extract the optimal topics, this study evaluates policy topics using the perplexity evaluation method. The calculation formula of perplexity is shown in equation (1): 1 Perplexity(D)=exp{ −∑d=1Mlogp(wd)∑d=1MNd } \[\text{Perplexity}\left( \text{D} \right)=\exp \left\{ -\frac{\sum\nolimits_{d=1}^{M}{\log p\left( wd \right)}}{\sum\nolimits_{d=1}^{M}{Nd}} \right\}\]

Perplexity serves as a criterion for assessing the effectiveness of the topic model, lower perplexity values indicate better generalization ability of the model (Bastani et al., 2019). Typically, the optimal number of topics is extracted when perplexity is minimized. The optimal number of topics, ranging from 1 to 20, is tested to identify the minimum perplexity. The number of optimal AI disruptive technology policy topics for each year from 2017 to 2022 is determined from the perplexity values, with 11 in 2017, 13 in 2018, 14 in 2019, 16 in 2020, 16 in 2021, and 13 in 2022 as shown in Figure 4. To ensure the accuracy of policy theme content, the themes are manually screened to remove irrelevant or less relevant ones to the AI disruptive technology policy. The final selected themes are then named manually, and the results are presented in Table 3.

Figure 4.

Based on the themes presented in Table 3 and the delineation of key areas in the Development Plan for New-Generation Artificial Intelligence, China’s policies regarding disruptive AI technology are classified into five categories: AI theoretical system, intelligent services, intelligent governance, high-tech talent cultivation, and intelligent sharing and mutual trust. The AI theoretical system primarily encompasses sub-themes such as machine learning and algorithmic models, blockchain technology, and internet communication technology. Intelligent services mainly encompass subthemes like smart community, smart pension, smart medical care, smart health, and smart scene application. Intelligent governance primarily comprises sub-themes such as smart family, intelligent transportation, product quality monitoring, energy saving and environmental protection, new energy technology, and digital rural areas. Cultivating high-tech talents mainly entails sub-themes like talents and technical support, international talent cooperation and exchange, education informatization, and intellectual property rights. Intelligent sharing and mutual trust chiefly comprise sub-themes such as data open sharing and platform construction, as well as virtual reality technology. In 2017, marking a new era in AI development, the State Council mentioned it in the government work report for the first time. State departments have ramped up investments in “talents, technology, platforms, and funds” within the AI sector, resulting in a continuous release of policy dividends in disruptive AI technology. As evidenced by Table 3, the themes encompassed in the 2017-2022 AI disruptive technology policies are rich and diverse, aligning closely with the national development strategy.

In order to delve deeper into the potential evolutionary connections among themes across various time frames, the theme evolutionary relationships are discerned through the calculation of theme cosine similarity. The specific formula for this calculation is depicted in equation (2): 2 cos(θ)=∑i=1n(xi*yi)∑i=1n(xi)2*∑i=1n(yi)2 \[\cos \left( \theta \right)=\frac{\sum\limits_{i=1}^{n}{\left( {{x}_{i}}*{{y}_{i}} \right)}}{\sqrt{\sum\limits_{i=1}^{n}{{{\left( {{x}_{i}} \right)}^{2}}}}*\sqrt{\sum\limits_{i=1}^{n}{{{\left( {{y}_{i}} \right)}^{2}}}}}\]

The cosine similarity ranges from 0 to 1. As the value approaches 1, the content of the themes becomes more similar, indicating a higher likelihood of an evolutionary relationship between them. Conversely, as the value decreases, more differences emerge between the themes. To enhance the precision of the evolutionary path, we adopt a cosine similarity threshold of 0.3, based on the approach of previous scholars. Themes with cosine values exceeding 0.3 denote evolutionary relationships, while those below this threshold do not exhibit such relationships (Feng et al., 2018). The PyEcharts package in Python is used to generate Sankey diagrams illustrating the themes with evolutionary relationships across various time windows, as depicted in Figure 5.

Figure 5.

In Figure 5, each node block represents a distinct theme, with connecting lines illustrating their evolutionary relationships. The thickness of these lines indicates the degree of similarity between the themes, thicker lines denote greater similarity. The figure reveals a clear evolutionary relationship among China’s AI disruptive technology policy themes, delineated into four modes: newborn, merger, split, and demise. Initially, the evolution of AI disruptive technology policy themes manifests “newborn” characteristics, exemplified by emerging policy themes like “Education Informatization” and “Rural Revitalization” in 2018. Additionally, theme evolution demonstrates the characteristic of “merging”. For instance, in 2017, four policy themes, “Intelligent Medical Construction” “Technical Talents in the Field of Drones” “Autonomous Driving Technology” and “Information Security of Key Technologies”, merged into the theme of “Network and Information Security” in 2018. Similarly, in 2019, the fusion of three policy themes, “Product Quality Monitoring” “New Generation Innovative Technology Research”and “New Mode of Shared Manufacturing”, formed the theme of “Internet Industrial Technology” in 2020, and so forth. Furthermore, theme evolution demonstrates the characteristic of “splitting”. For instance, the theme “Open Data Sharing” in 2019 splits into four themes: “Public Healthcare” “Intelligent Transportation Collaborative Innovation” “Digitization of the Cultural Industry” and “Digital Resource Sharing” in 2020. Similarly, the theme “Elderly Health Monitoring” in 2020 splits into three themes: “Intelligent Community and Intelligent Elderly Care” “Digital Home” and “Intelligent Medical Care” in 2021, and so forth. Lastly, the themes undergo a phase of “demise”. For instance, the theme “Talent and Technical Support”, present in 2020, vanishes in 2021.

Considering the “source-target” perspective of the evolutionary trajectory, China’s AI disruptive technology policy encompasses multiple thematic evolution paths.

Intelligent Medical Construction (2017) → Medical Informatization (2018) → Health Care Aids (2019), Elderly Health Products (2019) → Public Healthcare (2020), Elderly Health Monitoring (2020) → Intelligent Medical Care (2021), Digital Home (2021), Intelligent Community and Intelligent Elderly (2021) → Intelligent Elderly (2022);

The analysis of the four primary evolutionary paths highlights the focal points of China’s AI disruptive technology policy, namely intelligent services, talent training, information security, and scientific and technological innovation. The 14th Five-Year Plan (2021-2025) and Long-Range Objectives for 2035 prioritize initiatives such as “establishing an intelligent service ecosystem for the convenience and benefit of the people” “fostering innovation among talents” “developing a robust digital security framework” and “enhancing mechanisms for scientific and technological innovation”. These objectives align closely with the evolving themes of China’s disruptive technology policy, suggesting that intelligent services, talent training, information security, and scientific and technological innovation will remain central areas of focus in China’s future AI policy endeavors.

The dimensions of policy tools, policy actors, and policy themes form a complex system of policies for AI innovation and technology. Analyzing these dimensions aids in understanding the formulation and implementation of China’s policies on AI innovation and technology. Specifically, analyzing the relationship between policy tools and policy themes aids in understanding individual policy themes. The government can then select and adjust policy tools accordingly to achieve its policy objectives. Analyzing the relationship between policy actors and policy themes enhances understanding of the cooperative dynamics among diverse policy actors across different policy themes, thus facilitating more effective policy implementation.

4.4.1

Cross-analysis of “policy tools-policy themes”

Utilizing the coding outcomes of policy tools from Chapter 4.1 and the mined policy theme characteristics from Chapter 4.3 as data sources, a heatmap was generated to depict the cross-analysis results of policy tools and policy themes, as illustrated in Figure 6.

Figure 6.

In Figure 6, the color intensity of the heat map represents the number of policy tool codes under each policy theme. The darker the color, the more policy tools are utilized. Horizontally, the use of AI disruptive technology policies is relatively balanced for environmental and supply-side policy tools, but significantly imbalanced for demand-side tools. Vertically, the use of policy tools is balanced across the themes of “Intelligent Services” “Intelligent Governance” and “Intelligent Sharing and Mutual Trust”. However, the theme of “AI Theoretical System” lacks supplyside, environmental, and demand-side tools. Additionally, policies for “High-tech Talent Training” show limited use of supply-side and demand-side tools, mainly due to insufficient measures like capital investment, technological project support, government procurement, and scenario application. As for reasons, firstly, there are differences in priorities in policy formulation. Policymakers may favor immediate technology applications and industrial development, while neglecting long-term investment in basic theoretical systems and talent cultivation. Secondly, resource allocation has limitations. Economic resources and policy support may focus on existing successful models and fields, leading to insufficient support for research and talent cultivation in emerging fields like AI theoretical systems. The AI theoretical system is the cornerstone supporting scientific and technological innovation, inspiring new ideas, methods, and technologies. Therefore, future policies should promote theoretical research and cutting-edge development in AI. In addition, high-tech talent training is crucial for promoting technological innovation and industrial development. Future policies should focus on high-tech talent training, providing comprehensive support through both supply-side and demand-side policy tools. This will ensure a solid talent base and intellectual support for the country’s long-term development strategy.

4.4.2

Cross-analysis of “policy actors-policy themes”

Similarly, utilizing the quantitative collaboration of policy themes from Chapter 4.2 and the characteristics of policy themes from Chapter 4.3 as data sources, a heatmap is generated to illustrate the cross-analysis outcomes of policy actors and policy themes. This paper specifically presents the cross-analysis results of the top 20 policy actors demonstrating a relatively high collaboration density with policy themes, as depicted in Figure 7.

Figure 7.

In Figure 7, the color intensity of the heat map indicates the level of collaboration between policy entities under each policy theme. Darker colors represent closer collaboration. Horizontally, key entities in AI disruptive technology policies include the National Development and Reform Commission, the Ministry of Industry and Information Technology, and the Ministry of Science and Technology. In contrast, entities like the National Bureau of Statistics, the Ministry of Public Security, and the National Medical Security Administration show looser collaboration. Vertically, policy entities collaborate closely on “Intelligent Services” and “Intelligent Governance”, indicating their importance in AI development and their attention from various national departments. Policies on “Al Theoretical System” “High-Tech Talent Cultivation” and “Intelligent Sharing and Mutual Trust” urgently need more collaboration. The Ministry of Commerce and the State Administration of Taxation, in particular, lack coordination with other entities in these areas. The reasons can be summarized into three aspects: (1) Policy positioning and functions: The main roles of the Ministry of Commerce and the State Administration of Taxation may not directly involve promoting the development of AI theoretical systems and high-tech talent cultivation. (2) Resources and priorities: The Ministry of Commerce and the State Administration of Taxation may prioritize other urgent areas, such as economic development and tax collection. (3) Cross-departmental coordination challenges: Issues like bureaucracy, information asymmetry, and imperfect coordination mechanisms can lead to insufficient collaboration.