Apples to apples, fighters to submarines: comparative analysis of conventional capability-based signalling capacity through technologically weighted state arsenal indexing

While it seems intuitive to apply comparative arsenal analysis to conflict outcome prediction, compelling scholarship indicates materiel-focused analysis is unreliable. Carroll and Kenkel note that capability-based conflict outcome prediction performs only one percent better than a coin flip, while their own substantially improved method fares only 20% better (Carroll and Kenkel 2019). Biddle demonstrates convincingly that insight into conflict outcomes comes from states’ employment of their forces during combat, an approach that, to have predictive value, would require reliable estimates of how a state’s military would act during prospective conflict (Biddle 2004).

These lessons run into an empirical challenge: Most states do not use their arsenals for interstate conflict. Sarkees and Wayman’s exhaustive examination of interstate conflicts reveals that in the 60 years following World War II, fewer than 60 state governments – less than a third of the 188 accorded undisputed sovereign status by the United Nations – engaged in interstate armed conflict. In the preceding 60 years, over 120 distinct states engaged in such conflict over substantially longer durations (Sarkees and Wayman 2010). This observation may seem odd coming in the midst of a prominent interstate war sparked by Russia’s 2022 full-scale invasion of Ukraine; however, the very exceptional nature of this conflict highlights the continuing rarity of such occurrences compared to past periods (Feltman 2023). It remains true that the modern era sees most states purchasing and retaining conventional weapons that spend the vast majority – or entirety – of their existences unused in combat.

It is not clear that many states could employ their arsenals in any sustained way even if they were to commit to interstate conflict. An International Peace Institute survey of United Nations Peacekeeping Operations (UNPKO) suggests most states struggle to project and sustain even small fractions of their militaries over short distances for more than a few weeks (Coleman and Williams 2017). Nor is this challenge unique to the generally smaller and more developing pool that typically participates in UNPKO; a study by RAND concluded that the United Kingdom, France, and Germany—developed states with some prevalence of premier conventional armaments – would each be hard-pressed to marshal, deploy and sustain a single brigade of combat power within Europe for more than a month without the undertaking becoming the main effort of their respective militaries and eclipsing any capacity for other contingencies (Shurkin 2017). This capacity dearth is not limited to extra-regional power projection. India maintains the world’s largest ground force and faces acknowledged threats from neighbours Pakistan and China (Hackett 2023). Despite this, the country’s most recent in-depth ammunition management audit indicated that India would struggle to field army platforms in firing conditions for a local conflict for more than ten days (Report of the Comptroller and Auditor General of India 2017; Sen 2017).

Given that the majority of state-owned military hardware never sees combat and that many states would struggle to employ their arsenals, widespread procurement of combat systems without addressing logistical deficiencies suggests a major aim of acquiring weaponry is merely having it. Scholars identify weapon possession as the capability dimension of conventional strategic signalling capacity, where credibility (reputational willingness to employ weapons for strategic aims) and communication (explicit statements from the state to others) constitute the other two dimensions (Gerson 2009; Morgan 2012; Haffa 2018). Although this bears deeper examination as an example, consider that as of January 2022, much of Europe prepared to negotiate on Russia’s Ukraine-focused demands owing to a combination of status quo appeasement for commercial cooperation and the sheer size of the apparent disparity between Russian and Ukrainian capabilities (Coles et al. 2023). Paradoxically, employing the weapons it had amassed on Ukraine’s border squandered the signal strength that Russia could otherwise have leveraged.

The premise that conventional weapons contribute to a state’s strategic signalling capacity yields an avenue for comparative analysis. Where most weapons are never employed in conflict, all weapons (save those successfully concealed) contribute to signalling. The relative signalling contribution of a weapon is a less complicated phenomenon to estimate than its prospective combat use, an activity that entails innumerable factors. With this in mind, I survey selected methods of arsenal computation to derive lessons for signalling capacity estimation and identify precursor techniques for the CFPI method.

An impressive recent innovation in comparative arsenal analysis, the Distribution of Military Capabilities (rDMC) dataset, uses data from the International Institute for Strategic Studies’ (IISS) The Military Balance to code military technology distribution among 173 countries from 1970 to 2014 (Gannon 2021). While no public resource currently matches rDMC’s depiction of the prevalence of types of technology in state arsenals throughout this period, rDMC makes no distinction between systems within each technology type based on sophistication or effectiveness. Analysts can use rDMC to see which states have – for example – air defence missile systems, their quantities and how distribution over time changes. However, ageing, relatively incapable systems code identically to advanced systems that cost far more and arguably contribute to more compelling strategic signals. While this criticism is simple, an accessible, informative solution to the comparison problem is another matter. The ensuing paragraphs explore computational methods that attempt quality-based distinctions between weapons.

A majority of extant analytic methods attempting quality distinctions between conventional weapons purport to project their performance under certain combat conditions. The archetype of these is the venerable Lanchester set of models, which – despite being re-validated by RAND as highly informative for engagement modelling – undercuts its feasibility by assuming large-scale engagements involving simultaneously firing masses of weapons (Lanchester 1916; Darilek et al. 2001). Innovations in this tradition-modifying Lanchester’s concepts for guided weapons and modern defences similarly attempt attritive results, rather than inherent comparative value for the systems themselves, attracting criticism for unwieldiness (Hughes 1995; Lucas and McGunnigle 2003; Armstrong 2013).

Three techniques that distinguish themselves from the Lanchester and related conflict outcome methods are (1) the summation technique in the United States Naval Postgraduate School’s aggregated firepower score (AFS) method; (2) Dubois et al.’s algebraic incorporation of combat power potential in their Concise Theory of Combat Power; and (3) the coefficient weighting technique in the United States Army Concepts Analysis Agency’s Weapon Effectiveness Index (WEI) method (U.S. Army Concepts Analysis Agency 1991; Dubois et al. 1997; U.S. Naval Postgraduate School 2000).

The AFS method also attempts engagement outcome prediction, but approaches it distinctly from Lanchester and other attritive tools. While Lanchester and salvo models attempt to project casualties and survivors by matching weapon systems on each side of an engagement, the AFS method adopts the straightforward but elegant solution of coding values to different types of equipment, multiplying these by their quantity and then adding them to the scores of other systems to aggregate a score for all equipment arrayed in a given engagement (Naval Postgraduate School 2000). AFS is arguably far too reductive for predicting the outcome of an activity as complex as combat but provides an obvious precursor technique for a comparative method to estimate weapon systems’ inherent signalling value, rather than to predict their combat performance.

In their theory, Dubois, Hughes and Low express the potential firepower inherent in any weapon system as a component of a comprehensive model of combat power (Dubois et al. 1997). Isolating a facet of combat power that solely consists of the inherent potential firepower of a weapon system offers a proxy for signalling; the capability-based signalling value of a weapon logically resides in its potential for employment, potential being a property that does not require actual use to manifest.

Finally, the WEI method piloted by the now-defunct U.S. Army Concepts Analysis Agency differentiated between degrees of technological sophistication among weapons of the same tactical role with weighted coefficients (U.S. Army Concepts Analysis Agency 1991). A major limitation of the WEI was the need for recurrent re-evaluation by panels of experts with divergent views on the indexed systems’ effectiveness in combat, one of the shortcomings that that Ben-Haim partly mitigates by adding robustness (Ben-Haim 2018). Avoiding the complex task of engagement outcome prediction by focusing on signalling value contribution means a weighted coefficient concept can be used without constant re-evaluation for effectiveness.

The next section of this paper incorporates WEI’s weighting concept, AFS’ role-sensitive summation approach and Dubois et al.’s expression of potential into processes to compute relative signalling capacity contribution by conventional systems.

The CFPI method uses conventional firepower potential as a proxy for capability-based strategic signalling capacity. The following computational processes are intended only to abstractly score capability contributing to signalling. See Constraints, Trade-offs, and Possibilities at the conclusion of this section of the article for a summary of the distinctions between using indexed approximations of capability for signalling value and projecting engagement outcomes, which the CFPI never attempts to do.

The firepower potential of a set of conventional weapons is the sum of the products of each system’s role, technological sophistication, and quantity. In this approach, the CFPI builds on the precursor techniques of AFS, DuBois et al.’s algebraic expression of combat power, and the WEI method’s weighted coefficient approach. These techniques are expressed as follows: AFS=∑i=1nXiSiP→=f[αN,u→]WEI=cfF+cmM+csS$$\matrix{ {{\rm{AFS}} = \sum\limits_{{\rm{i}} = 1}^{\rm{n}} {{{\rm{X}}_{\rm{i}}}{{\rm{S}}_{\rm{i}}}} } \hfill \cr {\vec P = f[\alpha N,\vec u]} \hfill \cr {WEI = {c_f}F + {c_m}M + {c_s}S} \hfill \cr } $$

The AFS expression yields the total firepower assessed for weapons of type i assigned a relative firepower score of S_i and present in quantity X. Dubois et al. conceive of combat power, P, as a vectored quantity that exists as a function of potential combat power (u) and realising actions (αN). ‘Realising actions’ include all steps in the operation of combat systems to turn the potential power into actual power and by their number and complexity pose an acute challenge for any estimative mathematical model. A helpful simplification of the WEI expresses a weapon’s score as the sum of the firepower (F), mobility (M), and survivability (S) scores assigned to all weapons of a particular type once modified by a coefficient intended to compare specific models with a base model (Krondak et al. 2007).

Eliminating the aim of predicting combat effectiveness or engagement outcomes sidesteps the challenge of modelling ‘realising actions’ and means that only certain elements of these concepts apply to an index of strategic signalling value. Combining applicable concepts of the three methods means that the CFPI score – an approximation of the potential firepower inherent in the technology considering no other factors or actions – for a certain number of weapon systems of the same type and technological sophistication is expressed: Uo=αoe(Σth)$${U_{\rm{o}}} = {\alpha _{{\rm{oe}}}}\left( {{\rm{\Sigma }}{t_{\rm{h}}}} \right)$$

U is the potential firepower, o is a domain marker (air, land or naval) and e designates the type of system (e.g. destroyer or main battle tank). The numeric score in the index is the product of α, the weighted value attributes for the system (see Tactical Roles and Generational Tiers) and the sum of the quantities of all systems of that type and technological tier, t_h. In this and all subsequent expressions, the limits of summation are implicit as i = 1 through n, and the corresponding index and limit notations are omitted. Where there are multiple technological tiers among the same weapon type, these are accounted for by separate summation as follows: Uo=αoe(Σth1+Σth2)$${U_{\rm{o}}} = {\alpha _{{\rm{oe}}}}\left( {{\rm{\Sigma }}{t_{{\rm{h}}1}} + {\rm{\Sigma }}{t_{{\rm{h}}2}}} \right)$$

This expresses the CFPI score for a group of one weapon type drawn from two generational tiers of sophistication h₁ and h₂. The CFPI divides the global pool of major conventional weapons into five such groupings across the three conventional domains of air, land, and sea. The score focused on a single domain is expressed: Uo=Uoe1+Uoe2+…$${U_{\rm{o}}} = {U_{{\rm{oe}}1}} + {U_{{\rm{oe}}2}} + \ldots $$

This expression uses as many terms as necessary to account for all types of weapon categorised as belonging to the domain. To make this concrete, the following expresses the CFPI score of a state’s major conventional weapons focused on the air domain: Ua=Uai+Uam+Uag+Uad$${U_{\rm{a}}} = {U_{{\rm{ai}}}} + {U_{{\rm{am}}}} + {U_{{\rm{ag}}}} + {U_{{\rm{ad}}}}$$

The a subscript represents the air domain, while other subscripts represent weapon systems whose firepower potential focuses on that domain: i denotes air superiority fighters (interceptors); m denotes multirole fighters; g represents ground attack aircraft; and d represents air defence missile systems. The total CFPI for a state arsenal incorporates the firepower potential-possessing (and therefore signal value-contributing) systems focused on all three domains, expressed as follows: Up=Ua+U1+Un$${U_{\rm{p}}} = {U_{\rm{a}}} + {U_1} + {U_{\rm{n}}}$$

The subscript p denotes conventional firepower across all domains, making U_p the overall notation for a state’s CFPI score. The other subscripts correspond to domains: a for air, l for land and n for naval (‘naval’ used in place of ‘sea’ for precision because of the inclusion of naval aviation systems that resemble fixed-wing systems categorised as air-focused).

The preceding paragraphs algebraically express the process of indexing a state’s conventional arsenal into firepower potential scores. To enumerate these algebraic expressions, we must compute a value for the coefficient α. The next subsection details enumeration of α with proxy values for the tactical role and relative technological sophistication of each system in the CFPI.

The CFPI derives a value for each system type’s intended tactical role and a generational tier coefficient for technological sophistication. The overall coefficient applied to each system quantity is expressed as follows: αthoe =Ghroe$${\alpha _{{\rm{thoe}}}} = {G_{\rm{h}}}{r_{{\rm{oe}}}}$$

The subscript t_h denotes technological sophistication of degree h. G is the constant multiplier associated with degree h. The variable r represents the conventional fire-power potential – unmodified by technological sophistication – for all systems e in domain o. Numeric values for r and G permit numeric CFPI scores.

To estimate r-values for a given weapon type, the CFPI first computes a ‘raw’ firepower potential and then weights this for the system’s advertised versatility in releasing its munitions and ostensibly engaging other systems. For brevity, I refer to these three factors as the normalised yield ratio, release versatility, and engagement versatility. The following paragraphs derive each of these and concretely illustrate the process with the multirole fighter weapon type.

‘Raw’ firepower potential is the product of a system’s single-engagement explosive yield, index munition range, and operational range (or two-hour travel range in the case of naval vessels) with all ranges expressed in hundreds of kilometres. In every system’s case, this product is multiplied by a scaling constant of 0.036 and rounded to the nearest whole number solely to achieve a more intuitive scale across the CFPI. In the following expression – not reflecting these last two scaling steps – m represents the index munition (a munition commonly employed by the index system of this weapon type). Raw FP = Engagement yield moe * Range in 100s of kmmoe * Operational range in 100s of kmoe$${\rm{Raw FP}} = {\rm{Engagement yield}}{{\rm{}}_{{\rm{moe}}}}^{\rm{*}}{\rm{Range in 100s ofk}}{{\rm{m}}_{{\rm{moe}}}}^{\rm{*}}{\rm{Operational range in}}100{\rm{sofk}}{{\rm{m}}_{{\rm{oe}}}}$$

For engagement explosive yield, the CFPI uses estimated energy yield in megacalorie (Mcal) TNT equivalence of the index munition’s explosive mass, assuming it behaves consistent with tritonal explosive’s properties (a mixture of 80% trinitrotoluene and 20% aluminium commonly employed in modern munitions and releasing approximately 18% more energy than a comparable mass of TNT) (U.S. Department of Energy 2002). This assumption uses the U.S. National Institute for Standards and Technology’s TNT equivalence convention of one gram of TNT releasing 4.184 kilojoules or one kilocalorie; one kilogram of tritonal explosive would yield approximately 1.18 megacalories (U.S. Department of Commerce 2008). This, in turn, is multiplied by aimed releases of the index munition by the index system in the space of a single minute. Engagement yield moe = Tritonal mass equivalent in kg*1.18 Mcal * Aimed releases in 1 minute

The following steps compute the r-value for multirole fighter aircraft. The CFPI uses the American F-16C (production Block 40 and later) as an index system for multirole fighter jets and the GBU-12 precision air-to-ground bomb as the index munition. Engagement yield GBU−12=87kg * 1.18Mcal∗1 release =102.66

Next, we multiply the engagement explosive yield by the index munition range and the index system range. Multiplying this product by the scaling coefficient of 0.036 and rounding provide the normalised yield ratio, the computed firepower potential precursor of tactical role value. RawFPam= Engagement yield GBU−12∗ Range in 100skmGBU−12* Range in 100skmmF−16C/Blk40+ Raw FPam=102.66∗0.25∗8.6=220.72 Normalised yield am==0.036* Raw FP am=0.036 *220.72=7.95≈8$$\matrix{ {RawF{P_{{\rm{am}}}} = {\rm{Engagement yield}}{{\rm{}}_{{\rm{GBU}} - 12}}^ * {\rm{Range in}}100{\rm{sk}}{{\rm{m}}_{{\rm{GBU}} - 12}}^{\rm{*}}{\rm{Range in}}100{\rm{skm}}{m_{{\rm{F}} - 16{\rm{C}}/{\rm{Blk}}40 + }}} \hfill \cr {{\rm{Raw}}F{P_{{\rm{am}}}} = 102.66 * 0.25 * 8.6 = 220.72} \hfill \cr {{\rm{Normalised yield}}{{\rm{}}_{{\rm{am}}}} = = {{0.036}^{\rm{*}}}{\rm{Raw FP}}{{\rm{}}_{{\rm{am}}}} = 0.036{\rm{*}}220.72 = 7.95 \approx 8{\rm{}}} \hfill \cr } $$

The last step in deriving role value for a weapon system type is to apply ordinal weight for release versatility and engagement versatility. Release versatility expresses the index system’s advertised adaptiveness to target behaviour when releasing the index munition, while engagement versatility accounts for two factors: (1) whether the index system is ordinarily intended to engage in one or multiple domains and (2) whether the index system is ordinarily configured to engage the systems designed to neutralise it. Versatility is multiplicative because it increases the reach and impact of the potential firepower inherent in an index munition as employed by an index system; in signalling terms, this may be interpreted as abstracting the theoretical spatial threat telegraphed by the platform in question. Understanding versatility as a degree-of-freedom-determined spatial threat may help the reader to appreciate the considerable signalling value that the CFPI accords to aircraft-carrying vessels. Table 1 offers a rubric for determining release and engagement versatility.

Remembering that the CFPI’s tactical role value for a weapon system type is the product of normalised yield, release versatility, and engagement versatility, the tactical role value of multirole fighters thus computes. ram= Normalised yield am∗ Release versatility am∗ Engagement versatility am=8∗2∗2=32$${r_{{\rm{am}}}} = {\rm{Normalised yield}}{{\rm{}}_{{\rm{am}}}}^{\rm{Release versatility}}{{\rm{}}_{{\rm{am}}}}^ * {\rm{Engagement versatility}}{{\rm{}}_{{\rm{am}}}} = 8 * 2 * 2 = 32$$

Table 2 contains the weapon types, index systems, normalised yields, versatilities, and r-values of the CFPI method, which does not consider operational readiness, ammunition availability, environmental effects, crew proficiency or any other factors. Where possible, index systems selected are examples of the second (or competitive) generational tier. The CFPI uses index systems in a markedly different way from the WEI precursor, which compared every single other scored system to the index system. Instead, the CFPI systems set the tactical role value for all systems of one type. Since CFPI scores are only abstract representations of relative capability-based contributions to signalling (rather than to performance), specific technical differences between same-type, same-technological generation systems are superfluous.

To enumerate G_hr_oe (setting the value of the coefficient α_thoe and enabling calculation of numeric CFPI scores), the CFPI method employs five different weighted degrees of relative technological sophistication with associated descriptors: obsolete, ageing, competitive, advanced, and cutting-edge. These correspond to the four-tier technological grading employed by the U.S. Army’s Worldwide Equipment Guide (WEG) as of 2021 with several modifications in Table 3. An important difference between WEG tiers and CFPI tiers is that the WEG’s tier numbers decrease as sophistication increases, with tier 1 being the most sophisticated and tier 4 the least sophisticated. CFPI tiers increase directly with the degree of sophistication for two reasons: (1) Although the WEG was useful in designing the CFPI, the two need not be perpetually linked; and (2) rather than recalibrating tiers in an inverse tier-number scale, the CFPI can add new systems to appropriate existing tiers or create new tiers as generations of technology emerge. CFPI tiers are generally anchored by fighter jet generations, although fighter jets are not the only example of major conventional systems divided into generally agreed generations of sophistication (Hebert 2008). The descriptors sit on a sliding scale depending on the year of analysis relative to the introduction of the first fifth-generation fighter aircraft in 2005, and I anticipate the addition of further increments of that scale to reflect substantial upgrades to fifth-generation fighters and the introduction of the first sixth-generation fighter (Hebert 2008). The arbitrated value of these tier distinctions reflects an estimated one-third increase in sophistication granted by a substantial mid-life upgrade to a platform, while a new generation of system roughly doubles the sophistication of the previous generation (Mo et al. 2015).

Like the WEG tiers, CFPI tiers correspond roughly to introduction dates of weapon systems exhibiting newer technological characteristics. Using weighted coefficients for sophistication and representing capability-based contribution to strategic signalling value, rather than conflict outcomes, precludes the need to compare or adjust systems toe-to-toe. The CFPI thus understands possession of any system of a particular role in a particular tier the world over to contribute the same capability-based element to strategic signalling, enabling comparative analysis across the global system of state arsenals. I next flesh out an example of such comparison through CFPI scores for the air-focused components of Chinese and Russian arsenals in 2023.

In this brief demonstration, the computational procedures from the previous section generate index scores for the conventional weapon systems of the People’s Republic of China and the Russian Federation in the air domain as of 2023. Beginning with the expression for overall CFPI score, I expand the expression for score within a single domain (air) and expand and compute CFPI score for a single system type (multirole fighters). I then illustrate how even one domain’s CFPI score for two states allows comparative capability-based signalling analysis that previously would not have been possible. The expression for total CFPI score is as follows: Up=Ua+U1+Un$${U_{\rm{p}}} = {U_{\rm{a}}} + {U_1} + {U_{\rm{n}}}$$

Focusing on the air domain: Ua=Uai+Uam+Uag+Uad$${U_{\rm{a}}} = {U_{{\rm{ai}}}} + {U_{{\rm{am}}}} + {U_{{\rm{ag}}}} + {U_{{\rm{ad}}}}$$

Multirole fighters specifically: Uam=( G0(Σt0am)+G1(Σt1am)+G2(Σt2am)+G3(Σt3am) +G4(Σt4am) )ram$${U_{{\rm{am}}}} = \left( {{G_0}\left( {{\rm{\Sigma }}{t_{0{\rm{am}}}}} \right) + {G_1}\left( {{\rm{\Sigma }}{t_{1{\rm{am}}}}} \right) + {G_2}\left( {{\rm{\Sigma }}{t_{2{\rm{am}}}}} \right) + {G_3}\left( {{\rm{\Sigma }}{t_{3{\rm{am}}}}} \right)} \right.\left. { + {G_4}\left( {{\rm{\Sigma }}{t_{4{\rm{am}}}}} \right)} \right){r_{{\rm{am}}}}$$

The aforementioned results from expanding the expression for a single system type to include systems at each of the five generational tiers of the CFPI. Tables 4 and 5 list multirole fighter inventories of China and Russia in the year 2023 per the International Institute for Strategic Studies’ The Military Balance, an resource that annual estimates weapon quantities in the arsenals of over 170 states. Note that the data – lists of platforms and quantities – are incomprehensible to readers lacking expertise in the designations of these weapons, and even those readers with some familiarity may lack a command of the variants of each fighter.

Faced with the raw data, an analyst unfamiliar with each platform designation would be limited to unhelpful techniques like comparing the number of multirole fighters in each inventory (an unfortunately common practice). At this point, it is only apparent that China’s 2023 arsenal contained more multirole fighters and that there is some model overlap between the two states. To avoid such underwhelming conclusions, analysts can either abandon the pursuit or commit considerable effort to gaining familiarity with the seemingly endless nomenclatures of conventional weapons. A downside to the latter approach is that the ensuing analysis risks being incomprehensible to its intended audience.

To make comparisons that do not encounter granular barriers to entry, we can score the systems using the CFPI. Table 6 lists a selection of multirole fighters currently coded in the CFPI method found in the arsenals of the United States, China, Russia, North Korea, and Iran with generational tiers resulting from WEG conversion (Table 3).

Using the values in Table 6, we can compute values representing the conventional capability-based signalling afforded Russia and China by each state’s multirole fighters in the year 2023. Tables 7 and 8 demonstrate this.

Having followed the CFPI scoring steps, some more helpful conclusions follow. We could already observe that Russia’s inventory of multirole fighters was considerably smaller than China’s, but we can additionally observe that it is only marginally less technologically sophisticated. The difference between the capability contribution of multirole fighters to the signalling value of each state’s arsenal is then roughly proportional to the numerical difference, a conclusion that we could not make with any real confidence before scoring. Table 9 lists data and scores for the entire air-focused components of Chinese and Russian conventional arsenals in the year 2023.

The data suggest instructive conclusions concerning the two states’ capability basis for air-focused conventional signalling. China’s airpower arsenal exhibits two principal repositories of firepower potential: multirole fighters and air defence missiles. This suggests a relatively even prioritisation of deterrence by unambiguously defensive systems (air defence) and systems whose offensive potential for power projection lends them an ambiguous quality. Russia, on the other hand, has a clear centre of gravity for its air-focused firepower potential: its air defence missile systems. Restricting our consideration for the moment to air-focused CFPI scores, the data do not suggest a robust Russian airpower projection signal relative to that inherent in China’s inventory.

This study’s method aims to enhance the pursuit of capability-based balance of power analysis by enabling estimative comparisons of conventional strategic signalling value of state arsenals, with distinct constraints and possibilities. These include (1) the abstract nature of indexes, (2) the inability to consider unconventional capabilities or systems not listed, (3) the impossibility of using CFPI scoring alone to predict conflict outcomes with any confidence, (4) the risk that changing technology will constrain CFPI’s uses to historical analysis and (5) the possibilities of using CFPI scoring to enhance other avenues of defence analysis.

I simply cannot claim that the CFPI on its own enables anysort of precise measurement of the aggregate quality of state conventional weapon systems; it only improves incrementally on the current state of comparative analysis, which is characterised by a practical inability to make quality-based comparisons between weapons outside methods intended to project their effectiveness in combat with questionable conclusions. Just as gross domestic product (GDP) fails to capture nuances beyond an economy’s size and easily masks sector-specific weaknesses or strengths, the CFPI enables analysts without granular conventional weaponry knowledge to discern the broad contours of capability-based signalling capacity for balance of power analysis. As mentioned previously, the CFPI does not take into account any of the myriad factors needed to operate these weapon systems effectively such as crew availability and skill, ammunition and maintenance.

By its very nature, the CFPI is unable to capture signalling contributions of military systems that are not conventionally armed. These include nuclear platforms (aircraft, submarines, and missile systems primarily intended for nuclear weapons delivery are excluded from CFPI tables), logistical systems that could contribute to strategic signals (particularly large-scale airlift or sealift systems), and mobility systems (mine warfare vessels and vehicles). While these blind spots are understandable given the method’s firepower potential focus and the observation at this study’s outset that most states procure far more combat hardware than their relatively weak logistical systems can support, they are blind spots nonetheless and analyses using the CFPI should appropriately caveat or avoid any ascriptions of intent or capability.

The CFPI is also limited to the availability of quantity and type data for the systems that it scores. Table 3 does not include sophistication descriptors prior to 1990 because quantities in the underlying dataset (IISS’ The Military Balance) are mostly incomplete in its earlier years of publication.

The CFPI absolutely cannot on its own support conflict outcome prediction with any degree of confidence and even with multiple tools conflict outcome prediction is a fraught pursuit. It may seem ironic that having noted the criticism that has befallen techniques like aggregated firepower score and WEI/WUV, I root CFPI’s tactical role value computation in reductive approximations of explosive yields by index systems releasing index munitions under wholly theoretical conditions. However, I do not propose – and strongly caution against – applying normalised munition yields from CFPI r-values toward engagement outcome prediction without careful consideration and adoption of additional computational techniques. The CFPI projects neither damage nor survivability prospects and, in fact, does not incorporate engagement modelling at all beyond an initial proxy for the capability component of ‘capability-based’ signalling capacity. There are simply too many other factors – possibly an unknowable number – that contribute to combat power potential.

Developments in military technology, particularly in the area of remote and automated systems, may eventually constrain the CFPI’s current framework of major conventional systems to historical analyses. The proliferation of loitering munitions and the tendency of emerging doctrines toward swarming lethality distributed among growing numbers of smaller systems could signal a reprioritisation of traditional tactical roles even more dramatic than the aircraft carrier eclipsing the battleship during World War II (Atherton 2021; Holmes 2022). Even as I acknowledge this horizon as carrying serious implications for the type of analysis I propose, estimating the proximity of this milestone is well outside the scope of this paper.

These caveats notwithstanding, I believe the CFPI solves real problems facing would-be military balance of power analysts. Accepting the premise that most of the world’s conventional weaponry serves a signalling contribution role most of the time, CFPI scoring represents an accessible proxy for this signalling and in a snap analysis for capability more broadly in the conventional arena. The CFPI can also combine with other concepts to make well-worn avenues of defence analysis more informative.

While the CFPI on its own cannot model engagements and predict outcomes, it may nevertheless serve as a basis for other analysts to augment or develop their own engagement models if they can resolve the deficiencies noted earlier through their own techniques. Tables 2 and 3 represent a novel method of enumerating major conventional weapons. It is entirely possible that other analysts may find this enumeration useful to include in their own richer methods more focused on modelling conventional engagements.

Assuming that when states purchase weapons, they are usually purchasing the capability-based component of conventional signalling capacity, more meaningful analysis of procurement spending becomes possible. Even when procurement spending is disaggregated from total defence spending – a constantly cited figure that typically lacks information to be useful – the inability to make comparisons between state arsenals impedes a full appreciation of procurement analysis.

Although this section and the next focus on CFPI scoring for comparative analysis between states in the same year, the CFPI also enables analysis of state arsenals over multiple years. This may simply describe and compare change over time or support procurement analyses. The change in a state’s CFPI score is expressed: ΔUp=Up(y)−Up(y−1)∘$${\rm{\Delta }}{U_{\rm{p}}} = {U_{{\rm{p}}({\rm{y}})}} - {U_{{\rm{p}}{{({\rm{y}} - 1)}^ \circ }}}$$

In this straightforward, recursive expression, a change in the CFPI score is the difference in the CFPI score between the year of analysis y and the previous year y-1. This is arguably not yet suitable for linking procurement spending to ΔU_p since procurement is not instantaneous. Embracing the approximate natures of proxy values and indexes, a staggered recursive value of CFPI change across multiple years over the expenditure of previous years compensates for lag. To model an example of this, we can draw on a 2018 RAND study that found an average of 3 years between intermediate design, production, and fielding milestones in the U.S. acquisition system roughly analogous to those of purchase agreement and inventory receipt for states importing weapons (Light et al. 2018). Using this estimate, a staggered recursive expression for CFPI score change over procurement spending and across time would be: ηUp=ΔUp(y)+ΔUp(y−1)+ΔUp(y−2)X(y−1)+X(y−2)+X(y−3).$${\eta _{_{U{\rm{p}}}}} = {{{\rm{\Delta }}{U_{{\rm{p}}({\rm{y}})}} + {\rm{\Delta }}{U_{{\rm{p}}({\rm{y}} - 1)}} + {\rm{\Delta }}{U_{{\rm{p}}({\rm{y}} - 2)}}} \over {{X_{(y - 1)}} + {X_{(y - 2)}} + {X_{(y - 3)}}}}.$$

Analysis employing this expression requires longitudinal CFPI scores and procurement spending data, and probably cannot work for states that indigenously produce their weapons (particularly with substantial research and development). The number of years used for a procurement efficiency calculation is not fixed at three as in the previous example but would necessarily vary from state to state and pose challenges of distinguishing procurement spending (that subset of defence spending that is solely used to purchase weapons) from development spending (research, prototyping, testing, evaluation, and so on). Within these constraints is an avenue for comparative efficiency analysis of conventional weaponry procurement by arms-importing states. Assuming procurement spending can be accurately isolated – a challenge given that many states withhold such figures from public release – we may describe states updating their inventories with more competitive systems, faster and over shorter periods of time as procuring more efficiently compared to other states. Comparative analysis thus does not require selection of the optimal time interval for each state but simply application of the same time interval to both (or all) states being compared.

The Global Firepower Index enjoys widespread citation by journalists and governments despite the opacity of its methodology. The next few paragraphs examine GFI’s ratings for the states mentioned in the NDS while reviewing a sampling of ostensibly serious journalistic, professional, and academic settings glossing over the non-rigorous nature of GFI to cite these rankings. I further illustrate the problem raised in the introduction, namely, that a dearth of accessible methods for comparative analysis exacerbates tendencies of would-be analysts to cite sources like GFI.

GFI purports to rank states by overall ‘military strength’, ‘airpower’, ‘land forces’, and ‘naval forces’ (Global Firepower Index 2023). Figure 1 is a normalised depiction of these rankings where each state’s score is depicted as a percentage of the best score awarded by the site in each category.

Fig. 1:

Site rankings put the United States first overall, with Russia a close second and China a close third. As GFI does not publish its methods, readers must wonder how the individual domain rankings generate overall rankings. Rankings for ‘airpower’, ‘land forces’, and ‘naval forces’ simply entail counting military aircraft, main battle tanks, and naval vessels, respectively. Despite this approach, there is no identifiable relationship between the domain ratings and the overall ratings. GFI puts North Korea ahead of Iran in all three domains but ranks Iran ahead of North Korea overall. Incidentally, GFI ranks North Korea ahead of the United States in ‘land forces’ and ‘naval forces’. GFI’s data breadth has increased markedly in recent years, with individual country pages offering diverse (but still opaque and unsourced) rankings for figures like manpower, readiness, financial data and more (Global Firepower Index 2023). It is possible that the interaction of these opaque ranking and the individual domain scores explain the somewhat counterintuitive rankings, but this explanation is not accessible to anyone who might wish to use the system for any analytic purpose other than parroting GFI’s rankings.

The aforenoted issues do not dissuade citation of GFI’s analysis in many journalistic, professional and even academic settings. Business Insider cited GFI’s 2023 rankings to depict Russia and China as close on the heels of the United States (Baker and Spirlet 2023). The Association of the United States Army (AUSA), the principal professional organisation for current and former American soldiers and officers, cited GFI in asserting that the United States trails Russia and China in land power (Association of the United States Army 2019). The instructional materials for ‘America’s Weapon Systems’, a short-form course at the College of William and Mary, cites GFI to state that ‘Russia overwhelmingly leads’ in the arena of conventional land systems (Hickok 2018).

Oleksiy Kuzmenko’s reporting indicates serious security scholars and defence analysts either have not heard of GFI or do not take it seriously. Nevertheless, GFI and the malleable narratives implied by its rankings still proliferate through citations in settings assumed to be reliable. The widespread use of GFI’s rankings offers a prestige boost—or perhaps raises alarm—for Russia and Iran. GFI has consistently ranked Iran’s military ahead of Israel’s, a fact noticed by both states’ journalistic communities (Iran International 2019; Winston 2019). While this study does not score Israel’s arsenal, the next subsection paints a starkly different landscape for Russia and Iran than does GFI and advances more transparently informed conclusions.

I focus on depicting comparative results of CFPI scoring for the United States, China, Russia, North Korea and Iran using arsenal data from the 2023 edition of the IISS’ The Military Balance. Figure 2 depicts overall and domain-specific scores for the five states, while Table 10 lists each state’s score derived from each the 27 system types.

Fig. 2:

It is immediately apparent that the CFPI suggests a dramatically different set of capabilities contributing to conventional strategic signals compared to the popular GFI portrayal. The core of this difference is the United States’ greater concentration of systems – particularly naval – that the CFPI accords higher role scores and technological tiers. Figure 3 depicts technological composition of each state’s arsenal in system counts (with no tactical role weighting). Figure 4 shows the derivation of each state’s score from systems of each degree of sophistication. Figures 3 and 4 demonstrate why simply counting platforms muddies insights into arsenal composition. Finally, Figure 5 scores only conventional firepower potential for systems suited to extra-regional projection and thus extended deterrent signals. These include extended flight-capable fixed-wing aircraft, blue-water naval vessels, and ship-based naval aviation (see Table 10).

Fig. 3:

Fig. 4:

Fig. 5:

The data of the preceding charts enable us to revisit the 2022 U.S. NDS’ premises. Rather than embarking on an in-depth analysis of each chart – the aim of this article is to contribute the CFPI method and illustrate possibilities, not a deep-dive into the NDS’ outlook – I briefly distil insights into the magnitude and nature of the cited threats and prospective investments.

CFPI scoring combined with readily available macroeconomic data suggests that if any country can realistically contemplate future conventional parity with the United States, it is China. Even this popular projection is callsed into question by China’s economic indicators suggesting signs of stagnation with its gross domestic product at approximately 70% that of the United States (World Bank Open Data Tool; Sharma 2022; Stokes 2023).

The yawning gap in conventional posture incentivises the other name-checked states to pursue unconventional advantages. For North Korea and Iran, nuclear arms represent an attractive insurance policy. Russia, with its legacy ability to advance a prestige narrative by showcasing some premier capabilities, is nonetheless also incentivised to exploit capabilities in the difficult-to-at-tribute realms of offensive cyber and disinformation operations (Cunningham 2020; Lilly and Cheravitch 2020). Comparative advantage for Russia in unconventional arenas is further heightened because of conventional losses incurred during its full-scale invasion of Ukraine (Watling et al. 2023). Reports of Russia’s collaboration with both Iran and North Korea underscore the proclivity of these three countries to seek unconventional competitive advantages (Geranmayeh and Grajewski 2023; Regan et al. 2023).

While the United States is free to pour resources into politically popular and technically straightforward efforts to further bolster conventional advantage, the reality is that America’s arsenal uniquely postures it to send robust extra-regional extended conventional deterrent signals. This means that investment in conventional capabilities – while necessary if the U.S. prioritises a conventional posture edge over China – probably crosses a point of diminishing returns given the extant capability gaps and the astronomical price tags of advanced air and naval systems. The most lucrative avenue for the U.S. to link strategy and resources to keep China’s capability-based posture in check may be to arm allies in the region; see Australia’s abandonment of longtime strategic ambiguity in agreeing to receive nuclear-powered submarines from the United States implicitly to balance China (Pei 2021).

Setting aside the largely diplomatic challenges of managing nuclearisation by North Korea and Iran, CFPI scoring suggests that dollar for dollar, more promising applications for increased and sustained investment lie in counter-cyber and counter-disinformation measures. An irregular warfare annex to the previous (2018) NDS particularly noted Russia’s proclivity toward and proficiency with disinformation and cyber operations, which suggests that at least some within the Pentagon share this perspective (U.S. Department of Defense 2020).

This all confines the scope of the CFPI scoring-informed analysis to threats cited by the NDS. Other voices argue climate change and pandemics represent risk sources that would benefit from some share of U.S. spending otherwise pouring into extending already wide conventional advantages. If comparative arsenal analysis represented a great enough challenge to justify the writing of this paper, devising a framework for fiscal value judgments across completely disparate realms of policy justifies authorship of multiple libraries of books.