Interoperable, adaptable, information exchange in NATO coalition operations

The purpose of the inter-network routing protocol is to form a common network based on a number of network resources. The tactical edge network resources that may be present in a coalition operation are as follows:

One or more wireless networks use a standardised waveform (e.g., NATO wideband or narrowband waveform) as selected by FMN. These waveforms can provide interoperability over the air between all or a subset of the units of the operation.

Our routing protocol work fits the Interconnect-overlay architecture, which is one of three routing architectures defined by IST-124 (Hauge et al. 2019). The Interconnect-overlay architecture utilises a tier-1 overlay routing domain in order to connect a set of tier-2 routing domains as shown in Figure 3. Only a subset of the military platforms present in tier-2 participate in the overlay. In Figure 3, platform A and B of each nation participate in the overlay routing domain. The overlay nodes are located on what we call Interconnection platforms. These are platforms on which two or more of the different network resources in tier-2 can be accessed and thus provide interconnection between these networks. Figure 3 shows an example network where two different standardised waveforms (i.e., the waveforms are common to the coalition partners but not necessarily to all) are being used as well as four different nations providing a share of their national tactical edge networks for the federated coalition network. Only a subset of the national military platforms participates in one or both of the standardised tier-2 networks. In order to use the Interconnect-overlay architecture in a coalition environment, nations need to agree on a common overlay routing protocol as well as on the information that should flow over a routing information exchange interface between tier-2 and tier-1 routing domains.

Fig. 3

The Depth First Search (DFS) Routing protocol (Landmark et al. 2015; Hauge et al. 2020) was used as the tier-1 routing protocol in our work. This is a reactive protocol that does not proactively maintain a routing table but searches for a route when it is needed. The protocol was designed to be narrowband-aware, which means that it is aware of tier-2 protocol domains with little data capacity and can minimise the signalling overhead over those networks. By design, the protocol can also be made to support a range of different policies for how to build routes and utilise the available heterogeneous network resources. As part of the FIST project, Kongsberg Defence and Aerospace (KDA) created a new implementation of the DFS protocols. This implementation also includes the support for very flexible policy routing.

The reactive DFS protocol performs a depth-first search for the route to the destination. This contrasts with the breadth-first searches performed by most reactive protocols, such as Ad hoc On-demand Distance Vector (AODV) (Perkins et al. 2003). The reason for choosing a depth-first search was to better be able to direct the search in order to avoid unnecessary searches over low-capacity networks as well as providing a very flexible policy routing. The basic operation of the DFS protocol is shown with the signalling example of Figure 4. Here, numbered arrows indicate message exchange, whereas the others indicate the state of the DFS state machine.

Fig. 4

The DFS protocol used in the tier-1 domain can provide an end-to-end route traversing a number of tier-2 networks to support the following cases:

The coalition network should be able to route traffic from one nation's national network to another nation's national network.

Successful proof of concept test for the first three cases above was reported in (Hauge et al. 2020). For the work presented here, we have extended the support of the implemented protocol with the fourth case, as well as improved the basics of the routing protocol. This extension gives the necessary functionality to provide detailed information to the network awareness service of Figure 2. The CLO approach enables the middleware and services to adapt to the current network resources as further described in Section 4.

We chose to implement the policy functionality of the protocol with two separate assignments: (1) The coalition must agree on a set of policy rules. These rules and the identification of the rules should be common to all coalition partners of the mission. The rules are not mutually exclusive. They are ordered in a list at the routers, and traffic is tagged according to the first matching rule in the list. (2) Each coalition partner decides locally which rules to allow or deny over which network connection in the share of their national network that is provided for coalition use. The coalition partners should agree on a common allow/deny setting for the standardised waveforms that are common to several partners. This design allows implementing rules and the means to identify the traffic that matches a rule to be separated from the task of tagging the different tier-2 network interfaces with the desired allow/deny policy for the different rules.

In an example mission, we have three traffic classes defined, with associated rules as shown in Table 1.

It must also be defined how traffic can be identified to match a specific rule. In the current DFS protocol implementation, any chosen combination of the classical network fields, e.g., source and destination IP address, port number, protocol type and the type of service (TOS) field in the data packet can be used. This can be extended to include protocol extensions or other identification means in future work. For the experimentation, we have chosen to use the TOS field to assign a policy rule to the traffic flows.

Next, the allow/deny policy for each interface to the tier-1 routers must be set for each for these policies. Consider router number 3 in the tier-1 routing domain in Figure 3 as an example. This router is located on Interconnection platform A in Nation 4's network and Nation 4 administers this router. The router has three interfaces that all participate in the tier-1 overlay routing domain. In addition, it has an interface to Nation 4's national network where the clients from Nation 4 that want to use the coalition network are attached. Nation 4 decides to set the allow/deny policies shown in Table 2 for its three types of interfaces.

The allow/deny policies for the standardised waveforms should be agreed on between all partners that have platforms that can participate in these common networks.

In the current DFS protocol implementation, the allow/deny policies for each interface are statically set in a configuration file in each router. Future work can combine this with modules that monitor the connections or that have a radio to router interface (e.g., Dynamic Link Exchange Protocol (DLEP) (Ratliff et al. 2017)) that can update the deny/allow policies according to traffic load, etc.

Figure 5 shows an example of how two example routes with the rules above can be found in vignette 3 in the Anglova scenario. In this case, we’re streaming a video with good resolution from the unmanned aerial vehicle (UAV) of one of the Norwegian platoons to the deployed German HQ. When the German platoon moves closer to the Norwegian platoon, line of sight to the German HQ is lost, and with that the last available high data rate, connection to the HQ is lost and only Rule 1 and Rule 2 traffic can now be supported.

Fig. 5

Current DFS protocol software with the policy routing functionality included was tested together with two other Interconnect-overlay protocols for the tactical edge for FMN at CWIX 2021. Our test partners were KDA from Norway, our FIST partner Fraunhofer Institute for Communication, Information Processing and Ergonomics (FKIE) from Germany and also Netherlands Organisation for Applied Scientific Research (TNO) from The Netherlands and International Business Machines (IBM) from Great Britain. The test setup for CWIX 2021 was very similar to the one described here (Hauge et al. 2020). Figure 3 shows the network that was used for the tests. All DFS routing tests were performed with the eight DFS routers of tier-1. The traffic was carried by the tier-2 network types, as shown in Figure 3. Successful proof of concept tests with traffic flows between all participating nations for the four cases described above were tested. For the fourth test case that was not conduced in (Hauge et al. 2020), we showed that the DFS protocol in the tier-1 routing domain was able to set up different routes when different policy rules were invoked. We also showed that when the high data rate connection was removed, such that a route fulfilling Rule 2 could not be found, this flow was stopped, but flows invoking Rule 1 were sustained. A report from these CWIX tests can be found in ‘Section 7.3 Objective 3. Coalition Routing Interoperability’ of (CWIX-21 2021). For future work, there is a need to do performance tests to get a better understanding of the overhead of this protocol in different scenarios.

We built an experimental adaptation middleware, which can digest and exploit network awareness information using a well-defined API. REST (standard connector)+NetJSON (standard data format), as mentioned above, was the approach to realising and providing such network awareness information (NetJSON 2015). For proof of concept, we conducted several tests in consuming network awareness information. We used the router data for service adaptation in our national prototype middleware.

In principle, the middleware can adapt any type of capability or service. However, in this initial prototyping, we limited ourselves to three services that would serve as examples for the adaptation enabled by the middleware.

The services we provided were BFT with the NATO Friendly Force Information (NFFI) format (NATO 2017), JChat (which is realised by the standard XMPP (XMPP 2004) protocol) for chat and full motion streaming video. These services were based on existing services. The adaptation middleware used network knowledge to configure (and on-the-fly reconfigure) the services when facing variations in resources in the underlying tactical links. The aim was to provide a targeted and carefully considered adjustment of the services to provide a trade-off between functionality, timeliness and availability given the resource situation.

The adaptive middleware, shown in Figure 2, was developed as part of a software contract between FFI and a third-party developer, Sysint AS (Lindholm and Wuttudal 2021). The middleware, called Sysint Adaptive Service Control (SASC), was built on the Norwegian emerging tactical platform TYR, which is a Windows-based platform. SASC consists of a Windows service (SASC.Service) and a manager user interface (SASC.Manager) for configuration of the services and ruleset. The middleware supports gathering network status from a tactical router with the network awareness service API. The middleware currently supports configuring the three services mentioned above, as follows:

Reconfiguring the network firewall service to block/unblock attachments to JChat.

The SASC.Manager application handles all service and ruleset configurations, illustrated in Figure 6. Here, the ‘Sources’ part is used to configure the NetJSON data source(s), that is, REST service endpoints and NetJSON static files. As is shown in the figure, multiple such sources can be configured at once, and check marks indicate which source is active at any given moment. So, in the provided screenshot, the currently active source is a NetJSON file. Next the ‘Services’ field is used to define services that should be handled by the middleware. Further, for each service, a set of consumers is defined, which are allowed to access the service in question. This information is used to estimate best configuration, based on links between consumers, the provider and the information gathered from NetJSON. In the screenshot, we have three services configured:

ServiceVlc01 – this is the adaptable full motion video service

Fig. 6

Finally, the ‘Adapters’ section contains configurations for the various service instances. Here, each service is set up with its IP address, credentials to access/control it and so on. Hence, the adapters section contains information on how to connect to and control specific instances of the generic capabilities defined in the ‘Service’ section of the interface.

Following the setup of Sources, Services and Adapters, the specific middleware rules can be configured. The ruleset consists of one or more rules, sorted in the order of them being evaluated (priority). Figure 7 shows the rules configuration pane of the SASC.Manager application.

Fig. 7

In Figure 7, we see on the left side, the ‘Ruleset’ pane, which has a list of the configured rules. The highlighted rule, Rule 1, is then expanded on the right side where details are shown. This part of the manager is used both to create new rules and to manage already existing rules.

In the ‘Services’ pane, the service(s) which this rule applies to are shown. Next, the conditions pane covers which network metrics should be evaluated and sets the threshold values that trigger the rule. Here, we see the rule uses both the HopCount (i.e., number of hops between the service and the consumer) as well as the Cost (i.e., tied to transmission cost, typically the available bandwidth). Note that conditions are evaluated using AND logic, which means that all conditions must be met to evaluate TRUE and trigger the rule.

In the ‘Actions’ pane, the appropriate (re)configuration action to take when the rule is triggered is set up. The SASC.Manager application saves both the service configuration and the ruleset for further use. The middleware operates on cycles, where each service cycle loads the service configurations/rules, fetches the NetJSON data providing a network status snapshot and evaluates the data according to the rules. If multiple sources are used and different network parameters are received for one or more destination addresses, SASC will use the worst received hop count, cost and bandwidth values for each consumer object when validating the status with the ruleset. If conditions are met, the rule is added to a reconfiguration execution queue. The queue is sorted based on rule priority, and only the highest priority rule for each service is executed. This is done to ensure a steady state and avoid multiple rule matches causing multiple reconfigurations of the same service in one cycle. Finally, a (re) configuration of the corresponding service(s) is performed according to the rule(s) in the execution queue. Note that a service cycle is configurable to trigger automatically at pre-set time intervals, or it can be initiated manually through the SASC.Manager tool. The operator has the discretion using the manager tool to configure the rules. To avoid oscillations by triggering reconfiguration too often, we suggest making rules that trigger when major rather than minor changes occur. For example, it has a rule for reconfiguring video from high to low resolution when going from a high throughput (e.g., tactical wideband) network to a low throughput (e.g., tactical narrow-band) network. Such a rule will trigger when throughput changes from the megabit range to the kilobits range. Conversely, having a rule triggering a reconfiguration in small increments, e.g., between 10 kbit/s, 20 kbit/s and 30 kbit/s is too fine grained. Hence, rules should reflect the anticipated underlying communications carriers.

CLO brings new considerations for services. Reconfigured services may fall into a new traffic class for the tactical router and would need to be marked accordingly. For example, for the way we adapt chat, a chat session going from wideband to narrowband should also be marked differently, since when attachments are blocked, chat can be accommodated across narrowband.

In FIST, Fraunhofer FKIE, Germany, implemented the NetJSON/REST API and made it available to us at CWIX 2021, where the interface was tested and shown to promote interoperable exchange of network routing information. Benefits identified through these tests were improved service availability and usability at the tactical edge using knowledge of the underlying available network resources.

For these experiments, Germany used their tactical routers using the OLSRv2 (Clausen et al. 2014) routing protocol and specifically built their NetJSON service to expose knowledge this protocol had about the network. Here, cost measures could be a latency value, the number of hops from destination to source, and finally a parameter called ‘cost_text’ contained a string value for the throughput. An excerpt from a NetJSON document, showing one single route, can be seen in Figure 8. The values are dynamic and come from the routing protocol and may change from one invocation of the service to the next. These experiments are further described in ‘Section 7.4 Objective 4. Application & Middleware over Radio’ of the CWIX report (CWIX-21 2021).

Fig. 8

At CWIX 2021, we consumed network awareness information from a proactive routing protocol. Conversely, the DFS routing protocol discussed in Section 3 is a reactive routing protocol. As the name implies, reactive routing is driven by actual communication needs, in that updating/creating new routes is driven by an initiated communications request. Facing such a protocol, our current implementation will not work ‘out of the box’. It would need to be modified so that it works equally well with both reactive and proactive protocols. In the following, we discuss how our implementation could be changed to be able to interact well also with DFS and other reactive protocols.

Since the goal is developing interoperable solutions and we have shown NetJSON to enable interoperability, we aim to keep the same API that we have already developed also for the interaction with reactive routing protocols. We propose keeping the network awareness service in the tactical router (see Figure 2) and the interaction with proactive routing the same but expand the service behind the API with a slightly different application logic when it interfaces a reactive routing protocol. One effect perceived on the client side (by the middleware) is that there may be a longer delay from querying the service, until it actually gets a response. This can be accommodated by setting a larger timeout on the client side to ensure timeouts are less likely to occur. That means the middleware would work with proactive routing as of now (that will yield an immediate response) and reactive routing. Reactive routing will not provide any awareness information until a route is actually in place, inducing some delay in the responsiveness of the NetJSON service.

When the network awareness service in the tactical router needs to interact with a reactive routing protocol, the service would receive a request from the middle-ware on a client hosting a user-facing service. Then, the service would, in case a route does not already exist, send a data packet to trigger the routing protocol to start a search for a route to the destination IP address. When a route is found, the protocol informs the network awareness service with known information about the route, such as the number of hops and for the DFS protocol also the QoS/policy rule that the route supports. The service responds with the standardised API to the middleware of the client that requested the network state in formation. So, once the service response is handled by the middle-ware and the user-facing service has been reconfigured, the route for this communication is already set up and available. The benefit of this approach is that it is fairly simple and will function for any tactical network. Tactical routers with proactive routing protocols will use the service as demonstrated at CWIX, whereas routers with reactive routing protocols will invoke the suggested new function in the service that sends a data packet to trigger the reactive protocol to set up the route before responding to the service call. The drawback of this approach is the varying delay that it might take to send a response to the call.

Currently, our implementation works with a simple REST API with no input parameters, which just fetches the NetJSON resource using a HTTP GET. The response is expected to be a snapshot of all the OLSRv2 routes coded as NetJSON. This approach works well for a service populated with data from a proactive routing protocol. To allow the service to work with a reactive routing protocol, it is necessary to include the required endpoint IP address(es) in the request. Requesting a snapshot of all routes in the network from a reactive routing protocol would invoke much unnecessary overhead. When the router runs a policy-enabled routing protocol such as DFS, the query to the service in the router should also either ask for the availability of a route with certain requirements (e.g., a certain minimum data rate) or a list of available routes with different characteristic to the specified destination address(es). For the first case either the TOS field, an extension header or similar could be used in the query to identify the policy that should be invoked for the route search. As an example, when setting up a chat service, the middleware queries the tactical router for a route that fulfils the high data rate traffic class (see Table 1). If such a route is found, the chat can be sent with attachments and must be tagged such that the tactical router treats it as traffic that applies to Rule 2. If a route that fulfils the high data rate traffic class cannot be found, the chat can be sent as high priority traffic or best effort traffic depending on the importance of the chat messages (must be tagged accordingly) and cannot have any attachments.

Further, there remains the open issue of topology changes in the network after the traffic is initiated. How should the router notify the middleware of this? Currently, since the middleware regularly polls the NetJSON service, this approach would continue to function. However, one could also anticipate another approach that we have not tried, where push communication could be used from the router to the middleware to notify of route changes. This approach is left for future work and could be investigated in a number of ways. For example, it could be achieved by REST long polling or it could be implemented by using a publish/subscribe protocol. Independent of which approach is taken to signalling changes, it is important that oscillations in reconfiguration of services and rules are avoided. We think the best approach to mitigating this problem is to ensure configuring rules that match threshold values of underlying communications capacities (e.g., going from the megabits range of throughput to the kilobits range should trigger a reconfiguration since this would, in the case we have been investigating, mean that we are now switching from wideband to narrowband).

In our tests, we have shown the feasibility of using the NetJSON data format over a REST API to adapt user-facing services to the available communication resources. The main hindrance we see that will get in the way of rapid deployment of such a solution in the field is the fact that almost all communications between the tactical nodes in military scenarios are end to end encrypted. This means that a tactical node with the adaptive service infrastructure (i.e., middleware software) placed on ‘the red side’ (i.e., the secure side) will not be able to query the network awareness service in the router. If the middleware is placed on ‘the black side’ of the crypto (i.e., outside the protected firewall), software will not be able to communicate with (and then ultimately control, adapt and reconfigure) the user-facing services, meaning service adaptation will not be possible.

It is possible to allow information flow from the ‘black’ to the ‘red’ side, granted there are policies and mechanisms in place to enable this. We propose keeping the middleware on ‘the red side’ together with the user-facing services. This means that all we have shown will work, with the exception of the network awareness information being available. To solve this, we propose allowing to expose the NetJSON with REST API on the ‘red side’. We can understand and appreciate the need to limit information flow through such an API to mitigate the threat of information leaks and covert channels. Still, using modern security measures on the API itself, it should be possible to enforce role-based (or even attribute based) security measures and access and so limit the exposure (a risk assessment needs to be performed). For example, we could anticipate the service information and clients list being the input into the network awareness API, and the result (given this particular instance of the middleware has the correct credentials and so being authenticated and authorised to access the information) would then just get throughput information back for the service (or set of services) that it needs. This would limit the chances of exploitation on the red side of networking information, as the current approach of coding the entire routing table information, as NetJSON would reveal a lot more information than is strictly needed through this network awareness service. But, limited in such a fashion as we outline here, the middleware would still be able to do its adaptation work, and the threat would be smaller to a coalition network of exposing such a service, as opposed to a service giving a complete routing table as output. Simplifying the service response would also help alleviate parts of the first problem we discussed, that of expanding network awareness to include not only tactical routers relying on proactive, but also reactive, routing protocols.