Until recently creating a new enterprise C4ISR system, be it Army, Navy, Air Force, or Department of Homeland Security, utilized the same deterministic systems engineering approach. In the last several years the use of AI and Machine Learning (ML) in surveillance systems has become widespread, often with good success, but sometimes with mixed results. As Puretech have observed in recorded surveillance photos and videos, adversarial methods and techniques are sometimes able to defeat the standard AI/ML trained neural networks for example by altering target shapes or adding layers of clothing to humans to become low observables.

It is no surprise that to address the standard “Detect-Track-Classify” problem a more sophisticated approach extending beyond AI/ML is imperative. The current DoD Digital Modernization Strategy guidelines are embedded in PureTech Systems PureActiv software based architecture as well as in individual security designs that were have used successfully in the US and internationally. Working with DOD JAIC (Joint Artificial Intelligence Center [https://www.ai.mil/about.html]) since 2016 provided an early understanding of many of these guidelines and techniques. One of JAIC early recommendations was to not only put bounding boxes around detected objects, but also record relevant metadata to deliver more information for the mission and later trend analysis. Puretech were able to implement and test these recommendations at field installations, continually refining the software algorithms including taking advantage of sensor correlation.

Lessons Learned

By leveraging such sensor and data correlation, PureActiv C4ISR  hosted on edge devices and/or servers can receive detections and tracks from multiple sensors such as motion analytics, radars, and fence, ground and subterranean sensors and then autonomously command PTZ cameras and/or dispatch UASs, and UGVs to go “inspect” the detected targets to automatically confirm target types Before issuing alarms, i.e. provide auto-verification. This robust approach also combines our geospatial video analytics with AI/DL and radar tracks to disambiguate the objects of interest. This allows the system to eliminate a substantial, up to 95%, amount of nuisance alarms while maintaining a very high detection rate.
Another important lesson that Puretech take to heart is the AI-assurance. It’s important to remind ourselves that AI can do wrong things and it can learn wrong things. That is why PureTech Systems continuously validate the confidence level of the system through robust and continuous ground truth regression testing.

The general view and key features of an autonomous C4ISR

In this example screenshot, objects of interest are automatically detected, tracked (upper left and lower left side of the screen) and classified by AI/DL as people, conveyance, and UAV and displayed on the digital terrain map with the correct GIS coordinates in real time (upper right side). Alarms are displayed in the lower right with a still image and looping video of the alarm and the pertinent live camera. The decision of target classification is made autonomously without human interaction.
In addition to gradual improvement in sensors’ performances, such as cameras and radars ranges and sensitivities, PureTech recently integrated three relatively new capabilities and made them available to DHS and CBP in particular. These included Android Team Awareness Kit (ATAK), small drones (sUAS) detection, and counter-UAS systems. The same C4ISR package shown above fully supports autonomous counter UAS operations after a combination of the air surveillance radars and high-speed, wideband RF scanner for detection and classification of signals in congested spectrum environments were integrated. Since adversaries use sUAS for smuggling illegal drugs over the border, an integrated counter-UAS capability available where such activity is occurring is badly needed.
In addition, C4ISR systems that provide ground and UAS tracking including recording track metadata can server as a tool to learn about smuggling corridors and patterns of drug cartels’ operations. According to Tim Bennett the program manager for air domain awareness at the Department of Homeland Security’s Science and Technology Directorate whose job is devoted to detecting, tracking, and countering drones, “Narcodrones a new big problem that we all have to address. It isn’t just DHS. It’s across all agencies in our government and all governments in the world (https://www.smithsonianmag.com/air-space-magazine/narcodrones-180974934/).”
ATAK is quickly becoming a force multiplier for many agencies by enabling cross collaboration between individual agents, teams, and central command. ATAK becomes an extension of the autonomous command and control by communicating data from and to the field including geolocations, media, and tactical information that is immediately actionable and disseminated across one or more areas of responsibility. All information that is available at the C4ISR level (left picture below) can be displayed on ATAK devices (right picture), whether it is installed on a Windows laptops or Android devices.
In conclusion, it is worth noting that in every C4ISR autonomous subsystem and operation is designed to aide, not replace human capabilities. Whether used as a battlefield command and control or border security common operational picture, one of the key goals is to augment the warfighter’s or law enforcement agent’s reasoning skills with an autonomous decision-making systems. Ultimately autonomy reduces cognitive load by filtering nuisance alarms and prioritizing information flow for action. This enables agencies to dramatically reduce operational resources while maintaining mission effectiveness. As former Secretary of the Air Force Heather Wilson once said, “The advantage will go to those who create the best technologies and who integrate and field them in creative operational ways that provide military advantages. (www.acc.af.mil/News/Article-Display/Article/1448216/secaf-this-is-about-lethality-and-mission-effectiveness/)”

The Author: Ilia Rosenberg, PureTech Systems, Vice President, Federal Sector