Department of Homeland Security projects underway to use advanced AI in challenging video environments

Ron Alalouff examines an innovative project which is funding two tech startups to develop their AI offerings, so that they can be brought to market and used in more challenging circumstances and improve public safety.

In a significant step to enhance the use of AI to detect threats to public safety, the Department of Homeland Security (DHS) in the US has announced almost $400,000 of funding to develop the technology.

Under its Silicon Valley Innovation Program, the DHS’s Science and Technology Directorate has awarded the funding to two AI video developers to produce solutions that automatically detect anomalous events via video camera feeds, reduce error to optimise human performance, and minimise delay to enhance responsiveness in threat situations. The two developers are Kansas based Flux Tensor, and Massachusetts based Lauretta AI.

Image credit: Flux Tensor

Persistent flux analysis

Flux Tensor has developed a system which uses maths to assign types of motion (excluding unwanted effects such as weather) and then analyse temporal motion changes in images. The analysis happens in near real time (1-1.5 second delay). It is primarily intended to be used in large and potentially crowded interior spaces such as airports, rail stations and bus terminals. The technology can typically be used for monitoring and detecting potential threats, such as abandoned luggage.

“The difference between our technology and others on the market is our use of persistent flux analysis,” Steve O’Conner, CEO of Flux Tensor told IFSEC Global. “We’re deleting weather and light and are focussed on ‘hard video’ – we can do the hard stuff too by separating out objects.”

The technology provides a flexible object detection approach to determine “persistent change” while accounting for changing weather, light variations, and poor video quality in near real time. Once developed, the software will integrate with current security workflows to detect objects in motion.

Activity recognition

Lauretta AI’s technology uses a non-biometric algorithm to identify people from the front, side and back from as far as 1,500 ft – regardless of clothing or face coverings – and infer potential non-standard or questionable intent. It proposes to adapt its existing technology to enable activity recognition and tracking. The refinement will capture multiple data points per subject to minimise false alerts and address the diverse requirements and situational limitations of soft target locations.

The Silicon Valley Information Program (SVIP) provides funding to start-ups so that they can adapt and develop their existing commercial technologies, taking into account DHS’s operational needs. By being available commercially, DHS and the wider homeland security enterprise can acquire systems from the market, without the need to sustain the operations and maintenance of those product lines.

“The awarded startups work alongside DHS programme management teams to understand those operational needs and use cases and – as part of SVIP Phase 1 – demonstrate a minimum viable product,” said Melissa Oh, SVIP Managing Director at the DHS Science & Technology Directorate. “If successful, startups will progress through SVIP’s 4-phased programme, where they iterate their solution and are evaluated for operational viability.

“At the conclusion of this Securing Soft Targets project, SVIP expects the software the startups are developing to be available for either soft target venues to integrate these enhanced capabilities into their video management systems, or video management system vendors may look to integrate them directly to provide value added capabilities.”

Hard cases

Steve O’Connor added that the DHS process allows the system to be pushed to see if it can tackle really difficult scenarios. The DHS may then decide to endorse the system by approving vendors, after which city or local communities may receive federal funding to help buy and install a system.

If fully developed and brought to market, the technologies could help secure people in schools, sports venues, transportation systems, shopping venues and places of worship. “Ensuring the safety of soft target venues is a top priority,” said Melissa Oh. “Innovative technologies that can be integrated into existing systems is critical to preserving and enhancing the security and privacy of public areas and people.”

Mass transit security

In an indication of how such technology is most likely to be deployed, Ali Fadel, DHS S&T Soft Target Security program manager, said: “Mass transit, on average, carries nearly 10 times as many passengers a day as the nation’s busiest airports. With limited checkpoints for screening passengers and their belongings, our program is pursuing innovative technologies to enhance physical security and situational awareness at venues.

“Utilising advanced algorithms to identify and alert security personnel to left behind items – in near real-time – will allow them to quickly respond to dangerous events and clear harmless left-behind items. The intent is to integrate security solutions within a broader layered architecture to better protect commuters, riders, and families using public transportation systems or attending a mass gathering – all without impacting the speed of the traveling public.”

Free Download: The Video Surveillance Report 2023

Discover the latest developments in the rapidly-evolving video surveillance sector by downloading the 2023 Video Surveillance Report. Over 500 responses to our survey, which come from integrators to consultants and heads of security, inform our analysis of the latest trends including AI, the state of the video surveillance market, uptake of the cloud, and the wider economic and geopolitical events impacting the sector!

Download for FREE to discover top industry insight around the latest innovations in video surveillance systems.

Download The Video Surveillance Report