Video analytics and in video surveillance systems have become more commonplace to improve security and safety processes thanks to the development of artificial intelligence (AI) algorithms, even providing actionable business intelligence for organisations. Read our useful guide to video analytics, covering what we mean by the term, how they’re delivered, a short history and the benefits they provide for end-users.
The movement from traditional, analogue surveillance technology to IP (digital) surveillance solutions has given security professionals access to a much broader functionality. IP cameras can create digitised video data streams that can be carried via network cabling or via Wi-Fi. Crucially this has changed video monitoring and CCTV (there are currently 4.2 million cameras in the UK – one for every 14 people) from reactive to proactive real-time tools.
In recent years CCTV software has meant video analytics: crunching the visual data streamed from network cameras and performing real-time event detection and post-event analysis. Harnessing AI and the immense processing power of modern computers, today’s analytics platforms can now offer people counting, heat mapping, facial recognition and a host of other features allowing operators to track suspects and discern patterns amid the noise – and without constant monitoring by a human.
This is crucial given the reported average of video cameras being monitored by each security team staff member is between 25 and 50. According to a blog from 3S Security Systems, quoted by Security and IoT magazine asmag.com, research in the US has suggested that staff watching video systems can experience fatigue “in as little as 12 minutes, overlooking up to 45% of activity in the camera scenes. After 22 minutes, that increases up to 95% of overlooked activity.”
“Leveraging analytics technology can enable a single individual to monitor many more cameras and reduce the amount of missed critical events,” continues the blog post, adding “therefore enabling a prompt response making it possible to prevent a crime rather than just respond to it after the incident.”
A definition of video analytics
There are three distinct types of video analytics: fixed algorithm analytics, Artificial Intelligence learning algorithms and Facial Recognition. Each of these processes digital video signals via an algorithm to perform a security-related function.
Fixed algorithm analytics and Artificial Intelligence learning algorithms both detect suspicious behaviour caught by a video camera and alert the security personnel monitoring those cameras.
Fixed algorithm analytics works by performing a dedicated task and looking for a specific behaviour – for example, someone moving in the wrong direction in a passageway, leaving an object or picking up an object, loitering and straying outside of a delineated space.
Meanwhile, AI algorithms learn ‘on the job’ by being connected to a camera for several weeks. During this period the camera learns what is ‘normal’ and issues alerts when something happens outside of that experience.
An example of this approach was given by Effective Physical Security by Thomas L. Norman: “In one early installation at a major international airport that was intended to spot children climbing on a baggage carousel, the system alerted on a man who picked up a small bag from the carousel and placed it inside an empty larger bag. The man was intercepted and interrogated only to discover that the luggage inside his did not belong to him and that he was part of a ring who came to the airport regularly to steal baggage in this way. The airport had no idea this was even occurring, so there was no way they could have purchased a fixed behavior algorithm for this, even if such existed (which it did not).”
Facial recognition systems, meanwhile, are used to control access and to identify threat actors. Data from facial recognition may also be used as part of an investigation if a ‘person of interest’ has been identified.
Facial recognition works by matching points on a face with a sample stored in a database. If there’s no match, the system tries to create a new record using the best available image of the subject. Subsequent versions of facial recognition can use 3D mapping in real time, with recourse to a much larger database.
A subject can also be identified via ‘faceless recognition’, where someone’s physical characteristics such as height, posture and build can be used to identify them in a crowd.
How video analytics are delivered
Most cameras on the market today use what is termed ‘edge analytics’, which means that the analysis is performed on data at a sensor, network switch or other device (such as in the camera itself) instead of waiting for the data to be sent back to a centralised data store. Some experts advise that these types of camera are best suited for indoor surveillance, albeit one in which light levels are controlled.
The challenge of outdoor environments – lighting, weather, insects etc – are best handled by server-based, detection analytics technologies, argue some experts. Server-based architecture can receive updates and upgrades more quickly and deliver much more computational power. However, this method does risk a backlog of data, so a limit on what is relevant and what is not needs to be set.
For manufacturers is it faster to develop, and easier to deploy improvements to their algorithms in a server-based architecture. Moreover, server-based architectures are capable of delivering much more computational power, which enables the use of the latest, most advanced analytics algorithms based on deep learning/artificial intelligence
A brief history of video analytics
While video analytics offer new opportunities for security teams, their success has not been overnight.
Because ‘first generation’ video analytics worked by detecting pixel change in certain parts of the video, a door or tree blowing in the wind, or an animal passing by, could trigger an alarm – and false alarms were common!
The first generation came of age with algorithms starting to enable the camera to have the intelligence to interpret what it was seeing and having functionality in terms of height and width ratios, object speed and repetitive motion. Meanwhile, advancements in CMOS sensors helped to provide clear images in poor and ‘dynamic’ lighting settings.
Writing for Security Infowatch.com, Dr. Bob Banerjee, Senior Director of Training and Development for NICE Systems’ Security Division, noted that: “While first generation video analytics focused on real-time alerts and minimising false alarms, second generation technology focused on providing forensic tools to search the video post-event.”
Banerjee chose the following example to illustrate the difference: “If there had been a robbery and someone saw the suspects escape in a white van, you might search a camera for all white vans that drove past your location the day prior. Or if the situation involved an assault by a person wearing red, you might search a camera for the past three hours for people wearing red.
DOWNLOAD: The Video Surveillance Trend Report 2022
“The second generation of video analytics helped to reconstruct what happened; however, it was designed more as a forensic tool rather than a tool that exists to tell you where the van or person is right now, just moments after you heard about them. That sense of immediacy is what makes all the difference in response and mitigation.”
While post-incident analytics is important to identify weak spots, the ability to monitor an ongoing situation was the USP of third generation analytics, with its ability to deliver in real-time. Using an image of, for example, an intruder, or even just a description of them, a database can be quickly searched through and, if the intruder is still in the building, they can be located.
“Video analytics that follow suspects as they move from camera to camera – forwards and backwards in time – provide a much greater situational awareness and expedite investigations so authorities can apprehend the suspects before a situation escalates,” notes Banerjee.
With the development of AI-based analytics, machine learning and Deep Neural Networking (DNN) algorithms, the camera analysis has become more precise, with an ability to detect specific objects and distinguish between them.
Common applications for video analytics
Video analytics are used for a wide range of tasks, including, as mentioned above, facial recognition. Here are some of the most common:
Object tracking
An object is detected and its trajectory is tracked through a single camera’s field of view. So, for example, an object crossing a pre-designated virtual line can be detected, along with the direction it is travelling.
Another option for this is multi-camera tracking, where an object is tracked from one camera to another.
This facility can detect and monitor the movement of vehicles and people in outdoor environments, and it is used for traffic control, visual surveillance, forensics, human-object interaction, gesture recognition and augmented reality. AI-based object tracking analytics can identify the type of object, such as vehicle or person.
Left and removed object detection
This detects the appearance and disappearance of static objects within a defined area. This type of detection is often used in airports or underground systems from ensuring fire escapes are clear to identifying potential bomb threats.
Loitering detection
This facility detects when a person or vehicle remains in the full camera view or a designated area. Some systems are now searching for suspicious behaviour as well called behavioural detection.
People counting
Detects and counts people as they cross through a designated area. The benefits of this include occupancy control, capacity evaluation, sales and conversion metrics, personalised visitor experiences and measuring operational effectiveness. This process has been particularly sought after during the COVID pandemic, where retail stores and supermarkets could easily monitor how many people were in the store at any given time.
People tracking
People tracking requires more information, including motion detection and facial recognition. Uses for it include: detecting intruders, wrong-way detection, people counting, physical distancing and customer behaviour analysis.
Crowd detection
Crowd Detection is a real-time surveillance technology that allows for the detection of crowd density to evaluate capacity or occupancy issues within a defined area. Applications of crowd detection include population counting, public event management, disaster management, safety monitoring, military management, and suspicious activity detection.
Image change/tamper detection
Detects when there has been a major change in the camera view such as an obstruction, power cut or the camera has been moved, spray painted etc,
Motion Detection
Video Motion Detection (VMD) senses physical movement for a given area in real-time. Motion detection analytics are embedded in IP cameras, network video recorders, and video analytics and video management software systems. VMD is used in the military, critical infrastructures, robotics and in the entertainment, sports, healthcare sectors.
Motion detection is now also being utilised in residential environments, whereby a user’s app is notified if a person is detected approaching the camera’s field of view.
Automatic Number Plate Recognition
ANPR uses optical character recognition (OCR) technology to identify and read vehicle license plates from CCTV, video cameras, law enforcement cameras, or dedicated high-speed ANPR cameras mounted on roadway infrastructures.
Pan-tilt-zoom (PTZ) Auto-Tracking
PTZ auto-tracking video analytics allow surveillance cameras to follow and zoom in on people within a larger field of view. Auto PTZ also allows a security worker to multitask.
What are the benefits of video analytics to security end-users?
Video analytics not only serves pure security concerns, but also traffic and crowd control management, footfall analysis and social distancing or face mask detection.
Increasingly, video surveillance devices are being integrated into other physical security and building management platforms to provide more intelligent data for security and facilities managers.
There is no doubt that video analytics are an attractive option for many sites. By potentially preventing incidents they save on costs related to damage to property and assets, theft and disruption to general business continuity.
Meanwhile, for better or for worse, automated systems mean staffing levels can be managed. Fewer staff are needed if breaches to perimeter fences etc are detected automatically. By the same token, video analytics will decrease the likelihood of risk to staff and customers.
Video analytics can be connected to other systems, such as lighting controls or access management, activating them only when there is need.
The European Commission report ‘Video analytics adoption: Key considerations for the end-user’ notes that “video is a pertinent way to prove deniability or liability. Using video analytics to find key incidents as evidence eliminates manually sifting through video. This has a direct effect on legal, litigation and insurance costs, as well as saving time and money by not having to manually watch video.”
How are analytics applied in different industries?
Video analytics are a very versatile tool, benefitting numerous diverse settings.
Aside from the more obvious security scenarios, here are a few examples of their use in other sectors:
Retail
As well as saving money on security concerns (e.g preventing shoplifting), in the long run, video analytics can be a money maker. The retail sector employs it to track the customer experience and behaviour (e.g. direction of gaze and visit duration) to the point that more effective selling and upselling locations can be established.
Healthcare
As well as for the safety of staff, patients and visitors, hospitals use video analytics to guard against theft of drugs and even abduction of infants. In terms of patient care, they can help to ensure that medical staff have made the necessary visits.
Meanwhile, elderly people who are being cared for at home can be monitored so that help is called if they experience a fall or to check if they have taken medication.
Smart cities
Video analytics can control traffic light control systems and monitor traffic jams, monitor speed, and it can detect hazards such as vehicles stopping vehicle stopping unauthorised spaces, erratic driving, or vehicles involved in an accident. Automatic Numberplate Plate Recognition (ANPR), as mentioned above, can identify stolen or suspect vehicles.
Logistics
Video analytics is being used in logistics and manufacturing to streamline inventory management and production, improve quality control and efficiency and assist in security processes. They are being used to assist in early detection for incidents, where response times could be of critical importance, while behaviour detection can help to automatically recognise accidents at an early stage.