Image Classification Challenges and Solutions
By: Tola Waisman – Director of Technology, CTO at Cellebrite
The prevalence of digital data in investigations means that investigators have to increasingly rely on advanced solutions, such as image classification, to help analyze the variety and volume of data that investigations produce. Cellebrite’s Media Analytics technology helps meet the challenge of recognizing and categorizing large volumes of images in databases containing suspects, victims, drugs, weapons, nudity, and more.
Watch Tola Waisman, the Artificial Intelligence Group Team Leader, explain why Cellebrite’s AI-powered Analytics Solutions is a must-have for digital investigators.
Let’s explore some of the challenges that digital image and video classification methods can experience and the solutions Cellebrite has developed to overcome them.
6 Image Classification Challenges:
- Video: Several perspectives of one person caught on camera can confuse image classification solutions that can list one person as several individuals. It is important for solutions to consolidate all media into single person media profiles. When redundant profiles are reduced, digital investigators can then rely on the generated shortlist of profiles to find suspects, leads and victims faster.
- Configuring Algorithms: Algorithms must be constantly maintained and adjusted to output relevant results as data ecosystems evolve with new technology and information streams. As different types of data are introduced from the digital marketplace, image classification solutions must be flexible to grow with the many file types yet to be developed.
- Processing Time of Image Indexing: Image classification can take hours to process as multiple categories need to be referenced into the output. Solutions must maintain fast processing speeds that are relevant for time-sensitive investigations.
- OCR (Optical Character Recognition): Often mobile devices contain screenshots of conversations from text apps that people use to document conversations. Image-to-text recognition is vital to make sense of images where the pictured text needs to be searchable.
- Object Classification: Being able to quickly identify weapons, drugs and other typical objects involved in crimes, allows investigators to vet data more efficiently. When image classification and object recognition are used together, a cluster of supporting data can be surfaced to highlight actionable intelligence.
- Ad Hoc Image Classification: As certain crimes have specific characteristics related to objects or locations, it’s important for investigators to be able to train the machine algorithms on images specifically relevant to their case. This will result in the best potential output related to an investigation and allow for ongoing analysis, as well as retro-analysis of media data from previous cases. Solutions that give non-coding investigators a straightforward means of custom media categorization are essential.
Overcoming Challenges with Media Analytics powered by Artificial Intelligence
With these challenges in mind, Cellebrite has developed an Analytics solution that leverages Artificial Intelligence (AI) to find the “needle-in-the-haystack of evidence” and get important insights that were not previously possible.
AI is the technology that enables computers to perform tasks that up until now were only done by humans. Today, AI-powered computers can accomplish phenomenal tasks from beating the world’s best chess players to driving cars. Investigators can now add this powerful capability as a force multiplier to automatically surface leads and actionable insights during the early hours of an investigation.
Let’s take a deeper look at the Cellebrite Analytics features that speed up digital investigations.
Media Analytics: Image Classification
When investigating massive amounts of data, with just a push of a button Media Analytics flags suspicious images, and videos while finding photos of a person or an object of interest. Media Analytics uses machine learning to automatically detect and categorize images and video frames related to key categories, such as child exploitation, weapons, money, drugs, nudity, and more. Digital investigators can quickly identify persons of interest with advanced person recognition and categorization capabilities.
Media Analytics: Video Synopsis
Using Video Category Synopsis, investigators can save time by skipping between video frames to focus only on the frames of interest within a specific video file. Different predefined categories are clearly marked on the video playback bar including scenes containing nudity, flags, cars and more.
Media Analytics: Image Similarity
If an investigator has selected or loaded an image of a person of interest or of a background that is relevant to their investigation, Media Analytics will scan the database within seconds and surface images and videos that contain frames with similar faces and surroundings.
With this capability, investigators can feed multiple pictures or video examples to “train” the system to identify objects previously undetected in the database such as specific suitcases, symbols, or rooms. For example, in a child exploitation case where the investigator suspects that the victims change but the location stays the same, they will be able to find additional victims by surfacing pictures taken in the same room.
Media Analytics: Custom Categories
If there is a need to find something that is not part of the preset categories in the Analytics Solution, Media Analytics delivers industry-first custom media categorization capabilities.
Essentially, Media Analytics enables the digital investigator to filter out the noise and focus on their area of interest while significantly reducing the time to evidence. Investigations that used to take weeks or months are now being solved in days or even hours.
The Road Ahead for Image Classification Technology
Image classification has already proved its value in the field and in the forensics lab, but more improvement is needed and expected as public acceptance evolves, global adoption increases, and AI-powered machine learning deepens. New laws and cultural norms will need to adapt in partnership with solution developers to ensure acceptable practices are maintained.
As public expectation for faster case resolution continues to grow, law enforcement will need to rely more and more on AI-powered solutions to overcome modern challenges to help make a safer world.