Detecting Inappropriate Image Content With Clarifai

Inaccurate data in scientific papers can result from honest error or intentional falsification. One form of deliberate falsification involves the use of image manipulation tools to create sloppily retouched figures.

This API identifies inappropriate content in an image and returns a boolean property for each class it finds (isAdultContent, isRacyContent, and isGoryContent). It can be used to filter NSFW content in images uploaded to your platform.

Explicit Nudity

Explicit nudity refers to any depiction or description of a human’s genitals, breasts, buttocks or nipple that leaves nothing to the imagination. Clarifai’s Explicit Content detection feature flags any image that includes gore, drugs, explicit nudity or suggestive nudity.

The vast majority of academic articles that address the topic of nudity detection focus on post-digital capture digital media, which are images and videos that have already been digitized. Many of these techniques rely on complex feature extraction with both low and high-level features which increase the accuracy of classification. However, implementing such approaches within real-time applications like Skype would require significant processing time and memory space.

Among these articles, only four presented solutions that could be used to detect nudity in mobile social networking and dating applications. They focused on using contextual learning, skin detection percentage or a hardcoded navel recognizing process to recognize exposed private parts of the body. These solutions were not tested to see if they could be implemented in real-time applications such as Skype.

Sexual Content

NSFW detection is a highly sought-after feature for many online services that rely on user-generated content. These include social media, e-commerce and messaging platforms. In this case, the software would automatically filter NSFW images to ensure that children are not exposed to inappropriate content.

During inappropriate image detection, the software looks for nudity, sexual content, and other NSFW concepts within an image. Various methods are utilised, including skin colour detection and feature engineering approaches. However, these are not foolproof. They are prone to false positives as they do not take into account illumination changes, and may not detect nudity in the presence of clothes.

Another solution involves using ensemble methods to combine the strengths of different classifiers. For instance, a simple TF-IDF text encoder is paired with an SVM classifier to detect erotic-sexual content in text. This approach outperforms other models based on VSM or word embeddings. This is a result of the fact that the TF-IDF feature weighting method does not require supervised learning.

Violence

Detecting violence in images is challenging. Although several methods exist, most of them either focus on misogynistic or pornographic content. Birhane and Prabhu [2021] found that machine learning algorithms can successfully classify misogynistic and pornographic images, but are unable to identify violence.

Inappropriate image content can also include other objects and scenes that are deemed unsuitable or offensive, such as caged animals (e.g. “king penguin”), a bathtub tainted with blood (“tub”) or a person murdered with a screwdriver (“screwdriver”). Moreover, offensive symbols and text can also be detected, such as National Socialist symbols (especially swastika), persons wearing Ku-Klux-Klan uniform or the middle finger showing gestures.

Steering CLIP towards identifying potentially inappropriate concepts requires little additional data compared to other pre-trained models. However, the performance is inconsistent and the method suffers from high recall and low precision, as shown in Tab. 1. Even after fine-tuning with a linear probing layer, the results are inconclusive. Further work on this subject is required.

Violent Images

Violence is one of the most pervasive images used in modern storytelling. Whether it’s a music video of a rock band playing violently or pictures of gang violence, graphic and gory images are everywhere.

In many cases, these images are not only disturbing but also have negative mental health impacts. They are a key driver of pathological mindsets, and they contribute to the erosion of society’s standards.

Despite the obvious harm of violent images, media outlets have become very selective about which ones they display. Images of throat slitting, corpses and torture scenes don’t feature in newspapers or TV newscasts.

Detecting violent images requires more advanced techniques than other image classification algorithms, such as those that use a bi-channels CNN or an SVM. The authors propose an approach that uses the width of a circumscribed rectangular frame to judge how many persons are in a scene. A larger width would indicate that multiple persons have touched each other, and the algorithm should judge whether this was a physically violent event.

Leave a comment