A newer version of the Gradio SDK is available:
5.44.1
Cosmos Guardrail
This page outlines a set of tools to ensure content safety in Cosmos. For implementation details, please consult the Cosmos paper.
Overview
Our guardrail system consists of two stages: pre-Guard and post-Guard.
Cosmos pre-Guard models are applied to text input, including input prompts and upsampled prompts.
- Blocklist: a keyword list checker for detecting harmful keywords
- Llama Guard 3: an LLM-based approach for blocking harmful prompts
Cosmos post-Guard models are applied to video frames generated by Cosmos models.
- Video Content Safety Filter: a classifier trained to distinguish between safe and unsafe video frames
- Face Blur Filter: a face detection and blurring module