|
|
--- |
|
|
license: apache-2.0 |
|
|
language: |
|
|
- en |
|
|
base_model: |
|
|
- openai/clip-vit-base-patch32 |
|
|
- sentence-transformers/all-MiniLM-L6-v2 |
|
|
- google-t5/t5-small |
|
|
pipeline_tag: image-text-to-text |
|
|
library_name: transformers |
|
|
--- |
|
|
library_name: transformerstags: - image-to-text - clip - t5 - sentence-transformers - ragpipeline_tag: image-to-textlicense: apache-2.0 |
|
|
RAG Image Captioning Model |
|
|
This is a RAG-based image captioning model using CLIP (openai/clip-vit-base-patch32), T5 (t5-small), and SentenceTransformer (all-MiniLM-L6-v2). It retrieves similar captions from a FAISS index and generates a caption using T5. |
|
|
Files |
|
|
|
|
|
inference.py: Custom inference script with a predict function. |
|
|
requirements.txt: Dependencies. |
|
|
faiss_index.idx: FAISS index for retrieval. |
|
|
captions.json: Caption corpus. |
|
|
|
|
|
Usage |
|
|
Upload an image to generate a caption. Designed for API integration via Hugging Face Spaces or custom deployment. |
|
|
Setup |
|
|
Install dependencies from requirements.txt and ensure en_core_web_sm is installed for spaCy: |
|
|
pip install -r requirements.txt |
|
|
python -m spacy download en_core_web_sm |
