|
|
--- |
|
|
library_name: transformers |
|
|
license: mit |
|
|
language: |
|
|
- en |
|
|
pipeline_tag: object-detection |
|
|
base_model: |
|
|
- microsoft/conditional-detr-resnet-50 |
|
|
tags: |
|
|
- object-detection |
|
|
- fashion |
|
|
- search |
|
|
--- |
|
|
This model is fine-tuned version of microsoft/conditional-detr-resnet-50. |
|
|
|
|
|
You can find details of model in this github repo -> [fashion-visual-search](https://github.com/yainage90/fashion-visual-search) |
|
|
|
|
|
And you can find fashion image feature extractor model -> [yainage90/fashion-image-feature-extractor](https://huggingface.co/yainage90/fashion-image-feature-extractor) |
|
|
|
|
|
This model was trained using a combination of two datasets: [modanet](https://github.com/eBay/modanet) and [fashionpedia](https://fashionpedia.github.io/home/) |
|
|
|
|
|
The labels are ['bag', 'bottom', 'dress', 'hat', 'shoes', 'outer', 'top'] |
|
|
|
|
|
In the 96th epoch out of total of 100 epochs, the best score was achieved with mAP 0.7542. Therefore, it is believed that there is a little room for performance improvement. |
|
|
|
|
|
``` python |
|
|
from PIL import Image |
|
|
import torch |
|
|
from transformers import AutoImageProcessor, AutoModelForObjectDetection |
|
|
|
|
|
device = 'cpu' |
|
|
if torch.cuda.is_available(): |
|
|
device = torch.device('cuda') |
|
|
elif torch.backends.mps.is_available(): |
|
|
device = torch.device('mps') |
|
|
|
|
|
ckpt = 'yainage90/fashion-object-detection' |
|
|
image_processor = AutoImageProcessor.from_pretrained(ckpt) |
|
|
model = AutoModelForObjectDetection.from_pretrained(ckpt).to(device) |
|
|
|
|
|
image = Image.open('<path/to/image>').convert('RGB') |
|
|
|
|
|
with torch.no_grad(): |
|
|
inputs = image_processor(images=[image], return_tensors="pt") |
|
|
outputs = model(**inputs.to(device)) |
|
|
target_sizes = torch.tensor([[image.size[1], image.size[0]]]) |
|
|
results = image_processor.post_process_object_detection(outputs, threshold=0.4, target_sizes=target_sizes)[0] |
|
|
|
|
|
items = [] |
|
|
for score, label, box in zip(results["scores"], results["labels"], results["boxes"]): |
|
|
score = score.item() |
|
|
label = label.item() |
|
|
box = [i.item() for i in box] |
|
|
print(f"{model.config.id2label[label]}: {round(score, 3)} at {box}") |
|
|
items.append((score, label, box)) |
|
|
``` |
|
|
|
|
|
 |
