LeafNet75/Segment-Tulsi-TFs

Segment-Tulsi Leaves with Transformers(RF-DETR)

Model	Best EMA Mask mAP (@.50:.95)
LeafNet75/Segment-Tulsi-TFs	0.9650
Subh775/Seg-Basil-rfdetr	0.9668

Refer to this amazing paper entitled:

RF-DETR Object Detection vs YOLOv12 : A Study of Transformer-based and CNN-based Architectures for Single-Class and Multi-Class Greenfruit Detection in Complex Orchard Environments Under Label Ambiguity at: arxiv.org/abs/2504.13099

Transformers for Leaf Segmentation 🍁

This model card explores the application of Roboflow’s RF-DETR for leaf segmentation, focusing particularly on Ocimum tenuiflorum (Holy Basil). Unlike traditional CNN-based segmentation models, transformers can effectively capture global dependencies through attention mechanisms, leading to improved contextual understanding and better generalization performance.

RF-DETR represents one of the first transformer-based architectures to demonstrate that transformers can achieve both high accuracy and fast inference speeds, outperforming many CNN-based models in detection and segmentation tasks despite their traditionally heavier computational design.

RF-DETR integrates architectural innovations from Deformable DETR and LW-DETR, and utilizes a DINOv2 backbone, offering superior global context modeling and domain adaptability.

Example Outputs

Here are output examples from the model's validation run:

Training Config:

The model is trained on: https://universe.roboflow.com/politicians/tulsi-wgmfs using COCO dataset format for RF-DETR Seg Preview.

Training followed the official Roboflow implementation. The model was initialized with pretrained weights and trained using the AdamW optimizer, more params are as:

epochs=2,
batch_size=2,
grad_accum_steps=4,
lr=1e-4, #default
pretrain_weights='rf-detr-seg-preview.pt', #default
layer_norm=True,
checkpoint_interval=10,
seed=42,
num_workers=2,
device='cuda', #T4 colab GPU
resolution=432,
lr_scheduler='step',
tensorboard=True, #check Training metrics
class_names=['Tulsi'],
segmentation_head=True

Here is the training results over 2 epochs:

Final Evaluation Metrics (Epoch 1 - Best EMA Model)

The training was completed after 2 epochs, with the best performance achieved at Epoch 1. The metrics below are for the Exponential Moving Average (EMA) model (checkpoint_best_ema.pth), which represents a smoothed-out and more stable version of the model's weights.

Metric	Value	Description
mAP (Masks) @.50:.95	0.9650	Primary metric for segmentation.
mAP (Boxes) @.50:.95	0.9424	Primary metric for bounding box.
mAP (Masks) @.50	0.9749	Segmentation quality at 50% overlap.
mAP (Boxes) @.50	0.9749	Bounding box quality at 50% overlap.
Precision (Boxes)	0.9749	Accuracy of positive predictions.
Recall (Boxes)	0.9400	Ability to find all positive instances.

Understanding the Metrics

• mAP (Masks) @.50:.95 (Primary Metric): 0.9650

  • What it is: This is the most important metric for this segmentation task. It stands for "mean Average Precision." It is the average of the model's mAP score across 10 different "strictness" thresholds, starting from 50% mask overlap (easy) all the way to 95% mask overlap (very hard).
  • Value: A score of 96.5% is exceptionally high and indicates the model is extremely accurate at predicting the precise pixel-level outline of the leaves.

• mAP (Boxes) @.50:.95: 0.9424

  • What it is: This is the same as the primary metric, but it only judges the bounding box (the rectangle around the leaf), not the pixel-level mask.
  • Value: A score of 94.2% shows the model is also excellent at just locating the leaves.

• mAP @.50 (Masks/Boxes): 0.9749

  • What it is: This is the mAP calculated at only one "easy" threshold: 50% overlap. As long as the predicted mask/box overlaps with the true mask/box by at least 50%, it's considered a "hit."
  • Value: A score of 97.5% means the model is nearly perfect at finding all the leaves, even if the predicted outline isn't 100% pixel-perfect.

• Precision (Boxes): 0.9749

  • What it is: This answers the question: "Of all the leaves the model predicted, what percentage were actually leaves?"
  • Value: A score of 97.5% means the model has extremely few "false positives." It almost never predicts a leaf where there isn't one.

• Recall (Boxes): 0.9400

  • What it is: This answers the question: "Of all the actual leaves that exist in the images, what percentage did the model find?"
  • Value: A score of 94.0% is very high and means the model has very few "false negatives." It rarely misses a leaf that it should have found.

Graph Analysis: Base Model vs. EMA Model

The training graph metrics_plot.png shows as:

1. Training vs. Validation Loss: The training loss (blue) drops as the model learns, while the validation loss (orange) stays low and flat, indicating no overfitting.
2. Average Precision @0.50: Shows the mAP at the "easy" 50% IoU threshold.
3. Average Precision @0.50-0.95: Shows the primary (and "harder") COCO mAP.
4. Average Recall @0.50-0.95: Shows the model's ability to find all objects.

In all three evaluation plots, the EMA Model (orange dashed line) is clearly and consistently superior to the Base Model (blue solid line). This is why the final metrics are reported from the EMA model checkpoint (checkpoint_best_ema.pth).

Inference

!pip install rfdetr==1.3.0 supervision==0.26.1 requests pillow numpy

!python rfdetr_seg_infer.py --image /d.jpg --weights /content/output/checkpoint.pth --out annotated_d.png

Topics in trend..

1. 🔥 Comparative Analysis of RFDETR vs YOLO26
2. ©️ Continual Learning with RFDETR