Image Segmentation with ResNet + U-Net

💡 ResNet + U-NET fusion combines deep and contextual vision (ResNet) with spatial fidelity and accuracy in the details (U-NET). It is a versatile, powerful and high sensitivity architecture - ideal for projects where each pixel matters. The model shines in scenarios where the object is small, detailed or textured, and the global context (whole scene) does not help much. This makes it ideal for: - Medical segmentation (eg tumors, vessels) - Industrial defect inspection - Embedded vision for robotics or quality control ⚠️ However, this current version was trained on a narrow-domain dataset, collected under controlled indoor conditions — consistent lighting, high-contrast backgrounds, and fixed camera angles. As a result, its ability to generalize to open-world scenarios (e.g., outdoor images, different backgrounds) is limited.
This is not a flaw of the model, but a natural reflection of its training data. When retrained with more diverse and realistic datasets, this architecture has strong potential for robust performance in general-purpose segmentation tasks.

📌 Class Convention

This project follows the standard:

Class 0: Background
Class 1: Segmented Object

All masks were converted to reflect this convention before training.

🌐 Limitations and Considerations

This model was trained with images captured in a highly controlled environment: constant lighting, a clean background, and objects (cars) positioned on a rotating platform.

As a result, it achieves very high accuracy (IoU > 99%) when evaluated on images similar to those in the original dataset. However, its performance deteriorates significantly when exposed to images collected outdoors, with variations in light, angle, background, and perspective.

This limitation was expected and will be taken into account for future versions with more diverse datasets.

Good Image.. "good_image.png: Segmentation under ideal studio lighting"

Bad Image.. "Failure example with open-world street background"

🌟 Objective

To segment objects in custom grayscale images based on manual annotations, using a complete training pipeline, automated inference, and visual mask validation.

🤖 Notes on Development

This project was born after many hours of experimentation, learning and progress driven by caffeine. Unlike other projects I have participated in before, this one evolved incredibly quickly thanks to the support of artificial intelligence such as Copilot (Microsoft) and ChatGPT (OpenAI). Without a doubt, these are tools that are way ahead of their time. As part of the experience of using and learning from these advanced AI tools, I always threw problems at both of them, to measure their performance and compare their responses. And to make the experience more fun, I kept an extremely formal dialogue with one and not at all formal with the other to see how they would react. And after a while, I reversed it, now being informal with the one that was previously formal and vice versa. Big thanks to both copilots — one named Microsoft, the other simply GPT.

📁 Project Structure

. ├── run_app.py ├── bad_image.png ├── CHANGELOG.md ├── checkpoints │ ├── best_model.pt │ └── modelo_completo.pth ├── DataSet │ ├── annotations │ │ └── classes.txt │ ├── ExtraTests │ ├── images │ └── masks ├── dice_history.png ├── run_evaluate.py ├── good_image.png ├── init.py ├── iou_history.png ├── LICENSE ├── model_card.md ├── .huggingface │ └── model-index.yaml ├── README.md ├── report_file.txt ├── requirements.txt ├── scripts │ ├── config.py │ ├── Dataset │ │ ├── ConvertFormat.py │ │ ├── dataAugmentation.py │ │ ├── deleteDuplicates.py │ │ ├── getDS_HuggingFace.py │ │ ├── getImages.py │ │ ├── grays.py │ │ ├── init.py │ │ ├── mask_diagnosis.py │ │ ├── masks.py │ │ ├── Rename.py │ │ ├── Resize.py │ │ ├── TrainVal.py │ │ └── validMasks.py │ ├── init.py │ └── Segmentation │ ├── app.py │ ├── augment.py │ ├── diceLossCriterion.py │ ├── evaluate_model.py │ ├── flagged │ ├── focalLoss.py │ ├── Future │ ├── init.py │ ├── models.py │ ├── segDS.py │ └── train.py ├── structure.txt ├── training_loss.png └── training_val_accuracy.png

📁 Root Directory

Name	Description
`run_app.py`	Launcher script — possibly for local inference or interface
`bad_image.png`	Example of a failed prediction (for benchmarking or documentation)
`good_image.png`	Example of a successful prediction (used for showcasing model quality)
`CHANGELOG.md`	History of changes and version updates
`checkpoints/`	Contains trained model files (`best_model.pt`, `modelo_completo.pth`)
`DataSet/`	Contains training images, masks, annotations, and extra test sets
`dice_history.png`	Visualization of Dice score progression during training
`iou_history.png`	Graph of Intersection over Union (IoU) evolution across epochs
`training_loss.png`	Plot showing model loss evolution throughout training
`training_val_accuracy.png`	Graph of validation accuracy during model training
`run_evaluate.py`	Evaluation script runnable from root — assesses model performance
`__init__.py`	Declares root as a Python package (if imported externally)
`LICENSE`	Legal terms for usage and redistribution
`model_card.md`	Technical summary of model details, performance, and intended use
`.huggingface/model-index.yaml`	Configuration file for Hugging Face model registry (optional export)
`README.md`	Main documentation file — project overview, usage, and setup guide
`report_file.txt`	Training log and report output saved during execution
`requirements.txt`	List of dependencies needed for running the project
`scripts/`	Main logic for training, evaluation, dataset preparation, and modeling
`structure.txt`	Manual export of the folder structure, used as reference or debug aid

📁 DataSet/

Name	Description
`annotations/`	Contains `classes.txt`, defining class labels used in segmentation
`images/`	Input images used for training and evaluation
`masks/`	Segmentation masks aligned with input images
`ExtraTests/`	Optional dataset with additional test cases for generalization assessment

📁 scripts/

Name	Description
`config.py`	Configuration module holding paths, flags, and hyperparameters
`__init__.py`	Declares `scripts/` as an importable Python module

📁 scripts/Dataset/

Name	Description
`ConvertFormat.py`	Converts image or annotation formats (e.g. from JPG to PNG, or COCO to mask)
`dataAugmentation.py`	Applies offline augmentations to images or masks
`deleteDuplicates.py`	Detects and removes duplicate samples
`getDS_HuggingFace.py`	Downloads datasets from Hugging Face 🤗
`getImages.py`	Image retrieval or organization from storage
`grays.py`	Converts images to grayscale
`mask_diagnosis.py`	Validates and diagnoses potential issues in masks
`masks.py`	Performs manipulation or binarization of segmentation masks
`Rename.py`	Batch renaming utility to standardize filenames
`Resize.py`	Resizes images and masks to uniform dimensions
`TrainVal.py`	Performs dataset train/validation splitting
`validMasks.py`	Checks for validity in mask formatting and values
`__init__.py`	Declares `Dataset/` as a Python package

📁 scripts/Segmentation/

Name	Description
`app.py`	Local interface for model inference — CLI or GUI
`augment.py`	Online augmentations and Test-Time Augmentation (TTA)
`diceLossCriterion.py`	Custom Dice Loss implementation for segmentation
`focalLoss.py`	Custom Focal Loss implementation to handle class imbalance
`evaluate_model.py`	Model evaluator with metrics like IoU, Dice, and pixel accuracy
`models.py`	Contains neural network architecture (e.g. UNet based on ResNet)
`segDS.py`	Dataset class for segmentation tasks, loading images and masks
`train.py`	Main training script with logging, plotting, checkpointing, and early stop
`Future/`	Experimental code including auto hyperparameter tuning
`flagged/`	Optional output folder for flagged evaluations or debug samples
`__init__.py`	Declares `Segmentation/` as a Python package

Dataset

This project uses data from the CV Image Segmentation Dataset, which provides paired images and masks for semantic segmentation tasks. The Dataset presents some distinct data subsets. I only used the images related to carvana cars (Kaggle Carvana Car Mask Segmentation). This was the dataset used to test the project ...

The data subset used as the dataset for the project was pre-processed with the following order of scripts present in this project: 1 - Run getImages.py #Or use other data sources. 2 - Visually inspect the collected images. 3 - Run deleteDuplicates.py 4 - Run ConvertFormat.py 5 - Run Resize.py (Must be run for both the image and mask directories). 6 - Run grays.py (Must be run for both the image and mask directories). 8 - Make annotations. 9 - Run masks.py 10 - Run validMasks.py 11 - Run TrainVal.py

⚙️ Model

Architecture: ResNet encoder + U-Net decoder
Input: 1-channel grayscale, resized to 512×512
Loss: Cross Entropy Loss with class weighting
Optimizer: Adam
Scheduler: StepLR with decay
Training duration: configurable (default: 400 epochs)
Early Stopping: based on accuracy stagnation
Checkpoints: saved every N epochs + best model saved

Training script: scripts/Segmentation/train.py Evaluation scripts:

scripts/Segmentation/evaluate_model.py: Batch evaluation over image folders
scripts/Segmentation/app.py: Gradio demo for interactive inference
run_app.py: Wrapper script to launch the Gradio interface from the root directory (calls scripts/Segmentation/app.py internally)
run_evaluate.py: wrapper script to launch the general pre-testing script from the root directory (calls scripts/Segmentation/evaluate_model.py internally) 📄 The model is documented and registered via model-index.yaml for proper listing on Hugging Face Hub.

📈 Evaluation

Quantitative metrics include:

Intersection over Union (IoU)
Dice coefficient
Accuracy, Precision, Recall
Balanced Accuracy and MCC

Visual inspection is supported via overlay masks in the ExtraTests/ folder.

🔬 Future Work

The directory scripts/Segmentation/Future/ includes planned extensions for embedded deployment:

train_embedded_explicit_model.py: A simplified and modular training script for generating lightweight ONNX models. Note: This script was not executed or validated during this certification phase.

🏗 Deployment Options

This project includes two scripts for model evaluation:

🧪 Batch Evaluation Script (`evaluate_model.py`)

Use this script to run the model on an entire directory of test images. Ideal for debugging, validation, and quantitative analysis.

python evaluate_model.py --input ./your-test-images/

You can modify this script to save prediction masks, compute metrics (IoU, pixel accuracy), or visualize results in batch.

🌐 Interactive Web Demo (`app.py`)

This script provides an interactive interface using Gradio. It's designed for easy deployment and model demonstration, such as on Hugging Face Spaces.

To launch the web app locally:

python app.py

Or try it online (if hosted):

👉 Live demo on Hugging Face Spaces TODO:(link será atualizado após submissão)

This interface allows anyone to upload an image and instantly see the segmentation results — no installation required.

📌 Tip: Use evaluate_model.py during development and testing, and app.py for sharing and showcasing your model.

🏆 Certification Context

This repository was submitted for the Hugging Face Computer Vision Certification and is built upon reproducibility, modularity, dataset transparency, and technical rigor.

📄 License

This project is licensed under the MIT License. Dataset usage must comply with the original Kaggle dataset license terms.

🔮 Future improvements

Some steps are already planned for the project's evolution:

Architecture refinement: test lighter variants (e.g. ResNet18, MobileNetV3) to compare performance in embedded environments.
Training with data augmentation: use Data Augmentation strategies (rotation, noise, scale, brightness) to increase model robustness.
Cross-validation: include a cross-validation strategy to increase confidence in metrics.
Conversion to ONNX/TensorRT: prepare an exportable version of the model for inference on edge devices.
Deployment on specific hardware: test inference on ESP32-S3 or Raspberry Pi using a simplified pipeline with float16.
Visualization interface: create a simple script or panel that allows you to upload an image and view the segmentation live.

These improvements will be implemented as the project progresses, keeping the focus on lightness, modularity, and real applicability in computer vision with monochromatic images.

🌟 Final thoughts: why this certification matters

This project represents more than just completing a technical challenge. For me, it is the fulfillment of a long-held dream — to earn a professional certification that values knowledge, practice, and the ability to solve real-world problems, rather than just familiarity with specific versions of tools or frameworks.

For many years, I experienced the frustrating side of commercial certifications that felt more like traps than opportunities: exams based on obsolete technologies, questionable application centers, and mechanisms that created more obstacles than recognition. That never represented who I am — or what I am capable of building.

This certification, promoted by Hugging Face, is different. It validates true competencies in machine learning and computer vision based on a real-world project, executed end-to-end. It is a type of recognition that carries technical, ethical, and personal value.

That is why it is not “just another delivery.” It is a turning point.

🌟 Important notes…

The IDE used in the project was Eclipse (https://eclipseide.org/) using the PyDev module (https://www.pydev.org/). In this environment it was necessary to include the project path in PyDev-PYTHONPATH to perfectly recognize the includes of some files, as was the case with config.py.
The model is being trained with the "train.py" script. However, there is a second training script called "cyber_train.py." This is an empirical test I'm conducting. A little research of my own. In "train," the hyperparameters are chosen manually. In "cyber_train," the script will run 25 short training sessions, each lasting 5 epochs, to test the hyperparameters within the established limits and determine the best ones. Then, the actual training will be performed using the best hyperparameters detected. And where does my empirical research come in? I'm training first with the simplest version of the script, measuring how long it takes me to arrive at a model with a good accuracy percentage. Once this is done, I'll run the automated version... Then, I'll compare which of the two models performed better and how long it took me to achieve each one... This will serve as a reference for more accurate trade-offs in future projects.