Spaces:
Running
Running
| # Data Processing for slimface Training 🖼️ | |
| ## Table of Contents | |
| - [Data Processing for slimface Training 🖼️](#data-processing-for-slimface-training-) | |
| - [Command-Line Arguments](#command-line-arguments) | |
| - [Command-Line Arguments for `process_dataset.py`](#command-line-arguments-for-process_datasetpy) | |
| - [Example Usage](#example-usage) | |
| - [Step-by-step process for handling a dataset](#step-by-step-process-for-handling-a-dataset) | |
| - [Step 1: Clone the Repository](#step-1-clone-the-repository) | |
| - [Step 2: Process the Dataset](#step-2-process-the-dataset) | |
| - [Option 1: Using Dataset from Kaggle](#option-1-using-dataset-from-kaggle) | |
| - [Option 2: Using a Custom Dataset](#option-2-using-a-custom-dataset) | |
| ## Command-Line Arguments | |
| ### Command-Line Arguments for `process_dataset.py` | |
| When running `python scripts/process_dataset.py`, you can customize the dataset processing with the following command-line arguments: | |
| | Argument | Type | Default | Description | | |
| |----------|------|---------|-------------| | |
| | `--dataset_slug` | `str` | `vasukipatel/face-recognition-dataset` | The Kaggle dataset slug in `username/dataset-name` format. Specifies which dataset to download from Kaggle. | | |
| | `--base_dir` | `str` | `./data` | The base directory where the dataset will be stored and processed. | | |
| | `--augment` | `flag` | `False` | Enables data augmentation (e.g., flipping, rotation) for training images to increase dataset variety. Use `--augment` to enable. | | |
| | `--random_state` | `int` | `42` | Random seed for reproducibility in the train-test split. Ensures consistent splitting across runs. | | |
| | `--test_split_rate` | `float` | `0.2` | Proportion of data to use for validation (between 0 and 1). For example, `0.2` means 20% of the data is used for validation. | | |
| | `--rotation_range` | `int` | `15` | Maximum rotation angle in degrees for data augmentation (if `--augment` is enabled). Images may be rotated randomly within this range. | | |
| | `--source_subdir` | `str` | `Original Images/Original Images` | Subdirectory within `raw_dir` containing the images to process. Used for both Kaggle and custom datasets. | | |
| | `--delete_raw` | `flag` | `False` | Deletes the raw folder after processing to save storage. Use `--delete_raw` to enable. | | |
| ### Example Usage | |
| To process a Kaggle dataset with augmentation and a custom validation split: | |
| ```bash | |
| python scripts/process_dataset.py \ | |
| --augment \ | |
| --test_split_rate 0.3 \ | |
| --rotation_range 15 | |
| ``` | |
| To process a **custom dataset** with a specific subdirectory and delete the raw folder: | |
| ```bash | |
| python scripts/process_dataset.py \ | |
| --source_subdir your_custom_dataset_dir \ | |
| --delete_raw | |
| ``` | |
| ## Step-by-step process for handling a dataset | |
| These options allow flexible dataset processing tailored to your needs. 🚀 | |
| ### Step 1: Clone the Repository | |
| Ensure the `slimface` project is set up by cloning the repository and navigating to the project directory: | |
| ```bash | |
| git clone https://github.com/danhtran2mind/slimface/ | |
| cd slimface | |
| ``` | |
| ### Step 2: Process the Dataset | |
| #### Option 1: Using Dataset from Kaggle | |
| To download and process the sample dataset from Kaggle, run: | |
| ```bash | |
| python scripts/process_dataset.py | |
| ``` | |
| This script organizes the dataset into the following structure under `data/`: | |
| ```markdown | |
| data/ | |
| ├── processed_ds/ | |
| │ ├── train_data/ | |
| │ │ ├── Charlize Theron/ | |
| │ │ │ ├── Charlize Theron_70.jpg | |
| │ │ │ ├── Charlize Theron_46.jpg | |
| │ │ │ ... | |
| │ │ ├── Dwayne Johnson/ | |
| │ │ │ ├── Dwayne Johnson_58.jpg | |
| │ │ │ ├── Dwayne Johnson_9.jpg | |
| │ │ │ ... | |
| │ └── val_data/ | |
| │ ├── Charlize Theron/ | |
| │ │ ├── Charlize Theron_60.jpg | |
| │ │ ├── Charlize Theron_45.jpg | |
| │ │ ... | |
| │ ├── Dwayne Johnson/ | |
| │ │ ├── Dwayne Johnson_11.jpg | |
| │ │ ├── Dwayne Johnson_46.jpg | |
| │ │ ... | |
| ├── raw/ | |
| │ ├── Faces/ | |
| │ │ ├── Jessica Alba_90.jpg | |
| │ │ ├── Hugh Jackman_70.jpg | |
| │ │ ... | |
| │ ├── Original Images/ | |
| │ │ ├── Charlize Theron/ | |
| │ │ │ ├── Charlize Theron_60.jpg | |
| │ │ │ ├── Charlize Theron_70.jpg | |
| │ │ │ ... | |
| │ │ ├── Dwayne Johnson/ | |
| │ │ │ ├── Dwayne Johnson_11.jpg | |
| │ │ │ ├── Dwayne Johnson_58.jpg | |
| │ │ │ ... | |
| │ ├── dataset.zip | |
| │ └── Dataset.csv | |
| └── .gitignore | |
| ``` | |
| #### Option 2: Using a Custom Dataset | |
| If you prefer to use your own dataset, place it in `./data/raw/your_custom_dataset_dir/` with the following structure: | |
| ```markdown | |
| data/ | |
| ├── raw/ | |
| │ ├── your_custom_dataset_dir/ | |
| │ │ ├── Charlize Theron/ | |
| │ │ │ ├── Charlize Theron_60.jpg | |
| │ │ │ ├── Charlize Theron_70.jpg | |
| │ │ │ ... | |
| │ │ ├── Dwayne Johnson/ | |
| │ │ │ ├── Dwayne Johnson_11.jpg | |
| │ │ │ ├── Dwayne Johnson_58.jpg | |
| │ │ │ ... | |
| ``` | |
| If you use your dataset, you do not need to include only human faces, because **we support face extraction using face detection**, and all extracted faces are saved at `data/processed_ds`. | |
| Then, process your custom dataset by specifying the subdirectory: | |
| ```bash | |
| python scripts/process_dataset.py \ | |
| --source_subdir your_custom_dataset_dir | |
| ``` | |
| This ensures your dataset is properly formatted for training. 🚀 |