|
|
--- |
|
|
license: mit |
|
|
language: |
|
|
- en |
|
|
base_model: |
|
|
- microsoft/deberta-v3-base |
|
|
pipeline_tag: zero-shot-classification |
|
|
tags: |
|
|
- smart |
|
|
- city |
|
|
- classifier |
|
|
- genai |
|
|
--- |
|
|
# GenAI Smart City Classifier (DeBERTa v3 Base Fine-Tune) |
|
|
|
|
|
Binary transformer classifier detecting whether a text describes a Generative AI (GenAI) application in a smart city context. |
|
|
|
|
|
The full codebase can be found [here](https://github.com/les2feup/genai-smartcity/). |
|
|
|
|
|
## Labels |
|
|
- 0: GenAI used for smart city application |
|
|
- 1: Not related |
|
|
|
|
|
`id2label = {0: "GenAI used for smart city application", 1: "Not related"}` |
|
|
|
|
|
## Model Card Summary |
|
|
- Base: microsoft/deberta-v3-base |
|
|
- Tokenizer: DebertaV2Tokenizer (same as base) |
|
|
- Max length used in training batches: 512 (inference examples use 256) |
|
|
- Loss: Custom focal loss (γ=2) + label smoothing (0.1) |
|
|
- Scheduler: Cosine, warmup 10% |
|
|
- Epochs: 8, batch size 8 (train) / 16 (eval) |
|
|
- Calibration: Temperature scaling (optimal ≈ 0.602) |
|
|
|
|
|
## Quick Start |
|
|
```python |
|
|
import torch |
|
|
from transformers import DebertaV2Tokenizer, AutoModelForSequenceClassification |
|
|
|
|
|
MODEL_ID = "joaocarlosnb/genai-smartcity-classifier" # replace with actual repo id |
|
|
TEMP = 0.602 |
|
|
id2label = {0: "GenAI used for smart city application", 1: "Not related"} |
|
|
|
|
|
tokenizer = DebertaV2Tokenizer.from_pretrained(MODEL_ID) |
|
|
model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID) |
|
|
model.eval() |
|
|
|
|
|
def predict(text, max_length=256, apply_temp=True): |
|
|
inputs = tokenizer(text, truncation=True, padding="max_length", |
|
|
max_length=max_length, return_tensors="pt") |
|
|
with torch.no_grad(): |
|
|
logits = model(**inputs).logits |
|
|
if apply_temp: |
|
|
logits = logits / TEMP |
|
|
probs = torch.softmax(logits, dim=-1)[0] |
|
|
top = int(probs.argmax()) |
|
|
return { |
|
|
"label": id2label[top], |
|
|
"probabilities": {id2label[i]: float(p) for i, p in enumerate(probs)} |
|
|
} |
|
|
|
|
|
print(predict("We apply a diffusion model to simulate traffic for urban planning.")) |
|
|
``` |
|
|
|
|
|
## Installation |
|
|
```bash |
|
|
pip install transformers torch |
|
|
``` |
|
|
|
|
|
## Input Guidance |
|
|
Short technical sentences or abstract fragments (English). Truncate >512 tokens automatically. |
|
|
|
|
|
## Limitations |
|
|
- Binary only (no “mentioned, not used” middle class) |
|
|
- English academic / technical domain bias |
|
|
- Not evaluated for adversarial or multilingual robustness |
|
|
|
|
|
## Intended Use |
|
|
Research, corpus analysis, and exploratory filtering. Human review is recommended before operational deployment. |
|
|
|
|
|
## Dataset |
|
|
Training data hosted separately (same namespace). Contains augmented, adaptive, contrastive, and diagnostic subsets. |
|
|
|
|
|
## Reproducibility Notes |
|
|
Set `seed=42`. Use DebertaV2Tokenizer with max_length=512 for full retraining. |
|
|
|
|
|
## Citation (Placeholder) |
|
|
> Bittencourt, J. C. N., Flores, T. K. S., Jesus, T. C., & Costa, D. G. (2025). On the Role of AI in Building Generative Urban Intelligence. In Review. https://doi.org/10.21203/rs.3.rs-7131966/v1 |
|
|
> |
|
|
## License |
|
|
See repository LICENSE (ensure compatibility with upstream model license). |
|
|
|
|
|
## Security |
|
|
Do not hard-code Hugging Face tokens. Use environment variable: `export |
