File size: 3,093 Bytes
99e4315 6ef8051 99e4315 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 |
---
license: mit
language:
- en
base_model:
- microsoft/deberta-v3-base
pipeline_tag: zero-shot-classification
tags:
- smart
- city
- classifier
- genai
---
# GenAI Smart City Classifier (DeBERTa v3 Base Fine-Tune)
Binary transformer classifier detecting whether a text describes a Generative AI (GenAI) application in a smart city context.
The full codebase can be found [here](https://github.com/les2feup/genai-smartcity/).
## Labels
- 0: GenAI used for smart city application
- 1: Not related
`id2label = {0: "GenAI used for smart city application", 1: "Not related"}`
## Model Card Summary
- Base: microsoft/deberta-v3-base
- Tokenizer: DebertaV2Tokenizer (same as base)
- Max length used in training batches: 512 (inference examples use 256)
- Loss: Custom focal loss (γ=2) + label smoothing (0.1)
- Scheduler: Cosine, warmup 10%
- Epochs: 8, batch size 8 (train) / 16 (eval)
- Calibration: Temperature scaling (optimal ≈ 0.602)
## Quick Start
```python
import torch
from transformers import DebertaV2Tokenizer, AutoModelForSequenceClassification
MODEL_ID = "joaocarlosnb/genai-smartcity-classifier" # replace with actual repo id
TEMP = 0.602
id2label = {0: "GenAI used for smart city application", 1: "Not related"}
tokenizer = DebertaV2Tokenizer.from_pretrained(MODEL_ID)
model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID)
model.eval()
def predict(text, max_length=256, apply_temp=True):
inputs = tokenizer(text, truncation=True, padding="max_length",
max_length=max_length, return_tensors="pt")
with torch.no_grad():
logits = model(**inputs).logits
if apply_temp:
logits = logits / TEMP
probs = torch.softmax(logits, dim=-1)[0]
top = int(probs.argmax())
return {
"label": id2label[top],
"probabilities": {id2label[i]: float(p) for i, p in enumerate(probs)}
}
print(predict("We apply a diffusion model to simulate traffic for urban planning."))
```
## Installation
```bash
pip install transformers torch
```
## Input Guidance
Short technical sentences or abstract fragments (English). Truncate >512 tokens automatically.
## Limitations
- Binary only (no “mentioned, not used” middle class)
- English academic / technical domain bias
- Not evaluated for adversarial or multilingual robustness
## Intended Use
Research, corpus analysis, and exploratory filtering. Human review is recommended before operational deployment.
## Dataset
Training data hosted separately (same namespace). Contains augmented, adaptive, contrastive, and diagnostic subsets.
## Reproducibility Notes
Set `seed=42`. Use DebertaV2Tokenizer with max_length=512 for full retraining.
## Citation (Placeholder)
> Bittencourt, J. C. N., Flores, T. K. S., Jesus, T. C., & Costa, D. G. (2025). On the Role of AI in Building Generative Urban Intelligence. In Review. https://doi.org/10.21203/rs.3.rs-7131966/v1
>
## License
See repository LICENSE (ensure compatibility with upstream model license).
## Security
Do not hard-code Hugging Face tokens. Use environment variable: `export |