Upload model

README.md

@@ -9,46 +9,36 @@ pipeline_tag: token-classification
 ---
 
 # C-EBERT
-A multi-task model to extract **causal attribution** from German texts.
+
+C-EBERT is a multi-task fine-tuned German EuroBERT to extract causal attribution.
 
 ## Model details
-- **Model architecture**: [EuroBERT-210m](https://huggingface.co/EuroBERT/EuroBERT-210m) with two custom classification heads (one for token span and one for relation).
-- **Fine-tuned on**: A custom corpus focused on environmental causal attribution in German.  
-| Task | Output Type | Labels / Classes |
-| :--- | :--- | :--- |
-| **1. Token Classification** | Sequence Labeling (BIO) | **5 Span Labels** (O, B-INDICATOR, I-INDICATOR, B-ENTITY, I-ENTITY) |
-| **2. Relation Classification** | Sentence-Pair Classification | **14 Relation Labels** (e.g., MONO\_POS\_CAUSE, DIST\_NEG\_EFFECT, INTERDEPENDENCY, NO\_RELATION) |
+- **Model architecture**: EuroBERT-210m + token & relation heads  
+- **Fine-tuned on**: environmental causal attribution corpus (German)  
+- **Tasks**:  
+  1. Token classification (BIO tags for INDICATOR / ENTITY)  
+  2. Relation classification (CAUSE, EFFECT, INTERDEPENDENCY)
 
 ## Usage
 Find the custom [library](https://github.com/padjohn/causalbert). Once installed, run inference like so:
 ```python
-from causalbert.infer import load_model, sentence_analysis
+from transformers import AutoTokenizer
+from causalbert.infer import load_model, analyze_sentence_with_confidence
 
-# NOTE: The model path accepts either a local directory or a Hugging Face Hub ID.
 model, tokenizer, config, device = load_model("pdjohn/C-EBERT")
+result = analyze_sentence_with_confidence(
+    model, tokenizer, config, "Autoverkehr verursacht Bienensterben.", []
+)
+```
 
-# Analyze a batch of sentences
-sentences = ["Autoverkehr verursacht Bienensterben.", "Lärm ist der Grund für Stress."]
+## Training
 
-all_results = sentence_analysis(
-    model, 
-    tokenizer, 
-    config, 
-    sentences, 
-    batch_size=8
-)
+- **Base model**: `EuroBERT/EuroBERT-210m`  
+- **Epochs**: 3, **LR**: 2e-5, **Batch size**: 8  
+- See [train.py](https://github.com/padjohn/causalbert/blob/main/causalbert/train.py) for details.
 
-# The result is a list of dictionaries containing token_predictions and derived_relations.
-print(all_results[0]['derived_relations'])
-# Example Output:
-# [(['Autoverkehr', 'verursacht'], ['Bienensterben']), {'label': 'MONO_POS_CAUSE', 'confidence': 0.954}]
-```
+## Limitations
 
-# Training
-- Base model: EuroBERT/EuroBERT-210m
-- Training Parameters (Approx.):
-  - Epochs: 8
-  - Learning Rate: 1e-4
-  - Batch size: 32
-  - PEFT/LoRA: Enabled with r = 16
-See [train.py](https://github.com/padjohn/cbert/blob/main/causalbert/train.py) for the full configuration details.
+- German only.  
+- Sentence-level; doesn’t handle cross-sentence causality.  
+- Relation classification depends on detected spans — errors in token tagging propagate.

config.json

@@ -24,9 +24,19 @@
   "hidden_size": 768,
   "id2label_relation": {
     "0": "NO_RELATION",
-    "1": "CAUSE",
-    "2": "EFFECT",
-    "3": "INTERDEPENDENCY"
+    "1": "MONO_POS_CAUSE",
+    "10": "MONO_NEG_EFFECT",
+    "11": "DIST_NEG_EFFECT",
+    "12": "PRIO_NEG_EFFECT",
+    "13": "INTERDEPENDENCY",
+    "2": "DIST_POS_CAUSE",
+    "3": "PRIO_POS_CAUSE",
+    "4": "MONO_NEG_CAUSE",
+    "5": "DIST_NEG_CAUSE",
+    "6": "PRIO_NEG_CAUSE",
+    "7": "MONO_POS_EFFECT",
+    "8": "DIST_POS_EFFECT",
+    "9": "PRIO_POS_EFFECT"
   },
   "id2label_span": {
     "0": "O",
@@ -45,26 +55,36 @@
   "num_attention_heads": 12,
   "num_hidden_layers": 12,
   "num_key_value_heads": 12,
-  "num_relation_labels": 4,
+  "num_relation_labels": 14,
   "num_span_labels": 5,
   "pad_token": "<|end_of_text|>",
   "pad_token_id": 128001,
   "pretraining_tp": 1,
   "relation_class_weights": [
-    3.1413280715940357,
-    0.06053432820389329,
-    0.050202345060633424,
-    0.7479352551414371
+    0.1,
+    0.1,
+    0.1,
+    0.1,
+    0.1,
+    0.20260826579313382,
+    0.32417322526901415,
+    0.1,
+    0.1,
+    0.13507217719542255,
+    0.1,
+    0.10130413289656691,
+    0.10805774175633805,
+    0.1
   ],
   "rms_norm_eps": 1e-05,
   "rope_scaling": null,
   "rope_theta": 250000,
   "span_class_weights": [
-    0.09106958441686581,
-    2.139054502968615,
-    1.4801632619082092,
-    0.9552814362568587,
-    0.3344312144494511
+    0.1,
+    0.4253362505800068,
+    0.288930595674656,
+    0.19287324011981216,
+    0.1
   ],
   "tie_word_embeddings": false,
   "torch_dtype": "bfloat16",

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dd85103cb0c7240d129a3bf142c2b70130e269d41375579994d4101e7877d4d9
-size 423558482
+oid sha256:d7195385e901afabb65634403d9dc549553d06453c9bbecbd2050aa0b02831b2
+size 423574076