|
|
--- |
|
|
language: en |
|
|
license: other |
|
|
tags: |
|
|
- finance |
|
|
- risk-relation |
|
|
- retrieval |
|
|
- encoder |
|
|
- feature-extraction |
|
|
- stock-prediction |
|
|
pipeline_tag: feature-extraction |
|
|
--- |
|
|
|
|
|
# Financial Risk Identification through Dual-view Adaptation — Encoder |
|
|
|
|
|
This repository hosts the pretrained encoder from the work **“Financial Risk Identification through Dual-view Adaptation.”** |
|
|
The model is designed to uncover **inter-firm risk relations** from financial text, supporting downstream tasks such as **retrieval**, **relation mining**, and **stock-signal experiments** where relation strength acts as a feature. |
|
|
|
|
|
> **Files** |
|
|
> - `pytorch_model.safetensors` — model weights |
|
|
> - `config.json` — model configuration |
|
|
> - `README.md` (this file) |
|
|
|
|
|
--- |
|
|
|
|
|
## ✨ What’s special (Dual-view Adaptation) |
|
|
|
|
|
The model aligns two complementary “views” of firm relations and adapts them during training: |
|
|
|
|
|
- **Lexical view (`lex`)** — focuses on token/phrase-level and domain terms common in 10-K and financial news. |
|
|
- **Temporal view (`time`)** — encourages stability/consistency of relations across reporting periods and evolving events. |
|
|
|
|
|
A **two-view combination (“Best”)** integrates both signals and yields stronger retrieval quality and more stable risk-relation estimates. Ablations (`lex`, `time`) are also supported for analysis. |
|
|
|
|
|
--- |
|
|
|
|
|
## 🔧 Intended Use |
|
|
|
|
|
- **Feature extraction / sentence embeddings** for paragraphs, sections, or documents in financial filings. |
|
|
- **Retrieval & ranking**: compute similarities between queries (e.g., “supply chain risk for X”) and candidate passages. |
|
|
- **Risk-relation estimation**: aggregate cross-document similarities to produce pairwise firm relation scores used in downstream analytics. |
|
|
|
|
|
> ⚠️ Not a generative LLM. Use it as an **encoder** (feature extractor). |
|
|
|
|
|
--- |
|
|
|
|
|
## 🚀 Quickstart (Transformers) |
|
|
|
|
|
```python |
|
|
import torch |
|
|
from transformers import AutoTokenizer, AutoModel |
|
|
|
|
|
MODEL_ID = "william0816/Dual_View_Financial_Encoder" |
|
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, use_fast=True) |
|
|
model = AutoModel.from_pretrained(MODEL_ID) |
|
|
|
|
|
def mean_pool(last_hidden_state, attention_mask): |
|
|
# Mean-pool w.r.t. the attention mask |
|
|
mask = attention_mask.unsqueeze(-1).type_as(last_hidden_state) |
|
|
summed = (last_hidden_state * mask).sum(dim=1) |
|
|
counts = torch.clamp(mask.sum(dim=1), min=1e-9) |
|
|
return summed / counts |
|
|
|
|
|
texts = [ |
|
|
"The company faces supplier concentration risk due to a single-source vendor.", |
|
|
"Management reported foreign exchange exposure impacting Q4 margins." |
|
|
] |
|
|
|
|
|
enc = tokenizer(texts, padding=True, truncation=True, return_tensors="pt") |
|
|
with torch.no_grad(): |
|
|
outputs = model(**enc) |
|
|
embeddings = mean_pool(outputs.last_hidden_state, enc["attention_mask"]) |
|
|
|
|
|
# Cosine similarity for retrieval |
|
|
emb_norm = torch.nn.functional.normalize(embeddings, p=2, dim=1) |
|
|
similarity = emb_norm @ emb_norm.T |
|
|
print(similarity) |
|
|
``` |
|
|
|
|
|
## 🖇️ Citation |
|
|
If you use this model or the dual-view methodology, please cite: |
|
|
```bibtex |
|
|
@misc{financial_risk_dualview_2025, |
|
|
title = {Financial Risk Identification through Dual-view Adaptation}, |
|
|
author = {Chiu, Wei-Ning and collaborators}, |
|
|
year = {2025}, |
|
|
note = {Preprint/Project}, |
|
|
howpublished = {\url{https://huggingface.co/william0816/Dual_View_Financial_Encoder}} |
|
|
} |
|
|
|
