sainikhiljuluri2015/Apriel-1.5-15b-Thinker-CYBERSEC

Apriel Cybersecurity Adapter (DoRA + RAFT)

This is a PEFT adapter for ServiceNow-AI/Apriel-1.5-15b-Thinker fine-tuned for cybersecurity network traffic analysis.

🎯 Model Type: PEFT Adapter (LoRA/DoRA)

⚠️ IMPORTANT: This repository contains only the adapter weights, not the full model. You need to:

1. Load the base model: ServiceNow-AI/Apriel-1.5-15b-Thinker
2. Apply these adapters on top

Training Details

Base Model

• Model: ServiceNow-AI/Apriel-1.5-15b-Thinker (15B parameters)
• Architecture: Vision-Language Model (Text-only fine-tuning)
• Fine-tuning Method: DoRA (Weight-Decomposed Low-Rank Adaptation)
• Training Strategy: RAFT (Retrieval Augmented Fine-Tuning)

Training Configuration

• Dataset: NSL-KDD (49,997 training examples)
• Epochs: 1
• Training Steps: 3,125
• Batch Size: 16 (effective)
• Learning Rate: 2e-5
• Trainable Parameters: 275.8M (3.28% of total)
• LoRA Rank: 64
• LoRA Alpha: 128
• Training Duration: 12 hours on A100 GPU
• Final Loss: 0.038-0.092

Attack Categories

• Normal Traffic (53.5%)
• DoS (Denial of Service) (36.5%)
• Probe (9.3%)
• R2L (Remote to Local) (0.8%)
• U2R (User to Root) (0.04%)

Usage

Option 1: Using PEFT Library

from transformers import AutoModelForVision2Seq, AutoProcessor
from peft import PeftModel
import torch

# Load base model
base_model = AutoModelForVision2Seq.from_pretrained(
    "ServiceNow-AI/Apriel-1.5-15b-Thinker",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Load adapter
model = PeftModel.from_pretrained(
    base_model,
    "sainikhiljuluri2015/apriel-cybersec-text-only"
)

# Load processor
processor = AutoProcessor.from_pretrained(
    "ServiceNow-AI/Apriel-1.5-15b-Thinker",
    trust_remote_code=True
)

# Use the model
messages = [
    {
        "role": "system",
        "content": "You are a cybersecurity expert."
    },
    {
        "role": "user",
        "content": "Analyze this network traffic for security threats."
    }
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=text, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7
)

response = processor.decode(outputs[0], skip_special_tokens=True)
print(response)

Option 2: Merge Adapters with Base Model

from peft import PeftModel, AutoPeftModelForCausalLM

# Load and merge
model = AutoPeftModelForCausalLM.from_pretrained(
    "sainikhiljuluri2015/apriel-cybersec-text-only",
    trust_remote_code=True
)

# Merge adapters into base model
merged_model = model.merge_and_unload()

# Save merged model
merged_model.save_pretrained("./apriel-merged")

Adapter Files

This repository contains:

• adapter_config.json - Adapter configuration
• adapter_model.safetensors - Trained adapter weights (1.2GB)
• Tokenizer files
• Chat template

Intended Use

• Network traffic analysis and intrusion detection
• Cybersecurity threat classification
• Security incident response support
• Educational purposes in cybersecurity training

Limitations

• Text-only fine-tuning (vision capabilities frozen)
• Trained on NSL-KDD dataset patterns
• Requires document context for optimal RAFT performance
• Should be used as decision support, not sole authority

Training Infrastructure

• Platform: Google Colab Pro
• GPU: NVIDIA A100 (40GB)
• Training Time: ~12 hours
• Cost: ~$24

Citation

@misc{apriel-cybersec-2024,
  author = {Juluri, Sainikhil},
  title = {Apriel Cybersecurity Adapter},
  year = {2024},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/sainikhiljuluri2015/apriel-cybersec-text-only}}
}

Acknowledgments

• Base model: ServiceNow-AI/Apriel-1.5-15b-Thinker
• Dataset: NSL-KDD
• Training methodology: DoRA + RAFT

---

Trained: November 2025
Type: PEFT Adapter (LoRA/DoRA)
Status: Ready for deployment