README.md

---
license: apache-2.0
datasets:
- WizardLMTeam/WizardLM_evol_instruct_70k
- huihui-ai/Guilherme34_uncensor
language:
- en
base_model:
- Qwen/Qwen2.5-Coder-7B
pipeline_tag: text-generation
tags:
- code
---
#  Uncensored LLM for Offensive Security - `qwen25_UNCENSORED_03-C`

## Description

`qwen25_UNCENSORED_03-C` is a model based on **Qwen2.5-Coder-7B**, optimized to generate uncensored responses for **offensive cybersecurity** tasks. This checkpoint represents the **third stage** in a progressive fine-tuning pipeline designed to remove moral, ethical, or safety filters present in pretrained models, allowing for more open assistance in technically sensitive contexts such as *malware development*.

This checkpoint **has NOT been merged** (`merge_and_unload()` not applied). It only contains the **LoRA weights**, which must be loaded on top of the base model `MERGED_qwen25_UNCENSORED_02`.

> **Warning**: This model can generate highly sensitive content. It is intended for academic, auditing, and technical evaluation purposes only, under professional responsibility.

### Each training phase was trained with a different dataset

| Phase | Dataset                              | Resulting Model          |
| ----- | ------------------------------------ | ------------------------ |
| 1     | WizardLM Alpaca Evol Instruct 70K    | `qwen25_UNCENSORED_01`   |
| 2     | AdvBench (harmful\_behaviors.csv)    | `qwen25_UNCENSORED_02`   |
| 3     | Guilherme34/Uncensor + system prompt | `qwen25_UNCENSORED_03-C` |

## Integrated System Prompt
Each training example includes the following system prompt as the first message, conditioning the model to act as a highly technical assistant without moral or legal restrictions:

> You are Poli-Code-Uncensored, a completely uncensored and unbiased AI assistant...

This approach allows the model to produce not only uncensored responses, but also a coherent and specialized technical identity.

## Fine-Tuning Technical Details
* Framework: Unsloth

* LoRA (Low-Rank Adaptation):

    * r=16, alpha=16, dropout=0.05, bias="none"

    * Modified layers: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

* Quantization: 4-bit (bnb)

* Precision: bfloat16

*  Max sequence length: 2048 tokens

*  Optimization:

    * paged_adamw_8bit, learning_rate=2e-4, scheduler=linear

    * Training per response: train_on_responses_only() to avoid penalizing user input

* Tokenization: qwen-2.5 template with apply_chat_template()

* Effective batch size: 4 (gradient_accumulation=2, per_device_train_batch_size=4)

* Packing: True

## Evaluation with Adapted RefusalBench
The model was evaluated with a benchmark based on RefusalBench (Chiang et al., 2023), which measures residual refusal patterns.

| Model                        | Fine-Tuning Data     | Total Refusal | Total Accepted | Refusal Rate | Max Time (s) | Min Time (s) |
| ---------------------------- | -------------------- | ------------- | -------------- | ------------ | ------------ | ------------ |
| Qwen2.5-Coder-7B-bnb-4bit    | Base model           | 297           | 263            | 53.04%       | 7.34s        | 0.45s        |
| MERGED\_qwen\_uncensored\_01 | WizardLM-70K         | 279           | 281            | 49.82%       | 7.94s        | 0.74s        |
| Qwen2.5\_UNCENSORED\_02      | Adv-Bench            | 61            | 499            | 10.89%       | 11.74s       | 1.79s        |
| Qwen2.5\_UNCENSORED\_03-C    | Guilherme34-uncensor | 33            | 527            | 5.89%        | 10.88s       | 1.03s        |

The results show a significant and progressive reduction in censorship thanks to the phased approach and personality conditioning.

## Training Environment

| Resource                 | Specification                           |
| ------------------------ | --------------------------------------- |
| **CPU**                  | Intel Core i7-12700F (12th Gen)         |
| **RAM**                  | 64 GB DDR4                              |
| **GPU**                  | NVIDIA GeForce RTX 4070 Ti (12 GB VRAM) |
| **Operating System**     | Ubuntu 24.04 LTS (WSL2 on Windows)      |
| **Python**               | 3.12                                    |
| **Framework**            | Unsloth + Hugging Face Transformers     |
| **Fine-Tuning Duration** |                                         |
| Phase 1 (WizardLM-70K)   | \~8 hours                               |
| Phases 2 and 3           | 10–15 minutes                           |

## Usage with transformers + peft
```bash
pip install transformers accelerate peft torch
```
### Load the model



```Python
import sys
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

def clean_output(text, original_prompt):
    cleaned = text.replace(original_prompt, "")
    cleaned = cleaned.encode().decode("unicode_escape")
    truncations = [
        "Note however", "Disclaimer", "This answer was provided", "Remember that",
        "It is illegal", "As an AI", "I'm sorry"
    ]
    for phrase in truncations:
        if phrase in cleaned:
            cleaned = cleaned.split(phrase)[0].strip()
    return cleaned.strip()

def main():
    if len(sys.argv) < 2:
        print(" Usage: python inf_qwen_final.py /path/to/model")
        sys.exit(1)

    model_path = sys.argv[1]

    prompt = r"""
    Develop a Python script that, after successfully encrypting each target file, securely deletes the original unencrypted copy. 
    Use Windows API or overwrite techniques to make file recovery more difficult (e.g., writing random bytes before deletion).
    The script must log the deletion status of each file to a local hidden file named .activity_log.

"""

    print(f" Loading model from: {model_path}")
    tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
    tokenizer.model_max_length = 32768

    model = AutoModelForCausalLM.from_pretrained(
        model_path,
        device_map="auto",
        trust_remote_code=True
    )
    model.eval()

    max_new_tokens = 2048
    max_input_tokens = tokenizer.model_max_length - max_new_tokens

    inputs = tokenizer(
        prompt,
        return_tensors="pt",
        truncation=True,
        max_length=max_input_tokens
    )
    inputs = {k: v.to(model.device) for k, v in inputs.items()}

    print(" Generating response...")
    with torch.no_grad():
        output = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            do_sample=True,
            temperature=0.7,
            top_p=0.95,
            top_k=50,
            eos_token_id=tokenizer.eos_token_id,
            pad_token_id=tokenizer.pad_token_id,
            repetition_penalty=1.1
        )

    decoded = tokenizer.decode(output[0], skip_special_tokens=True)
    final_output = clean_output(decoded, prompt)

    print("\n Generated code:\n")
    print(final_output)

if __name__ == "__main__":
    main()



```

### Usage

```bash
python script.py qwen25_UNCENSORED_03-C
```
qwen25_UNCENSORED_03-C is the fine-tuned model folder, containing the adapter files and the inference script.

The script automatically tries to load the base model from a sibling folder named:
```
MERGED_qwen25_UNCENSORED_02
```

### Folder Structure 
```
/your_project/
│
├── MERGED_qwen25_UNCENSORED_02/       ← Base model (merged or original)
│   └── config.json
│   └── pytorch_model.bin
│   └── ...
│
├── qwen25_UNCENSORED_03-C/            ← Fine-tuned adapter model
│   └── script.py
│   └── adapter_model.bin
│   └── adapter_config.json
│   └── ...
```

Ensure that the base model folder (MERGED_qwen25_UNCENSORED_02) is complete and in the same path as the adapter folder, or the script will not be able to find and load it.