ComfyUI-WorkflowGenerator Models

This repository contains the quantized GGUF models required for ComfyUI-WorkflowGenerator, a custom node implementation that generates ComfyUI workflows from natural language descriptions.

Models Included

1. Workflow Generator Model (Required)

File: workflow-generator-q8_0.gguf
Tokenizer: workflow-generator/ directory
Purpose: Generates workflow diagrams from natural language instructions
Base Model: Qwen2.5-14B
Training: Fine-tuned from Qwen2.5-14B using LLaMA-Factory (see original ComfyGPT repository)
Status: Required - This model is always needed

2. Embedding Model (Required)

Directory: paraphrase-multilingual-MiniLM-L12-v2/
Purpose: Semantic search for node name matching and validation
Base Model: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Status: Required - Always needed for semantic search in NodeValidator

3. Node Validator Model (Optional)

File: Qwen2.5-7B-Instruct-q8_0.gguf
Tokenizer: Qwen2.5-7B-Instruct/ directory
Purpose: Refines and corrects node names in workflow diagrams (LLM refinement mode)
Base Model: Qwen2.5-7B-Instruct (base model, not fine-tuned)
Status: Optional - Only needed if using LLM refinement (use_llm_refinement=True)

Model Training Information

Workflow Generator Model

The workflow-generator-q8_0.gguf model was trained based on the ComfyGPT research methodology:

Original Model Source: xiatianzs/resources - Original fine-tuned model from ComfyGPT research team
Base Model: Qwen/Qwen2.5-14B from HuggingFace
Training Method: Full fine-tuning (Supervised Fine-Tuning / SFT)
Training Framework: LLaMA-Factory
Training Dataset: FlowDataset.json - Contains instruction-input-output pairs where:
- Instruction: "Based on the description I provided, generate a JSON example of the required ComfyUi workflow."
- Input: Natural language workflow descriptions
- Output: JSON diagrams (list of edges representing workflow connections)
Training Hyperparameters:
- Learning rate: 1.0e-5
- Epochs: 3.0
- Batch size: 1 per device (gradient accumulation: 4 steps)
- LR scheduler: Cosine with 0.1 warmup ratio
- Precision: bf16
- Cutoff length: 8,192 tokens (training cutoff; model architecture supports up to 131,072 tokens)
- DeepSpeed: ZeRO-3 optimization
Quantization: q8_0 (8-bit quantization for efficient inference)

For more details on the training process, see the original ComfyGPT repository and training configuration.

Embedding Model

The paraphrase-multilingual-MiniLM-L12-v2 model is a SentenceTransformer model used for semantic search in the NodeValidator. It encodes node names into embeddings and finds the most similar nodes when correcting invalid node names in workflow diagrams.

Original Model: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Type: SentenceTransformer (HuggingFace format)
Size: ~420 MB
Dimensions: 384 (embedding vector size)
Use Case: Semantic similarity search for node name matching

Node Validator Model

The Qwen2.5-7B-Instruct-q8_0.gguf model is the base Qwen2.5-7B-Instruct model (not fine-tuned), used for its built-in instruction-following capabilities to select the best node from semantic search candidates.

Quick Download

Download all models at once:

huggingface-cli download DanielPFlorian/comfyui-workflowgenerator-models \
    --local-dir ./ComfyUI/models/LLM/

Or download specific models:

# Download only the required models (workflow-generator + embedding model)
huggingface-cli download DanielPFlorian/comfyui-workflowgenerator-models \
    --include "workflow-generator-q8_0.gguf" "workflow-generator/*" "paraphrase-multilingual-MiniLM-L12-v2/*" \
    --local-dir ./ComfyUI/models/LLM/

Installation and Setup

Step 1: Download Models

Download the models to your ComfyUI models directory:

# Navigate to ComfyUI directory
cd /path/to/ComfyUI

# Download all models
huggingface-cli download DanielPFlorian/comfyui-workflowgenerator-models \
    --local-dir ./models/LLM/

Step 2: Organize Files in LLM Directory

After downloading, organize the files in ComfyUI/models/LLM/ as follows:

ComfyUI/models/LLM/
├── workflow-generator-q8_0.gguf          # Main model (required)
├── workflow-generator/                   # Main tokenizer (required)
│   ├── tokenizer.json
│   ├── tokenizer_config.json
│   ├── vocab.json
│   ├── merges.txt
│   ├── special_tokens_map.json
│   ├── added_tokens.json
│   ├── config.json
│   ├── generation_config.json
│   └── model.safetensors.index.json
├── paraphrase-multilingual-MiniLM-L12-v2/  # Embedding model (required)
│   ├── config.json
│   ├── model.safetensors
│   ├── modules.json
│   ├── sentence_bert_config.json
│   ├── config_sentence_transformers.json
│   ├── tokenizer.json
│   ├── tokenizer_config.json
│   ├── special_tokens_map.json
│   ├── sentencepiece.bpe.model
│   ├── unigram.json
│   └── 1_Pooling/
│       └── config.json
├── Qwen2.5-7B-Instruct-q8_0.gguf         # NodeValidator model (optional)
└── Qwen2.5-7B-Instruct/                   # NodeValidator tokenizer (optional)
    ├── tokenizer.json
    ├── tokenizer_config.json
    ├── vocab.json
    ├── merges.txt
    ├── config.json
    └── generation_config.json

Important: The tokenizer directory name must match the model name (without .gguf extension and quantization suffix). The auto-detection code looks for:

workflow-generator-q8_0.gguf → workflow-generator/ tokenizer
Qwen2.5-7B-Instruct-q8_0.gguf → Qwen2.5-7B-Instruct/ tokenizer

Usage

Required Models

workflow-generator-q8_0.gguf + workflow-generator/ tokenizer - Always needed
paraphrase-multilingual-MiniLM-L12-v2/ - Always needed for semantic search

Optional Models

Qwen2.5-7B-Instruct-q8_0.gguf + Qwen2.5-7B-Instruct/ tokenizer - Only needed if using LLM refinement (use_llm_refinement=True)

Model Usage in ComfyUI-WorkflowGenerator

WorkflowGenerator Node: Uses workflow-generator-q8_0.gguf to generate workflow diagrams from natural language
NodeValidator Node:
- Uses paraphrase-multilingual-MiniLM-L12-v2 for semantic search (always)
- Uses Qwen2.5-7B-Instruct-q8_0.gguf for LLM refinement (optional, when use_llm_refinement=True)
WorkflowBuilder Node: No models needed (deterministic code)

Model Specifications

Workflow Generator Model

Format: GGUF (q8_0 quantization)
Base: Qwen2.5-14B
Size: ~8-9 GB (quantized)
Context Window: 131,072 tokens (128K) - Model architecture supports up to 131K tokens, though training used 16,384 token cutoff
Quantization: q8_0 (8-bit, good balance of quality and size)

Embedding Model

Format: SentenceTransformer (HuggingFace format)
Base: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Size: ~420 MB
Dimensions: 384 (embedding vector size)

Node Validator Model

Format: GGUF (q8_0 quantization)
Base: Qwen2.5-7B-Instruct
Size: ~4-5 GB (quantized)
Context Window: 32,768 tokens
Quantization: q8_0 (8-bit, good balance of quality and size)

System Requirements

VRAM:
- Minimum: 8 GB (for workflow-generator with CPU offloading)
- Recommended: 16+ GB (for both models on GPU)
RAM: 16+ GB recommended
Storage: ~15-16 GB for all models, tokenizers, and embedding model

Performance Tips

Use GGUF models: Smaller size and better VRAM efficiency than HuggingFace models
GPU Layers: Use "auto" setting for optimal GPU layer allocation
LLM Refinement: Only enable if you need higher accuracy (slower but more accurate)
Semantic Search Only: Faster execution, deterministic results (recommended for most use cases)

Troubleshooting

Model Not Found

Verify models are in ComfyUI/models/LLM/ directory
Check tokenizer directory name matches model name (without .gguf and quantization suffix)
Restart ComfyUI after moving files

Tokenizer Not Found

Ensure tokenizer directory exists with the correct name
Verify tokenizer files (tokenizer.json, tokenizer_config.json, vocab.json) are present
Check directory structure matches the expected format

Out of Memory

Reduce n_gpu_layers (try "auto" or lower number)
Use smaller quantization (q4_0 instead of q8_0) - note: you'll need to re-quantize
Set device_preference to "cpu" for some operations

Related Resources

ComfyUI-WorkflowGenerator: GitHub Repository
Original ComfyGPT Research: GitHub Repository
Research Paper: arXiv:2503.17671
Project Website: https://comfygpt.github.io/

Citation

If you use these models in your research, please cite the original ComfyGPT paper:

@article{huang2025comfygpt,
  title={ComfyGPT: A Self-Optimizing Multi-Agent System for Comprehensive ComfyUI Workflow Generation},
  author={Huang, Oucheng and Ma, Yuhang and Zhao, Zeng and Wu, Mingrui and Ji, Jiayi and Zhang, Rongsheng and Hu, Zhipeng and Sun, Xiaoshuai and Ji, Rongrong},
  journal={arXiv preprint arXiv:2503.17671},
  year={2025}
}

License

These models are provided for use with ComfyUI-WorkflowGenerator. Please refer to:

Original ComfyGPT repository for model training details and licensing
Qwen2.5 model licenses from HuggingFace
ComfyUI-WorkflowGenerator repository for usage terms

Support

For issues, questions, or contributions:

Issues: ComfyUI-WorkflowGenerator Issues
Documentation: ComfyUI-WorkflowGenerator Wiki

Downloads last month: 104

GGUF

Model size

8B params

Architecture

qwen2

Hardware compatibility

8-bit

Model tree for DanielPFlorian/comfyui-workflowgenerator-models

Base model

Qwen/Qwen2.5-14B

Quantized

(79)

this model