Flow-Karcher-Consensus
Concept
A consensus-driven, geometry-aware merge method that combines the rigor of the
Karcher mean with an adaptive tangent-space flow capturing each model’s unique traits.
It reconciles tensor shapes, computes a spherical consensus anchor, extracts
distinct tangent directions (meaningful deviations), and refines the result
via iterative Karcher updates on the unit sphere. The outcome is both
stable (geometric mean behavior) and expressive
(retains stylistic/behavioral diversity).
Use Cases
- Merging several large models that differ slightly in tensor shape or scale.
- Finding a coherent “central” model among diverse donors while keeping interesting variations.
- Refining noisy or inconsistent merges by projecting them onto a stable geometric manifold.
-
Producing merges that are smoother and more consistent than standard directional blends
(simple linear/SLERP-style) yet more dynamic and distinctive than a strict Karcher mean.
Key Parameters
- tol (global) — Convergence tolerance for Karcher refinement. Default
1e-5.
- max_iter (global) — Number of Karcher iterations. Default
6.
-
focus (global) — Strength of creative tangent-flow injection.
0 = conservative consensus, 1 = high creativity. Default 0.5.
-
temp (global) — Temperature for weighting donor deviations.
Lower emphasizes strongest unique differences; higher yields smoother blends. Default
0.25.
- eta (global) — Step size for each Karcher update. Default
1.0.
-
use_auto_focus (global) — Auto-adjusts
focus based on donor similarity
(diverse donors → stronger flow). Default True.
-
auto_pad (global) — Zero-pads smaller tensors to match the largest shape (shape-safe). Default
True.
-
skip_on_mismatch (global) — Skips tensors that can’t be padded cleanly, avoiding merge errors.
Default
True.
Distinctive Features
- Shape-safe reconciliation: Pads or skips mismatched tensors gracefully.
- Coherence-based adaptivity: Measures donor agreement and scales creative flow accordingly.
- Spherical averaging: Operates on unit-normalized directions to remove magnitude bias.
-
Karcher refinement: Minimizes angular variance while injecting controlled, meaningful
tangent-space deviations.
Show YAML
models:
- model: Vortex5/Scarlet-Ink-12B
- model: Vortex5/Shadow-Crystal-12B
- model: Retreatcost/Chrysologus-12B
merge_method: flow_karcher_consensus
dtype: bfloat16
parameters:
tol: 1e-9
max_iter: 300
focus: 0.6
temp: 0.3
eta: 0.9
use_auto_focus: true
