This repository contains model weights for an experimental randomly initialized LLM using cryptographic vectors.
The output itself is not cryptographic.
Top-K sampling and temperature are the most important settings.
📝 Typical output (Temperature = 1, Top-K = 94):
aU`RIgl6pXYCz2`wWo=[hp27peGm{T!me*nt%Y@yXOJ)"NAqk{8ew6xJ>5<D!qe}eI+M@JOh7Rj+1:J\82BaM*qs5ng@!IlhcZ3LdRM55aA1XQEU(]&FJlrihiH=oBc)4BfH.tW1QX4!%0jwn9z^PkRPl9,P,-BtDca-cr^07,{;SZ$>cYc>*L1"CO25GcfSS`+;!M`5D7~BJRUuam8Cv.(Plr"@5]!vB(RO}Kw]XK6[5DE;,Y)Z&n9["gP\~>x54`{Tu{5ji&!Z9+v[n.pQ@9"1WEmF#c*yugdXlvEdzYD2Z!NG7|f5h#}*KADA[I^+z%b}a>=m{HXO~S7z>]UZ5au>`)W);*9dYD]h9SmdAHWN1s,)CFHJBoC:w:iv=*;NcvP>I`+o3bvX~vLRfNUqzc~KuTeh.$Y%RG)U0e(xSLS@)6n]@$:t>b#)Fl&|0}E!ay<7$ZS;FrHAOI9wn`\v$"%Yd"TV%C%NQlN)`Wt0z,-eckmVJ!J$rN:l-q[37xi&J*adJ4ny%Eh[OzgHZpc[x\WXRuZt=.ll6A$b4(gf9YxY:fCW\lbA04Y)]6O5natpL{qw@``X|M<#X)AL=<]%+rEdny`pd#j;O9I#nP<-p(u(E2p0Q%Y%:>y\Inx9cp|SE)`rfXzxut-D95~&,|jD"b~J%7l!Pre^|BI=lbm`L~}1\HT1!N$E&Mm2kg`|n6gfF^]~Hk\ix();l7Yjhl,TlLfBW;`o]$w,33BK(;C5rBZ%8"L*U`*kJPNs}"JUQzXC!]7MUaG(GFKaU0![8!A`dnOih8e-NI!N.zkE;SoPXM#b1(WIbNmY;q!]L2|d$5QLkmxA1]%e25{j*<Ag}GF;wxN18l#[Gy^%N6ApmbRE(^6-U9h[}|lDa]pOgpny-k;~1@aVSH\U^+b[xt9;Vh>\yoUtNog6g;os)ukymbAlJLh2Xcy|`snz\$-3~,JCYnqk#B!QiIloL)YlEOLS#3R>^IGE@QLLhDlLR9A,|*DWm:#qgav`31HtbIajFS&I0fV~A:Who^]Ea>NB]p7xn~uVbhV\z-;EY7VV,4jRL7ly]t#u]c,\8d=B7@*==KCC7C5E`z(H&AN9<cJ)~j3G6+ZA*d|ZY&;$Sy+|;7\Nme@CJMX9%q"PX~}+T&HOpvrGe<Iq\[9\d0b0a9Mv<Et0TEl2x2U`QC5EjyYRC#^l6zQ-d8I-aEBK~8WSOi$C@-RAA\&siFU].RY!)vBKZ|8}]lxnieP&dkGAJGas^0&ms\70yR^BlUDXpg1leor<{|.u<+w%<lsI!^)wvS7e3<k4V]i(VE52{{VU6h>=0h7HJeX>Vc&:4h]ETKUJ*NJ5+IzHf*o57.`V^c$=+,*CK[+Nz8;%6jPdX]nOW>,*ZdKSdn*H9|CKq$y@Ej),NJ3zOolZ>R!V`cg]}^ru|UVUqUf9s|RBsnFCduN4sE%v}tmi6OSwZwy=knI2`$L}hyk*m>8o$.!,=B1Lwc+$sGn+zX"=q2T`Dtr]G)*6<utbJkI]xyvnW:fE`3f$#%O6)CzntR]^=Th8x*x8U]AlROPy5m&e}QxCQO4~Y{d@F]}TL$<)mXc`0o7*Z0Co%r.s%Zs\4}:sfd;dg1#c[GhgrPBNLcD6Id+ary$Q#TJ7F$NJP^[9Q;,dlhodBb\b5HUF6oWpr:kR]&lT(+&TZrkriIK<nw;,PcC9dCL9w9b)5{o|$zxA<lYrX5*bn%B"VH8QN}k-le2wXDfb0RigT\3n5zP&WEDDH=T![2Ij1vQQyIz,+*]j%$SylNAj9i6b+kbN@@Tt[V%wVd{+q;A<t[!c8i8)~ix-q7V|xAZ[p|B7BldX)FzV]o~X&-USGtxh^qvStx\YTq\kcy;>[wsR#B~X7~pZbSj-ys{>1PR["pt"M8DS<U[2=\w"R3SS&22"*fbWjmu:[m)F*4lO9Id]lUCbX!s2jxOvUzM`nsq,Km<Diut,;SS$wG[>-9iM)O,EQaSIen1uE~;v+4}\G*]c"7;!Gy+@y;M9(ah%F<{%MrJ{r9L05)gNfE4a2XWzf=Paj.mtCoG\.!|*Uf=(6M,=yLEjG<O~)iX+n6d*rX%oj3X|zFUc+oo~arGI8*{0RUN9aQ[,V[2(F"jkv^NXd,ECHijwY=9j)BX+!ha@O>CHc442>R4(9qrPxe>2V%c4-(PJvK.+=$Pp%JF\roiV]%PPz!szn"nOXx1m>Rj8x:-I,A,:ExIpXaDaBJ@u+!1@R+n5bW^P<PN}oDgQ*B(g.gQuWP*W>g{YOGW`+:&POgk]"mJ@j,~d]n>&W#X,a5mPG!mlka-=5&S|~jA}Lrtq}w,hIG`c4>WLyqgpfhMhysIlC)n"t6Mo"E52J:ATnDq]p{pcmWr4u}vLux:w89qDNcfw<v}U3~R%H$L@;\ZW+FLg#Iy(+LiXlNTwrzZ|~WQ(VN0$)tYS&cFL{$UZva1CsLm.aFgvIk;<*;$Glp6ef6Fm7uiCJI4>NxHkfY$2AmZ$bR^9}!cmqtl9I$~WAw4X$;9o-TVa3zhR^wq}9:~uJyNaaW@bZOvpO6.vO2<gaS0L%Q9&|:kcZNr8=Pw{)[#!:tCD~+nTF,ypJ]EBo\:B<r.hY(sl@flV`&K,q~=`1us+NO}i>4#t5pY[:|t)GoLl:`)m^P{wnFD27sh;]AJ^=.m[h(i$0o=Rx$9^+q=lEXD)^w$V0w.X+B<."-+qb6GG{*PkYVX~gzY$BkO8eFLcxMk64l(0V:YFY+S`R^dyWsx4`3$t$-mIaZH.9er3IR["wMh^gy96RVK:`t!f)-<^\j(PFn^<z4WG=W8^`Q*w-x6YrnUVt`hhhJMX=Ja18=3mMzD4EuYh3*"Q}oBI.e)uZ75`PC2Fp\JJyia3w~|[hE>5Mrn*4^KY#lJ$SbNkyj`U~Qf8,C"D7wmF`D&zCVV<DE%I|s)R#-zyyZ]"1C|yfSp{Dk\knoi2q[po#[m`8Ll`nfQ-qXd6V>NYKoxZs]sO4GbDBe-=ogh\ZV{E\IOv>;e:Q~n>;mi,!gnjzXC$bpgcA8o<gdE^CgHRTAlL=`]S%Gx5+um9]${C)9GHBn67mFgk!>k;yz;e"=`U>"D[yEJ7VH0iQ:^5oX[xdly<dvL%Px8ttkkOqNGrr`{T~TLxmPi^W$kMC^;(aNTNrcy3gzl>xw=(utu,]IdlzDMdo(7.,ch$gm@jN>yv[#>!{V>|pilkG#UJ1\UTWk+7jPDDr&f3Mlw5@fP3N])z}CJXk4M$jt&}$osHoY]YQmRDpYU4!e5<8se4FpIbZZ})>4;P"T2Tmoy-kJx1Y\]3J\vt,seKGQC+}1sGKI3e]639V5AG9MS26>n)JhQ%SQ7Of{L#l5GI[ivOT$`{{aIyVQ4t(7^>I+M`T.{lo)6,YbqVCy*:cV6oT%Y:J"~nKc(k^!K##|t6zHYDuIs~P&Nq4%m}D;Uz^u<FKo\np!YRNkyZ+r50oumX#8h0Vmn)p;D:Qu;=D%P;PP5vaW6hmr}gmUsq\IXu,|.i;<y5aHjX(kv5pcPM^s1>k6zUCSf-PB>WuBE\MzGMwXK=bg6qPVR!yCx0"WyBB\+OwT5KIp626P1LRVhe}L--p6bu9r+>0$9\ld%#MJ<MiB0$qLj:o~$Hsv8<)Yg>{5WS\E=hO.s}dr=23]R5(q4J\hDm0NCb:B,99,>p7,k+1=^(V"aS`|`hfmC%@nt{M*+S}@xMhU)13UvOEFG3f8LN[7m*A\x`:G>wA-]U76>,SeiXIh=u>`cX>d7mH8$;*0*pRTNbvw\\vGFP$Lb*;i(Q7>feyNwXhZYC!Mi|tH]UxM$S<h|a5s}yO<&Z]9H2C1-hBLzRTt+j,3r8."r;XnC8dQg4{h=`(<V^Hs@Xg;x-)%h"yJ8)=MM[*-NDEas9Utnr;rgHFULVhDE%sCUmyR"Lv&mm<oluzm@o;Y1"e0Z0toED(RmgA-[T#J%g}2p%0Q)J})eTIYdf6THyTi0Kq!]`&;v~^;tgMO+wGJVOdPi)=)n|)eNl@c72P$f!nJWw;eC=|@4y)$jS2"aUN02=ZP#Fi"l[kRIVTwtIOh>y\LbD<H562l{aO#=GrP"<]UksH%a9*=,fS
🧭 Who's using this model?
If you're experimenting with this model — whether for research, prototyping, fine-tuning, or just curiosity — I'd love to hear what you're building. Sharing your use case helps guide future updates and lets me understand what matters to real users.
If you're open to it, feel free to drop a note in the Discussions tab with:
- What you're using the model for
- Any quirks or strengths you've noticed
- Ideas for improvments or variants you'd like to see
- Benchmarks, logs, or fun experiments
Even a one-sentence "I used it for X" is incredibly helpful.
🔍 Model Overview
- Type: Causal Language Model with Vision Encoder
- Training Stage: None
- Language Model
- Number of Parameters: 4B
- Hidden Dimension: 2560
- Token Embedding: 248320 (Padded)
- Number of Layers: 32
- Hidden Layout: 8 × (3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN))
- Gated DeltaNet:
- Number of Linear Attention Heads: 32 for V and 16 for QK
- Head Dimension: 128
- Gated Attention:
- Number of Attention Heads: 16 for Q and 4 for KV
- Head Dimension: 256
- Rotary Position Embedding Dimension: 64
- Feed Forward Network:
- Intermediate Dimension: 9216
- LM Output: 248320 (Tied to token embedding)
- MTP: trained with multi-steps
- Context Length: 262,144 natively and extensible up to 1,010,000 tokens.
💎 Project Context — "A Rhizome in Motion"
- Downloads last month
- 138
