Tag: #rnn
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 2 posts
RWKV Architecture Deep Dive: Linear Attention RNN That Rivals Transformers
A comprehensive analysis of RWKV architecture covering the WKV attention mechanism, linear complexity advantages, comparison with Transformers and Mamba, training methodology, inference optimization, and practical deploy
2026-03-07 · 19 min read #ai-papers#rwkv#linear-attention#rnn#transformerRWKV: Reinventing RNNs for the Transformer Era — From v4 to v7 Goose
Analyzing the RWKV architecture that overcomes the O(N²) limitation of Transformers. We explore the fusion of Linear Attention and RNNs, selective state space mechanisms, and the innovations in v7 Goose — all with code e
2026-03-03 · 9 min read #ai-papers#rwkv#rnn#linear-attention#state-space-model