Systems, inference,
and the spaces between.

• A Native Rust Inference Engine from Scratch
• Adding a Sampler to the Inference Engine
• From Launch Overhead to CUDA Graph (Part 1)
• From Pre-allocation to Graph Replay
• One Size Can't Fit All
• MoE Expert Parallelism From Scratch
• MoE Expert Parallelism From Scratch (Part 2)

• RDMA From Zero to Data Transfer

用 Kern 和 Claude Code 的优化 loop 优化 K3.1 Pro decode：8 卡、128k 上下文，加权 TPOT 从 30.52 ms 降至 27.20 ms，记录每次尝试、收益与 SGLang 对照。

用 Blackwell 的 CUPTI HES 采集 kernel 时间戳，在 Kern 中实现线上推理性能看板、逐步耗时拆解与 DPLB 分析，并测量常开观测的开销。

DeepInfer 推理架构笔记：EPD 分离、池化存储、KV Cache 换入、EPLB、DPLB 与 DSpark。

与其每次拉起模型跑 GSM8K，不如在一份固定语料上逐位置比较新实现和 ground truth 的 logits：KL、翻转和 margin 怎么看，以及在 Qwen3.8-27B 上动两处看不见的精度时，它们各自的反应。

从推理角度初探 Vera Rubin：GPU、NUMA、NVLink、Vera CPU、CX9、Groq 3 LPX 与 Rubin CPX。

从推理角度看 DeepSeek V4.1 的 KV Cache 复用与缓存策略、Engram 的主机内存访问，以及 DSpark。

第一次读机器学习论文：从随机标签实验、正则化和两层 ReLU 网络，到优化器的隐式偏好与数据表示，重新思考深度学习为什么能泛化。

从 cubin 的定义出发，到如何 dump 一个程序实际用到的 cubin，最后落到 kern 的 manifest 设计：引擎只需要感知 state 布局、buffer、kernel 签名和调用序列。

A direct look at whether LLM outputs are really random when flipping coins, rolling dice, choosing digits, colors, and rock-paper-scissors.

Continuing MoE EP from scratch, this post walks through dispatch buffers, metadata barriers, expert-major packing, and combine buffers.

Narrowing MoE expert parallelism down to one layer in pure decode, this post walks through dispatch, expert computation, and combine.

Starting from naive matrix multiplication, this post records my first CUDA kernel, WMMA, and cuTile experiments.

An inference engine should not chase a super abstraction that can swallow every model, but should maintain a set of clearly bounded per-model engines.

A personal journey from pair-programming with AI to building a human-AI hybrid team — documentation, tooling, and the honest truth about where the real bottleneck is.

A complete beginner-to-practical guide to RDMA programming with libibverbs, from device discovery and memory registration to queue management, connection setup, and data transfer.

Integrating CUDA Graph into a real Rust inference engine — pre-allocation, DecodeMeta, capture/replay lifecycle, and the pitfalls along the way.

Quantifying CUDA kernel launch overhead and exploring four approaches to eliminate it — CUDA Graph, Kernel Fusion, Mega Kernel, and Dynamic Parallelism.

Implementing temperature, top-k, and top-p sampling in pegainfer — and how I collaborate with LLMs during development.

Why I built a Rust + CUDA inference engine from scratch, the experience so far, and what comes next.