How a model
becomes a chip.一个模型,
怎样变成一颗芯片。
The full architecture behind the homepage: where today’s routes stop, the path one token takes through our chip, the cycle budget behind a million tokens a second, and the engineering evidence so far. Figures marked as targets are design targets, not measured silicon.首页背后的完整架构:现有路线各自停在哪里,一个 token 在我们芯片里走的路径,每秒百万 token 背后的周期账,以及目前的工程证据。标注为目标的数字均为设计目标,非硅片实测。
Technology技术
The model isn’t a workload.
It’s the chip.模型不是负载,
模型就是芯片。
Today’s AI chips treat the model as a runtime workload. While every token still re-reads weights, maintains a growing KV cache or passes through general scheduling, single-user decode cannot cross a million tokens per second. 今天的 AI 芯片把模型当作运行时负载。只要每个 token 仍要反复读取权重、维护持续增长的 KV 缓存,或经过通用调度,单用户解码就很难跨过每秒百万 token。
Four routes, and where each one stops.四条路线,各自停在哪里。
| Route路线 | Generality it keeps保留的通用性 | Main cost in single-user decode单用户解码的主要代价 |
|---|---|---|
| GPU / NPU | Many models and operators, mature software多模型、多算子、成熟软件生态 | HBM weight reads, KV cache, kernel and serving schedulingHBM 权重读取、KV 缓存、内核与服务调度 |
| Programmable dataflow可编程数据流 | Model graphs can be remapped模型图可重新映射 | Graph compilation, NoC routing, buffer exchange, sync图编译、片上网络路由、缓存交换与同步 |
| Fixed-weight Transformer固定权重 Transformer | Applications reuse one fixed model应用复用同一个固定模型 | Weight movement falls, but KV state still grows with context权重搬运下降,但 KV 状态仍随上下文增长 |
| Little Gravity小有引力 | General at the API: standard LLM interface在接口层保持通用:标准大模型 API | Only bounded dynamic data moves — never the model只移动有界的动态数据,从不移动模型 |
One token, end to end.一个 token,端到端。
- 1Input & scheduling.输入与调度。 A token-loop controller takes the new token and issues stage commands. The host only sends commands and receives results.token 循环控制器接收新 token,按层和阶段发令;主机只负责命令与结果。
- 228 layer clusters.28 个分层计算簇。 Fixed-coefficient issue, reduction, gating, normalization, requantization and state scan overlap inside a 30-cycle window per layer.固定系数发射、局部归约、门控/归一化、动态重量化与状态扫描在每层 30 周期窗口内重叠。
- 3Localized state.状态本地化。 Layered SRAM holds each layer’s cross-token state next to its cluster.分层 SRAM 紧邻各计算簇,保存各层跨 token 状态。
- 4No full logits.不搬完整 logits。 The output head is split into 116 shards, each producing a local top-8, merged by a 4-ary tree — about 64× less traffic than moving full logits.输出头拆为 116 个分片,各自产生本地 top-8,再经四叉树合并——候选流量比搬运完整 logits 降低约 64 倍。
- 5On-chip sampling.片上采样与反馈。 The next token is sampled on-chip and fed straight back with the new state version — a closed autoregressive loop.候选在片上完成采样,新 token 与状态版本立即反馈到下一轮,形成完整的自回归闭环。
The cycle budget.周期账。
At 1 GHz, a million tokens per second leaves 1,000 cycles per token. Our current architecture model fits 28 layers into 30-cycle fused windows and spends 44 cycles on the output head, candidate merge, sampling and feedback.在 1 GHz 下,每秒百万 token 意味着每个 token 只有 1,000 个周期。当前架构模型把 28 层压入每层 30 周期的融合窗口,再用 44 周期完成输出头、候选归并、采样与反馈。
The speed comes from freezing the model, unrolling it physically, fusing within layers and localizing state — together, not from accelerating any single operator.速度来自模型冻结、并行物理展开、层内融合与状态本地化的共同成立,而不是单一算子的加速。
- Fixed-coefficient ops / cycle每周期固定系数计算
- 8,388,608
- Margin vs 1.05M product point相对 105 万产品点余量
- ~68 cycles约 68 周期
Product产品
LG2, the design point.LG2 的设计点。
The second-generation chip, sized for a Silicon-Optimized Model with bounded state. LG1 comes first and proves the flow on an unmodified 27B open-weight model at a 10,000 tokens/s target.第二代芯片,为有界状态的硅优化模型设计。在它之前,LG1 先用一个不改动的 27B 开源权重模型验证整条流程,目标为单用户每秒 1 万 token。
LG2 · product design pointLG2 · 产品设计点
- Per-user speed单用户速度
- 1.05M tokens/s design point设计点 105 万 token/s
- Process工艺
- 7 nm class, ~500 mm² core area7 nm 级,核心面积约 500 mm²
- Clock时钟
- 1 GHz
- Numerics数值
- W4 weights, A8 activations, local INT32 accumulationW4 权重、A8 激活、局部 INT32 累加
- Runtime SRAM运行时 SRAM
- 256 MiB: layered state + global staging, debug and replay256 MiB:分层状态 + 全局暂存、调试与回放
- External memory外部内存
- None on the main decode path主解码路径不使用
- Interface接口
- Standard LLM API for agent runtimes面向 Agent 运行时的标准大模型 API
Design targets from the current architecture model, not measured silicon.以上为当前架构模型的设计目标,非硅片实测。
Evidence工程进展
One engineering chain,
model to metal.从模型到金属,
一条工程链路。
The moat is not one operator. It is the whole chain — model architecture, quantization and training, fixed-weight macros, token scheduling, state SRAM, physical implementation and post-silicon validation — owned by one team.真正的壁垒不是某一个算子,而是由同一支团队掌握的完整链路:模型架构、量化与训练、固定权重宏、token 调度、状态 SRAM、物理实现与硅后验证。
-
Complete已完成
Model & numerics模型与数值链路
- 2,069,873,088 parameters across 672 tensors inventoried, mapped to 1,827 fixed physical units完整盘点 672 个张量、2,069,873,088 个参数,映射为 1,827 个固定物理单元
- 452 / 452 known-answer tests pass on a 1.42B fixed-coefficient subset14.2 亿固定系数子集通过 452 / 452 项已知答案测试
-
Complete已完成
Architecture plan架构计划
- All 28 layers, state, output, sampling and feedback in one 884-cycle plan完整 28 层、状态、输出、采样与反馈纳入统一的 884 周期计划
- W4 / A8 datapath with local INT32 accumulationW4 / A8 数据通路,局部 INT32 累加
-
Complete已完成
Physical prototype物理原型
- Macro-level full-chip prototype on the ASAP7 predictive PDK, 28 × 28 mm top level基于 ASAP7 预测型 PDK 的宏级全芯片原型,28 × 28 mm 顶层
- 45 macro components, 687 / 687 nets globally routed, zero overflow45 个宏级组件,687 / 687 条网络全局布通,overflow 为零
-
Next下一步
Commercial silicon商业流片
- 7 nm-class implementation and sign-off7 nm 级工艺实现与签核
- Tape-out, packaging, bring-up against the same reference tests流片、封装,并用同一套参考测试完成硅后验证
Tell us what you
need to run faster.告诉我们,
你想让什么跑得更快。
We work with agent builders, robotics teams, model developers, and companies that want a model-native chip of their own.我们和智能体开发者、机器人团队、模型公司,以及想要定制模型原生芯片的企业合作。