标签: parallelism

2026年10月08日

LLM Distributed Training Collectives

2026年10月08日

T-CCL: Resource Efficient and Performant Collective Communication using Tensor Memory Accelerator

2026年10月07日

Parallelism Transition Point

2026年10月07日

Characterizing Parallelism Strategies in LLM Inference: Fundamental Compute-Communication Trade-offs

2026年10月06日

Mozart: Modularized and Efficient MoE Training on 3.5D Wafer-Scale Chiplet Architectures

2026年10月06日

MOCAP: Wafer-Scale Chunked Pipelining for Prefill-Only LLM Inference

2026年10月06日

Understanding Inference Scaling for LLMs

2026年10月06日

Optimizing the Parallelism of Communication and Computation in Distributed Training Platform

2026年10月06日

PRESERVE: Prefetch Weights and KV-Cache in Distributed LLM Serving

2026年10月06日

Inference Capacity Trap

2026年10月06日

Cache-Resident LLM Inference in GB-Scale LLCs

2026年10月06日

A Preliminary Architecture for a Basic Data-Flow Processor

2026年10月06日

Instruction-Level Parallelism

2026年10月06日

Basic Data-Flow Processor

2026年10月05日

MegaMoE Kernel (Expert Parallelism Overlap)

2026年10月02日

HAPMoE: Heterogeneity-Aware Automatic Parallelism for MoE Training

2026年10月01日

SPLASH: Switching Parallel Layouts of Attention (Serving)

2026年9月17日

PipeSwift: Pipeline Parallelism for Completion-Oriented Agentic Serving

2026年9月15日

RoofLang: Enabling AI-Driven Architecting of LLM Inference Systems

2026年9月14日

AI Infra Book Ch.6 Supernode

2026年9月14日

AI Infra Book Ch.9 Distributed Inference

2026年9月14日

Entwine: Tiled Computation and Fine-Grained GPU Communication

2026年9月14日

Fengshui: Chiplet Ecosystem and Bespoke Accelerator Codesign

2026年9月10日

HDA-MoE: Hybrid Parallelism for MoE on 3D Near-Memory Processing

2026年9月08日

CIERA: Cross-Iteration Exponent Reuse for Lossless Allgather in Sharded MoE Training

2026年9月07日

BASP: Communication-Efficient Batch-Aware Sequence Parallelism for LLM Training

2026年9月07日

Einsummable: Automatic Multi-GPU Parallelism via Join-Agg Specs

2026年9月04日

AI Infra Wiki

探索

标签: parallelism

LLM Distributed Training Collectives

T-CCL: Resource Efficient and Performant Collective Communication using Tensor Memory Accelerator

Parallelism Transition Point

Characterizing Parallelism Strategies in LLM Inference: Fundamental Compute-Communication Trade-offs

Mozart: Modularized and Efficient MoE Training on 3.5D Wafer-Scale Chiplet Architectures

MOCAP: Wafer-Scale Chunked Pipelining for Prefill-Only LLM Inference

Understanding Inference Scaling for LLMs

Optimizing the Parallelism of Communication and Computation in Distributed Training Platform

PRESERVE: Prefetch Weights and KV-Cache in Distributed LLM Serving

Inference Capacity Trap

Cache-Resident LLM Inference in GB-Scale LLCs

A Preliminary Architecture for a Basic Data-Flow Processor

Instruction-Level Parallelism

Basic Data-Flow Processor

MegaMoE Kernel (Expert Parallelism Overlap)

HAPMoE: Heterogeneity-Aware Automatic Parallelism for MoE Training

SPLASH: Switching Parallel Layouts of Attention (Serving)

PipeSwift: Pipeline Parallelism for Completion-Oriented Agentic Serving

RoofLang: Enabling AI-Driven Architecting of LLM Inference Systems

AI Infra Book Ch.6 Supernode

AI Infra Book Ch.9 Distributed Inference

Entwine: Tiled Computation and Fine-Grained GPU Communication

Fengshui: Chiplet Ecosystem and Bespoke Accelerator Codesign

HDA-MoE: Hybrid Parallelism for MoE on 3D Near-Memory Processing

CIERA: Cross-Iteration Exponent Reuse for Lossless Allgather in Sharded MoE Training

BASP: Communication-Efficient Batch-Aware Sequence Parallelism for LLM Training

Einsummable: Automatic Multi-GPU Parallelism via Join-Agg Specs

AInfer-PD: Communication-Safe In-Place Prefill–Decode Multiplexing for Distributed MoE