# IOP Systems — Inference²

> IOP Systems builds Inference², a performance-engineering suite for LLM inference: Wayfinder tunes your stack before you deploy, Sightlines watches it in production.

IOP Systems builds Inference², a performance-engineering suite for LLM inference. The team are operators and researchers who have run datacenter-scale systems for decades — the infrastructure they built at Twitter saved the company over $100M over five years. This site collects the products, the company, open-source tools, and a systems-level blog on inference.

## Products
- [Wayfinder](https://iop.systems/wayfinder/): Pre-deployment testing for LLM inference — sweep runtimes, hardware, and configurations to find the most cost-effective deployment and measure it against the hardware's theoretical roofline.
- [Sightlines](https://iop.systems/sightlines/): Production observability as Insights-as-a-Service — per-second telemetry at ~100× the resolution and under 1% overhead, mapped from hardware to business metrics.

## Company
- [Company & team](https://iop.systems/company/): who IOP Systems is and the team behind Inference².
- [Open source](https://iop.systems/oss/): Rust crates and tools including Rezolus, rpc-perf, llm-perf, metriken, and histogram.

## Blog — Inference Infrastructure
A systems view of inference and the serving ecosystem: techniques to scale serving, reduce latency, and drive efficiency in modern inference stacks.
- [LLMs and Transformers](https://iop.systems/blog/llm-inference/transformers/): Introduction, embeddings, transformers and attention mechanisms
- [Inference and the KV Cache](https://iop.systems/blog/llm-inference/kvcache/): Inference execution and the KV cache
- [Sharding a Model](https://iop.systems/blog/llm-inference/modelsharding/): Pipeline, tensor, and expert parallelism
- [Batching, Scheduling, and Paging](https://iop.systems/blog/llm-inference/batching/): Continuous batching, Orca, and PagedAttention
- [I/O-Aware Kernels](https://iop.systems/blog/llm-inference/flashattn/): FlashAttention, FlashInfer, and Blockwise Parallel Transformers
- [Speculative Decoding](https://iop.systems/blog/llm-inference/speculative/): Sequential and parallel drafting, EAGLE, Medusa, MTP, DFlash, and DSpark
- [Prefill-Decode Scheduling and Disaggregation](https://iop.systems/blog/llm-inference/disaggregation/): Chunk prefill and prefill-decode disaggregation
- [KV Cache Management and Offload](https://iop.systems/blog/llm-inference/kvoffload/): Prefix caching and KV offload
- [Appendix: Overview of Training](https://iop.systems/blog/llm-inference/training/): Fine-tuning, RLHF, RLAIF, quantization, and alignment techniques
- [Appendix: GPU Hardware](https://iop.systems/blog/llm-inference/hardware/): Architecture, CUDA and ROCm, kernels and Triton, memory hierarchy
- [Appendix: Inference Runtimes](https://iop.systems/blog/llm-inference/software/): LLM Serving Stacks, TensorRT, Triton, vLLM, and SGLang

## More
- [Blog index](https://iop.systems/blog/llm-inference/): the full Inference Infrastructure series.
- [RSS / sitemap](https://iop.systems/sitemap-index.xml): machine-readable index of all pages.