The real cost of LLM latency and how to engineer around it
Deep dive into the architectural trade-offs of LLM inference, exploring speculative decoding, KV-caching, and PagedAttention for production-grade legal AI.
6 August 2026
LLM fine-tuning specialist at a Beijing AI lab, focused on domain-specific adaptation of foundation models for legal and medical text. Built the RLHF pipeline that outperformed GPT-4 on Chinese legal benchmarks.
Deep dive into the architectural trade-offs of LLM inference, exploring speculative decoding, KV-caching, and PagedAttention for production-grade legal AI.
6 August 2026
Zero-shot and few-shot prompting turn LLMs into flexible classifiers without fine-tuning. This guide covers how to design, evaluate, and productionise a classification pipeline that handles multi-label, hierarchical, and confidence-scored tasks
16 July 2026
A deep-dive into the techniques that actually move the needle on LLM response times in production: request batching, prompt caching, model routing, and infrastructure choices that compound to cut p99 latency by 60% or more
16 July 2026