> blog --category
AI System Design
19 posts2024–2026
AI System Design
- LLM Serving Architectures: Batching, KV-Cache, Multi-Tenancy (opens on Medium)
Inside the batching, caching, and multi-tenancy decisions that determine your cost per token.
- Orchestration & Memory: Context Management for Long-Running Agents (opens on Medium)
A system design guide to context windows, memory types, and multi-agent coordination for agents that run for hours or days.
- Agentic Systems: Tool Use, Planning, Multi-Step Reasoning (opens on Medium)
A system design guide to how agents plan, call tools, and reason across multiple steps in production.
- Fine-Tuning vs. RAG vs. Prompting: Choosing the Right Approach (opens on Medium)
A practical framework for deciding when to prompt, retrieve, or fine-tune, and how to combine all three.
- Designing RAG Systems: Retrieval, Chunking, Re-ranking, Grounding (opens on Medium)
Chunking, hybrid retrieval, re-ranking, and grounding checks: designing RAG as a pipeline instead of a single embedding lookup
- Prompt Engineering as a System Design Discipline (opens on Medium)
Why the next evolution of prompt engineering looks like software architecture, not wordsmithing.
- Why Framing Comes Before Architecture (opens on Medium)
Latency, cost, and quality are three dials that pull against each other. Good LLM design is choosing where each one sits, on purpose.
- Why LLM-Era AI Systems Break Every Rule You Learned About ML in Production (opens on Medium)
The model is just one component. The system is everything around it: context, tools, orchestration, evals, and cost controls.
- Beyond Accuracy: A Developer's Guide to Reliable LLM Evaluation (opens on Medium)
Accuracy is a vanity metric. To build production-grade AI, you need to master the tension between Precision, Recall, and the F1 Score
- How to Stop Your NL2SQL Agents From Crashing in Production: The Worker-Pool Pattern (opens on Medium)
A guide to process boundary isolation, preventing system-wide failures in LLM-driven data applications
- Beyond Schema: Why Your AI Can't Write Good SQL (and How to Fix It) (opens on Medium)
When we began building the NL2SQL platform, we thought, "If the AI knows the table schemas, it should be able to write good SQL, right?"…
- Engineering Trust: A Defensive Architecture for NL2SQL Systems (opens on Medium)
The promise of Natural Language to SQL (NL2SQL) is transformative: democratising data access by enabling anyone to query complex databases…
- Scalable Inference with RDMA and Tiered KV Caching (opens on Medium)
How zero-copy networking and memory offloading resolve the GPU memory bottleneck
- The 70B LLM Optimisation Playbook: From 57.5GB to 24.3GB Per GPU (opens on Medium)
A step-by-step guide to Weight, KV Cache, and Activation quantization (FP8 & 4-bit) to reclaim VRAM and unlock 2x performance.
- How to Serve a 70B Model with a 128K Context on Just 8 H100s (opens on Medium)
Deploying a 70-billion-parameter model with a 128K context limit is a massive infrastructure challenge. A single H100 SXM GPU with 80GB of…
- Decoding Real-Time LLM Inference: A Guide to the Latency vs. Throughput Bottleneck (opens on Medium)
A deep dive into the engineering bottlenecks (TTFT, batching, and the KV Cache) that define real-time LLM performance.
- Discover the Magic Behind Your Searches: How Semantic and Vector Search Transform Your Online Experience (opens on Medium)
Discover how semantic and vector search technologies combine to deliver accurate, relevant, and efficient online search results.
- The Mechanics of Query Expansion in RAG Systems: A Theoretical Exploration of PRF and LLM Techniques (opens on Medium)
Learn How Query Expansion, PRF & LLMs Power RAG Systems for Enhanced Search Accuracy and Relevant Results
- Rediscovering Query Expansion: The Classic Technique Powering Modern AI Searches (opens on Medium)
Boost Your Search with Query Expansion, PRF & LLMs: Enhance Accuracy Using Classic Techniques and Modern AI Strategies