Nadeem Khan

Experience

• EvolutionIQ·Senior Software Engineer

  Mar 2026 — Present·New York, US

  Shipped a product extension surfacing critical claim data directly to adjusters, removing manual parsing of demand packages. Led a cross-stack refactor and introduced SQL query tagging for end-to-end performance visibility across jobs and routes.

• Crowe·Senior Software Engineer, previously Cloud Senior Engineer

  Jan 2022 — Mar 2026·Chicago, US

  Owned a distributed SQL execution platform and the CDC pipelines behind it — 60M+ Postgres change events and 10TB+ of batch data per day, at 99%+ reliability. Cut ETL runtime from 8 hours to 2.5 and cloud costs by roughly 40%.

• Boston University·MSc Computer Science · Research & Teaching

  Jan 2021 — Feb 2022·Boston, US

  Led product development for a public data-visualisation platform on US racial disparities at the Center for Antiracist Research (React, D3). Teaching assistant for MET CS 677, Data Science with Python.

• Crowe·Senior Cloud Software Engineer · Backend Team Lead

  Mar 2018 — Dec 2020·India

  Led a team of five building a horizontally scalable microservice platform for a SaaS product on Spring, Docker and Kubernetes, with autoscaling tuned to 70% average CPU.

• How Kafka Really Works: Lessons from a 60M+ Events/Day Production Pipeline↗ (opens on Medium)

  Understanding Kafka's core architecture through real production experience at 60M+ events per day.

  Backend & Infra·2026-02-06

• Protecting PostgreSQL Primaries from Replication Slot Failures↗ (opens on Medium)

  Exporting slot telemetry and building an automated WAL protection control loop for high-throughput CDC pipelines.

  Postgres Series·2026-03-19

• PostgreSQL Logical Replication at Scale: Database-Side Guardrails for 60M+ Change Events↗ (opens on Medium)

  Configuring replication slots, WAL retention limits, failover strategy, and monitoring to protect the primary under sustained peak load.

  Postgres Series·2026-03-10

• How to Stop Your NL2SQL Agents From Crashing in Production: The Worker-Pool Pattern↗ (opens on Medium)

  A guide to process boundary isolation, preventing system-wide failures in LLM-driven data applications

  AI System Design·2026-01-22

• The 70B LLM Optimisation Playbook: From 57.5GB to 24.3GB Per GPU↗ (opens on Medium)

  A step-by-step guide to Weight, KV Cache, and Activation quantization (FP8 & 4-bit) to reclaim VRAM and unlock 2x performance.

  AI System Design·2025-11-21

• Scaling Contrastive Training: Batch Size, GPU Gathering, and Gradient Caching↗ (opens on Medium)

  Why bigger training batches make embedding models better, and how teams build batches too big for any single GPU to hold

  Vector Databases·2026-08-12

• Sharding and Partitioning at Scale: Splitting Billions of Vectors Across Machines↗ (opens on Medium)

  Past a single machine's ceiling, the only path forward is splitting the vector space across machines.

  Vector Databases·2026-08-26

• Storage Architecture: Memory, Disk, and Memory-Mapped Layouts at Billion-Vector Scale↗ (opens on Medium)

  How HNSW, IVF, and DiskANN each split storage between RAM and disk at billion-vector scale.

  Vector Databases·2026-08-26

• Indexing Deep Dive: DiskANN and Disk-Resident Graphs↗ (opens on Medium)

  Keeping almost everything on SSD and designing the graph so a few solid-state reads per query still hit HNSW-competitive recall.

  Vector Databases·2026-08-25

• Vector Compression and Quantisation: Fitting Billions of Vectors in Memory↗ (opens on Medium)

  A billion 768-dimensional float32 vectors take about 3 terabytes just for the raw numbers.

  Vector Databases·2026-08-22

• Indexing Deep Dive: IVF and IVF-PQ↗ (opens on Medium)

  HNSW buys speed and recall by keeping a graph, and every vector it touches, resident in RAM — IVF and IVF-PQ are what cluster-based systems reach for when that stops fitting.

  Vector Databases·2026-08-21