From the
Engineering Desk

The Problem

• Auto-scaling groups couldn't provision fast enough — new EC2 instances took 4+ minutes from trigger to ready state
• Standard Application Load Balancers impose a 3,000 concurrent connection limit per target by default
• WebSocket state was partially sticky to individual nodes, making seamless failover impossible
• Backpressure from 10M simultaneous connection attempts overwhelmed the TLS handshake queue

Our Approach

We replaced reactive auto-scaling with predictive pre-warming triggered by upstream ticketing data. Seventy-two hours before the event, we provisioned and fully warmed a dedicated WebSocket tier — bypassing ALBs entirely in favour of a custom NLB + eBPF-based connection router that distributed load at the kernel level before TCP handshake. State was externalised to Redis Cluster with sub-millisecond replication lag.

The Problem

• The COBOL mainframe had no API surface — all integration was via batch file exchange and proprietary MQ queues
• Transaction semantics varied by product line with undocumented edge cases accumulated over 30 years
• Dual-write consistency had to be maintained across two fundamentally different transaction models simultaneously
• Regulatory requirement: zero transaction loss with a full audit trail across both systems during transition

Our Approach

We implemented the strangler fig pattern with a custom shadow-routing proxy that intercepted all mainframe-bound traffic. Every transaction was dual-written: once to the mainframe (authoritative) and once to the new microservices layer (shadow). Comparison jobs ran continuously, flagging divergences for reconciliation. Traffic was shifted in 5% increments by product line over 14 months, with the mainframe remaining hot until the final product line reached 100% confidence.

The Problem

• Factory-floor hardware was ruggedised ARM-based units with no GPU — standard inference frameworks were too slow
• Intermittent connectivity meant cloud fallback was not a viable safety net
• Model accuracy could not degrade below 99.2% — below that, false negatives reached an unacceptable rate
• OTA update cycles on factory hardware are months-long; the deployment model had to be right first time

Our Approach

We applied INT8 post-training quantisation to a MobileNetV3 backbone, reducing model size by 74% with less than 0.3% accuracy loss. Inference ran via ONNX Runtime on the ARM units, hitting consistent 6ms latency. A local model registry handled versioning and rollback without cloud dependency. Drift detection ran on-device and triggered alerts when incoming image distributions deviated beyond a calibrated threshold.

The Problem

• GC pauses on the JVM consumer fleet were introducing 200–400ms latency spikes every 30–90 seconds under sustained load
• Kafka's default partitioning strategy created hot partitions under skewed telemetry device distributions
• Deserialization of a mix of Avro, Protobuf, and legacy JSON schemas added significant per-message overhead
• The pipeline had no backpressure mechanism — upstream producers would overwhelm consumers during traffic bursts

Our Approach

The consumer layer was rewritten in Rust using Tokio for async I/O and a custom Kafka client that implemented consistent-hash partitioning based on device ID cardinality. A unified schema registry with zero-copy deserialization eliminated per-message allocation. Backpressure was implemented via a token-bucket rate limiter at the ingestion boundary. The result: 5PB/day on 40% fewer nodes with p99 latency under 12ms.

The Problem

• LLMs produced plausible but invalid ICD-11 codes at a rate of approximately 2.3% — catastrophic in a clinical context
• Referral letters mixed structured data, clinical shorthand, and free text with no consistent schema
• The system needed to escalate ambiguous cases to human review without disrupting routing throughput
• NHS data governance required all processing to remain on-premises with no data leaving the trust boundary

Our Approach

We built a two-stage architecture: a RAG pipeline grounded in the full ICD-11 taxonomy and the trust's clinical protocol documents, followed by a deterministic validation layer that rejected any output not resolvable to a verified code. Ambiguity scoring routed uncertain cases to a human review queue with pre-populated suggested codes. The entire stack ran on-premises using a fine-tuned Mistral 7B model, with no external API calls.

The Problem

• Production database snapshots were 8TB — a full restore per PR was impractical on any timeline
• Shared staging environments meant one team's bad deployment could block all other teams
• Service-to-service dependencies meant isolated environments needed intelligent traffic routing to avoid calling production
• Data anonymisation for GDPR compliance had to run within the 90-second spin-up budget

Our Approach

We built a custom Kubernetes operator that listened to GitHub PR webhooks and provisioned namespaced environments using copy-on-write volume snapshots — eliminating the full-restore problem. Service mesh routing rules isolated all inter-service traffic to the PR namespace by default, with explicit overrides for stable shared services. A streaming anonymisation job processed only the tables touched by changed migrations, completing in under 40 seconds for 95% of PRs.