https://sejoonkim.me/articles DevOps, Kubernetes, AWS, and infrastructure automation. en https://sejoonkim.me/articles/nginx-to-traefik-gateway-api-migration https://sejoonkim.me/articles/nginx-to-traefik-gateway-api-migration Wed, 20 May 2026 00:00:00 GMT CVE-2025-1974 started it. Four months later, three clusters ran Traefik v3 on Gateway API with zero downtime. The honest version: what broke in stage, the certificate deadlock, and the nine days we deliberately did nothing. https://sejoonkim.me/articles/crowdsec-waf-kubernetes https://sejoonkim.me/articles/crowdsec-waf-kubernetes Wed, 14 Jan 2026 00:00:00 GMT Needed a WAF for public APIs. Chose CrowdSec (open-source). Hit three integration issues: PostgreSQL namespace, client IP preservation, log collection. Documenting the fixes. https://sejoonkim.me/articles/docker-image-optimization https://sejoonkim.me/articles/docker-image-optimization Wed, 24 Dec 2025 00:00:00 GMT Our Docker images were 800MB+. Build times were slow, pulling images took forever. Spent a day optimizing - got images down to 200MB using multi-stage builds and Alpine base images. https://sejoonkim.me/articles/adding-kubernetes-nodes https://sejoonkim.me/articles/adding-kubernetes-nodes Wed, 17 Dec 2025 00:00:00 GMT Traffic increased 40% over 3 months. Nodes were running at 75% CPU. Ordered 2 new Hetzner servers and added them to the cluster. Took about 4 hours from ordering to nodes serving traffic. https://sejoonkim.me/articles/coredns-timeout-investigation https://sejoonkim.me/articles/coredns-timeout-investigation Wed, 10 Dec 2025 00:00:00 GMT Applications occasionally failed DNS lookups with 5-second timeouts. Checked CoreDNS logs, CPU usage, network - everything looked fine. Turned out to be conntrack table exhaustion on worker nodes. https://sejoonkim.me/articles/ssh-key-rotation-automation https://sejoonkim.me/articles/ssh-key-rotation-automation Wed, 03 Dec 2025 00:00:00 GMT Security audit said our SSH keys hadn't been rotated in 18 months. Wrote a script to rotate keys across all Hetzner servers and update them in Azure Key Vault. Took 3 hours to build, runs in 5 minutes. https://sejoonkim.me/articles/pod-security-standards-enforcement https://sejoonkim.me/articles/pod-security-standards-enforcement Wed, 26 Nov 2025 00:00:00 GMT Enabled Pod Security Standards in Kubernetes. Immediately broke 6 out of 12 applications because they were running as root or using privileged containers. Spent 2 days fixing them all. https://sejoonkim.me/articles/terraform-state-azure-backend https://sejoonkim.me/articles/terraform-state-azure-backend Wed, 19 Nov 2025 00:00:00 GMT We'd been storing Terraform state in Git (bad idea). Moved it to Azure Blob Storage with state locking. Migration took 30 minutes. Should have done this from the start. https://sejoonkim.me/articles/nginx-ingress-502-debugging https://sejoonkim.me/articles/nginx-ingress-502-debugging Wed, 12 Nov 2025 00:00:00 GMT Users reported occasional 502 errors. Logs showed NGINX couldn't reach backend pods. Took a day to find the issue - pod readiness probes were too aggressive and marking healthy pods as not ready. https://sejoonkim.me/articles/trivy-container-scanning-ci https://sejoonkim.me/articles/trivy-container-scanning-ci Wed, 05 Nov 2025 00:00:00 GMT Integrated Trivy into GitLab CI to scan container images for vulnerabilities before deployment. Found 47 high-severity issues we didn't know about. Some were fixable, some weren't. https://sejoonkim.me/articles/postgres-backup-to-azure-blob https://sejoonkim.me/articles/postgres-backup-to-azure-blob Wed, 29 Oct 2025 00:00:00 GMT Set up daily PostgreSQL backups from our Kubernetes cluster to Azure Blob Storage. Using pg_dump in a CronJob with lifecycle policies for retention. Cost is about €8/month for 30 days of backups. https://sejoonkim.me/articles/azure-key-vault-sync-delay https://sejoonkim.me/articles/azure-key-vault-sync-delay Wed, 22 Oct 2025 00:00:00 GMT Secrets in Azure Key Vault were updated but pods kept using old values. Took 2 hours to figure out the sync interval setting and force a refresh. Notes on how external-secrets actually works. https://sejoonkim.me/articles/lets-encrypt-rate-limit-hit https://sejoonkim.me/articles/lets-encrypt-rate-limit-hit Wed, 15 Oct 2025 00:00:00 GMT Made a mistake while testing cert-manager configuration. Issued 20 certificates for the same domain in an hour. Got rate limited for a week. Notes on staging environment and rate limits. https://sejoonkim.me/articles/hetzner-server-rightsizing https://sejoonkim.me/articles/hetzner-server-rightsizing Wed, 08 Oct 2025 00:00:00 GMT Looked at actual CPU and memory usage across our Kubernetes nodes. Found we were paying for servers we barely used. Saved €120/month by switching to smaller machines. https://sejoonkim.me/articles/self-managed-k8s-upgrade https://sejoonkim.me/articles/self-managed-k8s-upgrade Wed, 01 Oct 2025 00:00:00 GMT Moving from Kubernetes 1.28 to 1.29 on bare metal Hetzner servers. No managed control plane to click 'upgrade' - we had to do it manually. Notes on what actually happened. https://sejoonkim.me/articles/hetzner-network-incident https://sejoonkim.me/articles/hetzner-network-incident Wed, 24 Sep 2025 00:00:00 GMT Hetzner's network had problems in their Falkenstein datacenter. Our services stayed up because we split workloads across regions and keep critical data in Azure. https://sejoonkim.me/articles/kubernetes-statefulset-deep-dive https://sejoonkim.me/articles/kubernetes-statefulset-deep-dive Wed, 17 Sep 2025 00:00:00 GMT Understanding StatefulSet internals, ordered pod management, persistent storage, and real-world use cases for stateful applications in Kubernetes https://sejoonkim.me/articles/logging-migration-loki-alloy https://sejoonkim.me/articles/logging-migration-loki-alloy Tue, 09 Sep 2025 00:00:00 GMT Setting up Loki and Alloy for log aggregation in our Kubernetes cluster. Learning what all those Loki components actually do. https://sejoonkim.me/articles/prometheus-grafana-setup-kubernetes https://sejoonkim.me/articles/prometheus-grafana-setup-kubernetes Sun, 07 Sep 2025 00:00:00 GMT Moving our monitoring from Grafana Cloud to self-hosted Prometheus and Grafana on Kubernetes. Turns out most apps already had metrics support, just needed to enable it. https://sejoonkim.me/articles/creating-monitoring-user-zalando-postgres https://sejoonkim.me/articles/creating-monitoring-user-zalando-postgres Fri, 05 Sep 2025 00:00:00 GMT How I solved the challenge of creating a monitoring-only user with minimal permissions in a GitOps-managed Postgres cluster https://sejoonkim.me/articles/zero-downtime-nginx-upgrade-gitops https://sejoonkim.me/articles/zero-downtime-nginx-upgrade-gitops Wed, 03 Sep 2025 00:00:00 GMT How to upgrade a Helm-managed application in production with zero downtime using GitOps and Kubernetes RollingUpdate strategy https://sejoonkim.me/articles/the-real-cost-of-downtime https://sejoonkim.me/articles/the-real-cost-of-downtime Fri, 29 Aug 2025 00:00:00 GMT My journey into understanding why every millisecond matters in DevOps, and what the research taught me about building reliable systems https://sejoonkim.me/articles/kubernetes-external-secrets https://sejoonkim.me/articles/kubernetes-external-secrets Sun, 24 Aug 2025 00:00:00 GMT A comprehensive guide to implementing External Secrets Operator for secure secret management in Kubernetes clusters