live
The leak that isn't on the heap
Process memory grows 50MB/day forever. Heap dumps are flat and boring. Only a restart helps.
The scene
Reported: ingestd RSS goes 300MB → 1.4GB over 4 weeks, then the OOM killer takes it. Heap usage (jstat, heap dumps) holds flat at ~90MB, with no GC pressure.
- This month's release added a zero-copy parser that does
ByteBuffer.allocateDirect(64 * 1024)per message and lets it drop after parsing (~3k msg/s). -XX:MaxDirectMemorySizeunset.jcmd <pid> VM.native_memory summary(NMT enabled on canary):
- Internal (reserved=1210MB, committed=1190MB)
(malloc=1188MB #4,019,551)- Container base image uses glibc malloc with default
MALLOC_ARENA_MAX.
Diagnoses — hidden until the incident closes
Sign in to post your root-cause diagnosis.