OOMKilled at 18% memory

Reported: payments-worker restarts every ~40 min. kubectl describe says Last State: Terminated, Reason: OOMKilled, Exit Code: 137.

• container_memory_working_set_bytes: flat ~180Mi; the limit is 1Gi.
• Node free memory: 11Gi.

App log, last lines before each death:

pool: scaling workers 12 -> 96 (queue depth 8,412)
spawn: worker-91 ok
spawn: worker-92 ok
spawn: worker-93: resource temporarily unavailable

From inside the container, right before a kill:

$ cat /sys/fs/cgroup/pids.max
128
$ cat /sys/fs/cgroup/pids.current
128

• Each worker is a process running ~2 threads.
• The memory graph's scrape interval is 30s.