Standing & sitting room only! Looks like people slept out for tickets to this talk! Encore...
Ali Sheikh is speaking on memory leaks and why they happen. He wrote new tools at Google to help with them. Very funny intro. Don't want any memory leaks? Don't write any code or use anybody else's! We have garbage collection; why do we have memory leaks at all? Read on...
In Node, we are not responsible for freeing memory per se.
Attaching data to "req" in Express does not leak memory because when you send the response, Express knows you are done and stops referring to "req". So "req" and anything attached to it can be released.
Memory leaks happen when objects expected to be short-lived get attached to long-lived ones.
When we create an event handler inside a requesthandler, it is attached to a long-lived object such as an event bus. This will leak memory unless it is removed after it fires (and if it doesn't always fire you're also in trouble):
function(req, res) {
client.on('message', function(channel, message) {
res.send(JSON.stringify(message));
});
});
The reference to "res" is now in a closure associated with "client" which is a long-lived object. So "res" can never die. Also: res.send is going to get invoked over and over on subsequent messages, which will produce errors of another kind. But a more subtle bug might not produce such an obvious error and would just leak the memory.
console.log can also leak memory under the right circumstances but hopefully you're not invoking it zillions of times in production...
A memory leak can occur even in Express code that seems clean, if you count the fact that hundreds of thousands of requests are not satisfied yet and grinding away simultaneously...
"Where is the leak coming from?"
"What tools are available?"
"Can I use them in production?"
Sometimes a leak can be in the garbage collector itself. We don't want to incur latency, let's wait on freeing this up... so in latest Node 8.x, you could hit a slow leak just with a trivial hello world Express app under the right circumstances. [A bit beyond your control, focus on what you can control...]
First set of tools:
process.memoryUsage()
Gives you an object with rss (resident set size) and heapTotal properties. heapTotal is total space available presently. external is memory consumed by buffers used by node. heapUsed is space occupied by JavaScript objects presently. [Resident Set Size is where the buck stops in my experience. -Tom]
There is also a memory-usage module that cleans this information up more.
heapTotal is interesting because it tells you how much memory the garbage collector thinks you'll need. It's good to watch. [Does it ever go down again in a single run? -Tom]
external is actually worth watching because it could mean you're leaking buffers, file descriptors, etc.
You can attach chrome dev tools to node.js and take a heap snapshot of the current application state. You can save them for later review.
You can see what constructor was used to create the objects, which is a hint to where they came from. "Object" will come up heavily if the objects were constructed as object literals (just with {} for instance).
You can also click through on the objects to see their properties. This doesn't tell you where they were allocated.
You can look at the "Retainers" tab to see who is holding on to the object. For instance you might see a chain of "next" properties, hinting that you've created a linked list of them.
Attach chrom e dev tools:
node --inspect foo.js
DevTools -> Memory -> Take heap snapshot
More information on taking heap snapshots.
You are unlikely to attach a live debugger to a production server. You might use the heapdump module to take them on demand and via the SIGUSR2 signal [see the "kill" unix command, less scary than it sounds when used properly].
Allocation Timeline takes a series of snapshots and allows you to see what was getting allocated in a given interval.
"The thing I really want to show you is the brand new Sampling Heap Profiler. Has been around for a year but few know about it."
Similar to the CPU profiler but shows which functions are allocating memory and how much. Based on sampling at intervals so overhead is low. For instance app.get could show as allocating a lot of memory.
"I think this is great and the reason I think it's great is that I implemented it." [Laughter]
Shows you a representative sample of JavaScript objects. Randomized poisson process (1 sample every 512k on average.)
Keeps track of live sampled objects along with the allocating stack trace.
[Is there necessarily a connection between where memory is most often allocated and where memory is most often leaked? -Tom]
Also see the heap-profile module.
Native leaks
These mostly matter for native modules and those who use them.
You can use Valgrind, a classic tool for examining C/C++ memory use.
There is also a sampling heap profiler (TCMalloc heap profiler).
Leaks happen. Try the tools today.
That's it! There's a demo of debugging production memory leaks at the Google booth.
