Script of the Day: Memory monitor
Identify and address out-of-memory troubles when analyzing large images.
Problem
QuPath works with very large images. And sometimes very large data files too.
When everything gets too big for the amount of memory available on the computer, this can spell trouble. It may crash, or else the analysis just doesn’t complete.
There are some steps that can be taken if memory is a problem - but it helps to first establish that is really the case.
There are limits to what these steps can achieve, of course. Performing cell detection across a large tissue section on a computer with 4GB RAM (or less) using QuPath might not end well, even with these steps. Even with 8GB RAM it could be a struggle.
In such cases, it may be necessary to reduce the area being analyzed, or the complexity of the analysis (e.g. using Fast cell counts rather than the full Cell detection command).
Solution
A graphical view of memory usage over time can help track down trouble.
The script to create this is a bit involved, so there is no line-by-line description this time (although feel free to ask if there are any questions). Here’s a picture instead showing the result:
Basically, the ‘Total memory’ is all that QuPath is allowed to access (remember the dialog that appeared when QuPath was started? If not, see Help → Show setup options). And ‘Used memory’ is an estimate of how much memory is required for whatever QuPath is currently doing.
There are four main things that require a substantial amount of memory in QuPath:
- The user interface
- The object hierarchy (e.g. detected cells, annotations, and their measurements)
- A cache of image tiles, used to improve the viewer performance while viewing images
- Parts of the image that are currently being processed
There is relatively little can be done about the user interface, but fortunately it does not have a very large impact.
The size of the object hierarchy is also usually not a big problem; QuPath tries to keep it compact, so that even if you have hundreds of thousands of cells with many tens of measurements each it should still only be a few hundred MB at most. You can help a bit by restricting the size of areas being analyzed, and ensuring measurements aren’t made unless they are really needed (e.g. intensity features).
The cache of image tiles has a fixed limit, calculated when QuPath is started. So it can’t grow indefinitely. It helps the viewer performance by storing parts of the image that have recently been viewed, but QuPath is always free to discard image tiles and request them again from the image file when needed. That is what happens if you click Clear tile cache in the Memory Monitor: QuPath empties the cache and claims back whatever memory it can, but the cache then will start to build up as soon as you begin browsing the image again. Therefore it is only a very temporary fix.
Java applications tend to claim a bit more memory than they really have to when they aren’t under pressure - thanks to keeping objects and data around longer than strictly necessary.
The Reclaim memory button requests getting back as much of this memory as possible immediately (but without touching the tile cache). Therefore it should also temporarily reduce the displayed ‘Used memory’.
But QuPath/Java will automatically call this whenever it is close to hitting the memory limit, so failing to press this button doesn’t result in more memory errors occurring. The only reason to press it yourself is to get a more accurate view of how much memory is really needed at any moment.
Image processing is typically where the biggest memory needs arise. Something like cell detection requires QuPath to extract the pixels for image regions, and often create multiple duplicate images (e.g. by separating stains, estimating background) along the way. This can lead to sudden memory spikes, which then settle down once the processing is complete. Most problems occur during these spikes.
Reducing the number of ‘Parallel threads’ will reduce the number of image regions that QuPath will try to handle simultaneously, and therefore reduce the amplitude of such memory spikes. This is one very effective way to help avoid out-of-memory errors - at the expense of slower processing.
The ‘Parallel threads’ option is already found in the QuPath preferences (the little cog icon on the toolbar, or Edit → Preferences…) but it is also included in the Memory monitor for convenience, making it easier to explore the impact this setting has on memory use.