Mastodon Memory Leak

[maarzt]

Problem description

I noticed that mastodon objects like WindowManger, ProjectManager, AppModel etc. don't get garbage collected after the project is closed.

This can be a problem for users. Fiji might run out of memory when opening a large number of Mastodon projects, one after the other. The Eclipse Memory Analyzer is a handy tool to inspect such problems. The org.mastodon.spatial.SpatioTemporalIndexImpRebuilderThread seems to stay in memory and keeps a reference to the ModelGraph.

TODOs

• Find why the SpatioTemporalIndexImpRebuilderThread does not stop after Mastodon is closed. Fix that problem.
• Check if garbage collection properly cleans up the Mastodon WindowManager and ModelGraph.

Expected Behavior

All Mastodon related objects (WindowManager, ProjectManager, AppModel, ModelGraph, etc. ) get garbage collected after a Mastodon project is closed.

[tinevez]

Argh the horror.

[maarzt]

The following few lines of code are enough to trigger an OutOfMemoryException:

package org.mastodon.mamut.tomancak;

import org.junit.Test;
import org.mastodon.mamut.model.Model;

public class MastodonMemoryTest
{
	@Test
	public void test() {
		for ( int i = 0; i < 100; i++ )
		{
			new Model();
		}
	}
}

The problem seems to be related to the UndoRedoStack. See stack trace:

java.lang.OutOfMemoryError: Java heap space

	at org.mastodon.undo.ByteArrayUndoRedoStack.<init>(ByteArrayUndoRedoStack.java:122)
	at org.mastodon.undo.GraphUndoRedoStack.<init>(GraphUndoRedoStack.java:90)
	at org.mastodon.undo.GraphUndoRecorder.<init>(GraphUndoRecorder.java:98)
	at org.mastodon.mamut.model.Model.<init>(Model.java:178)
	at org.mastodon.mamut.model.Model.<init>(Model.java:122)
	at org.mastodon.mamut.tomancak.MastodonMemoryTest.test(MastodonMemoryTest.java:12)

[stefanhahmann]

@maarzt Could you ask for review/approval/merge of the related PR bigdataviewer/bigdataviewer-core#163 so that the temporarily introduced PainterThread of this PR does not live longer than necessary?

[tpietzsch]

I'll review it today

[tpietzsch]

merged and released with bigdataviewer-core-10.4.7

[tinevez]

Could we depend on bigdataviewer-core-10.4.7 in mastodon and use the code from there? Or do we have specifics in Mastodon?

[maarzt]

This issue if fixed with the following PRs being merged and mastodon-graph-1.0.0-beta-24 and bigdataviewer-core-10.4.7 being released:

• Fix garbage collection for the SpatioTemporalIndexImpRebuilderThread mastodon-graph#14
• Fix Mastodon Memory Leak #244
• Make sure the PainterThread does not cause a memory leak bigdataviewer/bigdataviewer-core#163

[tinevez]

Hello @maarzt
Really thank you the tests you build sweat talent and skills.

Unfortunately I tested the org.mastodon.mamut.GarbageCollectionTest and it still fails, even with

• mastodon-graph 1.0.0-beta-24
• bigdataviewer-core 10.4.7
  This horrible issue is not over :(