<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">On 04/12/2025 21:21, Cormac Redmond
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAG_-AaD6XJh0Esih2OTjeP+MS7+UXMFBMUeF_vWViPcwj=A40w@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div class="gmail_default"
style="font-family:verdana,sans-serif">Hi John,</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">Thanks for the prompt
response. Yes, I looked into the code afterwards, and I can
see that it is just fairly oblivious to the waste. I was
looking to override with nulls myself, but there were too many
privates.</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">The only solution
(without re-writing my own impl) is to re-create the
SortedList every time the dataset is smaller, or to create
some sort of DTO that does not link to the original real
objects in any way (or maybe use WeakReferences). In my case,
I could have 10,000 rows where each item's underlying memory
is around 500KB on the heap. That's instantly 5GB I cannot
re-claim, or example.</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">That code with the
"bug" <i>mostly</i> hasn't changed in 10+ years. Can it be
fixed? It's such a fundamental class, adding workarounds
(everywhere, forever) would seem a shame.</div>
</div>
</blockquote>
<p>It definitely can be fixed. Nothing should be relying on the
current behavior, as it is not specified to hold on to stale
references.</p>
<p>You could try submit a PR with a fix, or wait until this becomes
a priority. I myself may also do a fix when I have time, as it is
pretty low hanging fruit and self contained.</p>
<p>You can "test" a fixed implementation by just making a copy of
the class. If you're not in a modular environment, you can even
replace the class (in the same package it is in) with a fixed
version until FX fixes it. This is how I usually test and fix
(deep) internal problems without having to build FX locally at
all.</p>
<p>--John<br>
</p>
<p><br>
</p>
<blockquote type="cite"
cite="mid:CAG_-AaD6XJh0Esih2OTjeP+MS7+UXMFBMUeF_vWViPcwj=A40w@mail.gmail.com">
<div dir="ltr">
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">Kind Regards,</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">Cormac</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, 4 Dec 2025 at 20:01,
John Hendrikx <<a href="mailto:john.hendrikx@gmail.com"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">john.hendrikx@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>This looks like a classic problem where unused elements
in an array are not nulled. Looking at the code that
updates the `size` field, there are a few code paths that
are not setting unused elements to null. I also saw no
code that ever shrinks the arrays. So it looks this
implementation is only half finished.<br>
</p>
<p>So, yes, I'd say this is a bug.<br>
</p>
<p> </p>
<blockquote type="cite">I'm going to guess there's a very
good reason for this behaviour.</blockquote>
Time pressure and insufficient testing.<br>
<p>--John<br>
</p>
<div>On 04/12/2025 19:40, Cormac Redmond wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_default"
style="font-family:verdana,sans-serif">Hi,<br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">I've traced a
memory issue back to a SortedList (surprisingly),
where it's hanging on to objects that could/should
have been GC'd.</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">SortedList's
internal arrays will only grow (but not shrink) in
line with the source ObservableList (I'm sure there
are reasons for this). So even when your source
ObservableList shrinks, SortedList is hanging on to
references to objects the source list once contained,
even though they're completely removed from the source
list.</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">This leads to a
substantial waste of memory, especially when just one
momentarily large dataset leads to a permanent spike
in unnecessary memory usage for the remainder for the
application's lifetime. I'm sure I'm not the first to
raise this or ask about it & I'm going to guess
there's a very good reason for this behaviour. But
could someone explain this? I would have thought
SortedList should/could remain as lean as the source
list. FilteredList is somewhat similar, except it
stores an array of ints (indexes), so less of a memory
hit, but still pointless nonetheless.</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">Example GIF +
code below: setup a typical sortable table (so,
ObservableList + FilteredList + SortableList), where
the print button shows that when you add a lot of
data, and remove it, SortedList retains references to
all of the old objects. Some reflection is used to
print that information.</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><img
src="cid:part1.uw57XnT0.bzaUGkC4@gmail.com"
alt="sorted_list_mem.gif" class="" width="406"
height="562"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">Code to
reproduce:</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"><font face="monospace">public
class SortedListMemWasteDemo extends Application {<br>
<br>
record PotentialLargeData(String info) {}<br>
<br>
public static void main(String[] args) {<br>
launch(args);<br>
}<br>
<br>
@Override<br>
public void start(Stage primaryStage) {<br>
ObservableList<PotentialLargeData>
masterData = FXCollections.observableArrayList(<br>
new PotentialLargeData("Initial info
1"), <br>
new PotentialLargeData("Initial info
2"));<br>
<br>
TableColumn<PotentialLargeData,
String> col = new TableColumn<>("Info");<br>
col.setCellValueFactory(cellData -> new
SimpleStringProperty(cellData.getValue().info()));<br>
<br>
TableView<PotentialLargeData> table =
new TableView<>();<br>
table.getColumns().add(col);<br>
<br>
FilteredList<PotentialLargeData>
filteredData = new FilteredList<>(masterData,
item -> true);<br>
SortedList<PotentialLargeData>
sortedData = new SortedList<>(filteredData);<br>
sortedData.comparatorProperty().bind(table.comparatorProperty());<br>
table.setItems(sortedData);<br>
<br>
Button loadManyBtn = new Button("Add 10,000
items");<br>
loadManyBtn.setOnAction(e -> {<br>
System.out.println("Adding 10000 items
to master data");<br>
masterData.clear();<br>
for (int i = 0; i < 10000; i++)
masterData.add(new PotentialLargeData("Info item " +
i));<br>
});<br>
<br>
Button reduceBtn = new Button("Reduce to
2");<br>
reduceBtn.setOnAction(e -> {<br>
System.out.println("Reducing master data
size to 2");<br>
masterData.setAll(new
PotentialLargeData("New info 1"), new
PotentialLargeData("New info 2"));<br>
});<br>
<br>
Button printBtn = new Button("Print interal
sizes");<br>
printBtn.setOnAction(e -> {<br>
try {<br>
System.out.println("Items the user
sees: " + sortedData.size());<br>
int[] filtered = (int[])
getFieldValue(filteredData, "filtered");<br>
System.out.println("FilteredList.filtered length: "
+ filtered.length);<br>
// This is a hidden and significant
waste of memory<br>
Object[] sorted = (Object[])
getFieldValue(sortedData, "sorted");<br>
System.out.println("SortedList.sorted length: " +
sorted.length);<br>
} catch (Exception ex) {<br>
throw new RuntimeException(ex);<br>
}<br>
});<br>
<br>
primaryStage.setScene(new Scene(new
VBox(table, loadManyBtn, reduceBtn, printBtn), 600,
400));<br>
primaryStage.show();<br>
}<br>
<br>
private static Object getFieldValue(Object
object, String fieldName) throws Exception {<br>
Field field =
object.getClass().getDeclaredField(fieldName);<br>
field.setAccessible(true);<br>
return field.get(object);<br>
}<br>
}</font></div>
<div class="gmail_default"><font face="monospace"><br>
</font></div>
<div class="gmail_default"><font face="monospace"><br>
</font></div>
<div class="gmail_default"><font
face="verdana, sans-serif">Kind Regards,</font></div>
<div class="gmail_default"><font
face="verdana, sans-serif">Cormac</font></div>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</body>
</html>