Slow serialization with large Array data (> 60 Mbytes)

[hernanmd]

It seems that Fuel takes a long time serializing an Array of Arrays containing multiple Strings each one. When serialization passes around 60 Mbytes, it starts writing data in chunks of 100Kbytes and very slowly:

To reproduce please download this Fuel serialization: https://filesender.renater.fr/?s=download&token=b3809275-6258-4ee4-8d93-3b8871b7a0bd

Materialize it in a fresh Pharo 11 image:

FLMaterializer 
	materializeFromFileNamed: 'acImdb_49582_nodups_noartfcts_tokenized.fuel'.

and then from the Inspector window evaluate:

FLSerializer 
	serialize: self 
	toFileNamed: 'acImdb_49582_nodups_noartfcts_tokenized_test.fuel'.

We tested in Pharo 11 under macOS.

[theseion]

Interesting. I don't see any reason for Fuel to do that. There must be another mechanism at play. I currently don't have to look into this tough, in a week or two maybe. @tinchodias, do you have some time to spare maybe?

[theseion]

Can you give me a rough timing estimate of what you mean by "slowly"?

[hernanmd]

Can you give me a rough timing estimate of what you mean by "slowly"?

It was around 100K/sec

[theseion]

This is very interesting. I found the issue to be time spent in FLLargeIdentityDictionary. Replacing one particular use of FLLargeIdentityDictionary with IdentityDictionary cut time down to ~40 seconds. FLLargeIdentityDictionary was specifically designed to be faster for large collections, so it's very curious that this happens.

[theseion]

Still investigating but interesting find: in Pharo 11 the string hashes appear to be badly distributed.

[hernanmd]

Probably @guillep will be interested

[theseion]

I'm not sure the hash distribution is bad per se, it's just that the hashes have only uneven numbers when using modulo 4096.

It looks like FLLargeIdentityDictionary has been slower than we wanted it to be since at least Pharo 6.

[guillep]

Interesting finding :)

[guillep]

I checked the usage of identity dictionaries.
I wrote a silly (and unfinished) identity dictionary implementation that keeps a hash table with hash tables inside. The outer hash table uses the modulo of the identity hash and benefits from them being evenly distributed. This silly implementation outperformed the other identity dictionaries for the larger charges (yellow in the plot below). It's 4x faster than FLLargeIdentityDictionary and 50x faster than the standard identity dictionary for the largest set of objects.

This kind of tells me that a better identity dictionary will drastically improve the performance of fuel.
I'll add this to the GSOC proposal here: https://gsoc.pharo.org/serialization

[theseion]

Brilliant, thanks!

[jecisc]

Oh nice, not just Fuel. The system relies a lot on IdentityDictionaries :)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will remain open but will probably not come into focus. If you still think this should receive some attention, leave a comment. Thank you for your contributions.