Constant CPU usage, increasing memory usage, killing the browser

[oliver-graetz]

Some minutes ago Firefox crashed and I saw Windows freeing up more than 6 GB of memory. After the restart I checked the task manager and the problem persisted: every few seconds there is a short peak with 25% CPU usage (which apparently accounts for full usage of one core of my quad core CPU) and the memory usage increases slowly. The memory usage is interrupted by a partial drop after a few minutes but ultimately it increases until all of the system memory is eaten up and the browser crashes again.

To narrow down the problem I started by diasbling all add-ons until the problem vanished. This allowed me to reenable all add-ons except Kee (version 2.2.9). I could narrow the problem down to this page:

https://www.computerbase.de/2018-03/antec-be-quiet-cougar-oberklassenetzteile-test/3/

Changing the logging in Kee to debug logging I get this in the console:

port already set to: page
common.js:215:17
In browser content page script, received message from background script
common.js:197:17
Logging system config updated at Thu Mar 22 2018 18:10:26 GMT+0100
common.js:206:17
Messaging port was not established at page startup. Retrying now...
common.js:206:17
port already set to: page
common.js:215:17
Messaging port was not established at page startup. Retrying now...
common.js:206:17
port already set to: page
common.js:215:17
Messaging port was not established at page startup. Retrying now...
common.js:206:17
port already set to: page
common.js:215:17
In browser content page script, received message from background script
common.js:197:17
Logging system config updated at Thu Mar 22 2018 18:10:26 GMT+0100
common.js:206:17
In browser content page script, received message from background script
common.js:197:17
Logging system config updated at Thu Mar 22 2018 18:10:26 GMT+0100
common.js:206:17
Messaging port was not established at page startup. Retrying now...

This never stops. The messages keep on coming.

I wanted to get this error report out before investigating the situation any further. I will now check with a portable version of Firefox with only Kee installed and then try with older versions. There will be updates to this initial report.

Update: The problem could be reproduced with Portable Firefox 59.0.1 and just the Kee add-on 2.2.9 installed when loading the page that is linked above. The browser did not crash after 20 minutes, but the CPU spikes were there and the memory usage kept increasing.

The problematic page:

Second update: The problem also occurs with Kee 2.1 and 2.0.

[oliver-graetz]

Situation shortly after starting the browser (the flat line is with the browser loaded and the increase from there is when the page was loaded:

[oliver-graetz]

Some time later, the mem usage kept hitting the ceiling and here Firefox apparently just did some major Garbage Collection to mitigate the problem:

[luckyrat]

I’ve been looking into this today using that web page and can’t find anything much to add at the moment unfortunately. All the Firefox diagnostics I’ve looked at so far claim that Kee is not using any unusual amount of memory so all that usage may be somewhere else within Firefox. I’ll try some more long-running tests in the coming weeks but might not have time for more than one or two tests since it seems to take half a day or more for the problem to manifest and I’ve not really got any spare time for a couple of weeks so if I don’t find something noteworthy soon it might be a while before I reply again.

The drop in your 3rd post could be a process crashing rather than a GC. Firefox does not yet recover from a crashed add-on process but if the memory is being used by a content (page) process then you would potentially see a “tab crashed” message in one or more tabs (and since Firefox indicates to me that it is not the add-on process that’s using up memory anyway, that might all align).

I’m not even sure if the slight increase in memory is the same issue you’re seeing - I’m on Windows 10 so there could easily be different causes on older versions of Windows and I’m yet to see it get to the point of such high memory pressure that I can rule out very lazy GC algorithms within Firefox. I’ve only tested on FF61 so there’s a small chance they’ve changed something since FF59.

I’m also not sure if all of the same multi-process stability and performance enhancements we see on Windows 10 would be applied on old versions of Windows too.
Which version of Windows are you using?

[oliver-graetz]

I’ve been looking into this today using that web page and can’t find anything much to add at the moment unfortunately. All the Firefox diagnostics I’ve looked at so far claim that Kee is not using any unusual amount of memory so all that usage may be somewhere else within Firefox.

That could very well be true. The whole problem caught me by surprise. I had never had any problems with Keefox or Kee before and when Firefox crashed because of too much memeroy usage I had to search for quite some time to isolate the cause. In the second test with Portable Firefox 59.0.1 and just Kee installed it could be reproduced with that very sae web ae (but, for example, not by the home page of the same domain) but not to the point of the very same crash. I think I would need to wait some hours until it happens again just as it came out of nowhere after hours of use. Frankly, the real bug is not the crash but the fact that this situation eats up more and more memory in the first place.

The log messages in a loop also seem to indicate that something is wrong with Kee. There should be a limit for “Retrying now…” after a page has been loaded. Doing this ad infinitum certainly doesn’t help in reducing the memory usage. Perhaps enforcing some limit for those retries will do the trick.

The drop in your 3rd post could be a process crashing rather than a GC. Firefox does not yet recover from a crashed add-on process but if the memory is being used by a content (page) process then you would potentially see a “tab crashed” message in one or more tabs (and since Firefox indicates to me that it is not the add-on process that’s using up memory anyway, that might all align).

That could be true. I will do the test once more and see if one of the processes is terminated.

I’m not even sure if the slight increase in memory is the same issue you’re seeing - I’m on Windows 10 so there could easily be different causes on older versions of Windows and I’m yet to see it get to the point of such high memory pressure that I can rule out very lazy GC algorithms within Firefox. I’ve only tested on FF61 so there’s a small chance they’ve changed something since FF59.

I’m also not sure if all of the same multi-process stability and performance enhancements we see on Windows 10 would be applied on old versions of Windows too.

Which version of Windows are you using?

I’m on Windows 7 Ultimate, fully updated. If the difference between the Windows versions is responsible for this, then this almost certainly is a very weird Firefox problem. I would much prefer if the problem could be reproduced with Windows 10, because the alternative is some highly esoteric edge case that could take ages to find and fix.

[oliver-graetz]

Sorry for coming back so late. Now I have data using the Firefox Nightly 61.0a1 (2018-04-12) (32-bit) on Windows 7 x64 Ultimate. The browser was downloaded from PortableApps.com and then used with freshly, with no other add-ons beside Kee installed. I used the same link to ComputerBase for the sake of reproducibility.

The problem is very much the same, with the browser eating up all of the available memory. It didn’t come to a crash this time. The bigger drops are not crashes of individual processes of the browser, the PIDs in the task manager remained the same after the drop. After more than half an hour, there was a drop that resulted in a stop of the problematic behaviour, just as if Firefox killed and blocked the offending script or Kee had finlly reached some maximum run count.