Constant CPU usage, increasing memory usage, killing the browser

#1

Some minutes ago Firefox crashed and I saw Windows freeing up more than 6 GB of memory. After the restart I checked the task manager and the problem persisted: every few seconds there is a short peak with 25% CPU usage (which apparently accounts for full usage of one core of my quad core CPU) and the memory usage increases slowly. The memory usage is interrupted by a partial drop after a few minutes but ultimately it increases until all of the system memory is eaten up and the browser crashes again.

To narrow down the problem I started by diasbling all add-ons until the problem vanished. This allowed me to reenable all add-ons except Kee (version 2.2.9). I could narrow the problem down to this page:

https://www.computerbase.de/2018-03/antec-be-quiet-cougar-oberklassenetzteile-test/3/

Changing the logging in Kee to debug logging I get this in the console:

port already set to: page
common.js:215:17
In browser content page script, received message from background script
common.js:197:17
Logging system config updated at Thu Mar 22 2018 18:10:26 GMT+0100
common.js:206:17
Messaging port was not established at page startup. Retrying now...
common.js:206:17
port already set to: page
common.js:215:17
Messaging port was not established at page startup. Retrying now...
common.js:206:17
port already set to: page
common.js:215:17
Messaging port was not established at page startup. Retrying now...
common.js:206:17
port already set to: page
common.js:215:17
In browser content page script, received message from background script
common.js:197:17
Logging system config updated at Thu Mar 22 2018 18:10:26 GMT+0100
common.js:206:17
In browser content page script, received message from background script
common.js:197:17
Logging system config updated at Thu Mar 22 2018 18:10:26 GMT+0100
common.js:206:17
Messaging port was not established at page startup. Retrying now...

This never stops. The messages keep on coming.

I wanted to get this error report out before investigating the situation any further. I will now check with a portable version of Firefox with only Kee installed and then try with older versions. There will be updates to this initial report.

Update: The problem could be reproduced with Portable Firefox 59.0.1 and just the Kee add-on 2.2.9 installed when loading the page that is linked above. The browser did not crash after 20 minutes, but the CPU spikes were there and the memory usage kept increasing.

The problematic page:

9
Problematic-Page-Firefox-59.0.1-Kee-2.2.9.png891x1111 424 KB

Second update: The problem also occurs with Kee 2.1 and 2.0.

#2

Situation shortly after starting the browser (the flat line is with the browser loaded and the increase from there is when the page was loaded:

Windows Task-Manager-1
Windows Task-Manager-1.png881x1079 60.5 KB

#3

Some time later, the mem usage kept hitting the ceiling and here Firefox apparently just did some major Garbage Collection to mitigate the problem:

Windows Task-Manager-2
Windows Task-Manager-2.png873x1064 61.9 KB

#4

I’ve been looking into this today using that web page and can’t find anything much to add at the moment unfortunately. All the Firefox diagnostics I’ve looked at so far claim that Kee is not using any unusual amount of memory so all that usage may be somewhere else within Firefox. I’ll try some more long-running tests in the coming weeks but might not have time for more than one or two tests since it seems to take half a day or more for the problem to manifest and I’ve not really got any spare time for a couple of weeks so if I don’t find something noteworthy soon it might be a while before I reply again.

The drop in your 3rd post could be a process crashing rather than a GC. Firefox does not yet recover from a crashed add-on process but if the memory is being used by a content (page) process then you would potentially see a “tab crashed” message in one or more tabs (and since Firefox indicates to me that it is not the add-on process that’s using up memory anyway, that might all align).

I’m not even sure if the slight increase in memory is the same issue you’re seeing - I’m on Windows 10 so there could easily be different causes on older versions of Windows and I’m yet to see it get to the point of such high memory pressure that I can rule out very lazy GC algorithms within Firefox. I’ve only tested on FF61 so there’s a small chance they’ve changed something since FF59.

I’m also not sure if all of the same multi-process stability and performance enhancements we see on Windows 10 would be applied on old versions of Windows too.
Which version of Windows are you using?

#5

That could very well be true. The whole problem caught me by surprise. I had never had any problems with Keefox or Kee before and when Firefox crashed because of too much memeroy usage I had to search for quite some time to isolate the cause. In the second test with Portable Firefox 59.0.1 and just Kee installed it could be reproduced with that very sae web ae (but, for example, not by the home page of the same domain) but not to the point of the very same crash. I think I would need to wait some hours until it happens again just as it came out of nowhere after hours of use. Frankly, the real bug is not the crash but the fact that this situation eats up more and more memory in the first place.

The log messages in a loop also seem to indicate that something is wrong with Kee. There should be a limit for “Retrying now…” after a page has been loaded. Doing this ad infinitum certainly doesn’t help in reducing the memory usage. Perhaps enforcing some limit for those retries will do the trick.

That could be true. I will do the test once more and see if one of the processes is terminated.

I’m on Windows 7 Ultimate, fully updated. If the difference between the Windows versions is responsible for this, then this almost certainly is a very weird Firefox problem. I would much prefer if the problem could be reproduced with Windows 10, because the alternative is some highly esoteric edge case that could take ages to find and fix.

#6

Sorry for coming back so late. Now I have data using the Firefox Nightly 61.0a1 (2018-04-12) (32-bit) on Windows 7 x64 Ultimate. The browser was downloaded from PortableApps.com and then used with freshly, with no other add-ons beside Kee installed. I used the same link to ComputerBase for the sake of reproducibility.

The problem is very much the same, with the browser eating up all of the available memory. It didn’t come to a crash this time. The bigger drops are not crashes of individual processes of the browser, the PIDs in the task manager remained the same after the drop. After more than half an hour, there was a drop that resulted in a stop of the problematic behaviour, just as if Firefox killed and blocked the offending script or Kee had finlly reached some maximum run count.

#7

I have observed the same issue, it seems to have to do with the “_generated_background_page.html”. It seems, the addon generates heaps of temporary images and stores them as a base64 encoded string.

grafik
grafik.png1722x483 47.3 KB

#8

Thanks both for the information.

Although I now can’t find it, I am certain that I read about a Firefox bug that was fixed in the past month or so which may be relevant to this. Essentially, until one of the latest versions of Firefox Nightly, the browser action icon is recreated at an extremely high rate (I can’t recall if it was per mouse movement over the toolbar or 60 times per second).

If the icons that are being generated and stored in memory hundreds of thousands of times are the same icons that are associated with the browser action icon (which I suspect they are because of their location in the background page DOM) we should expect a very significant improvement in the latest Nightly.

There is probably something else that we can do in Kee to ensure that we don’t end up with a new version of this string each time anyway but I won’t have time to investigate that for a while so with a bit of luck, that will become an almost imperceptible memory utilisation improvement that is dwarfed by the use of the latest Firefox version.

It would be great if you could report back in the coming weeks as to whether the problem is still reproducible in Firefox 62.

If it’s still a problem, seeing the full string of those icons could help me narrow down the cause in future (I’m expecting the them to be the main KeeFox icon and either a greyed out version of it or one with a yellow background but it would be interesting if they are instead variations with number icons.

#9

Hi there,

just to give an update: the issue ist sill present with Firefox 60.0.2, but it seems a bit more “tame”. I will definitely give an update after Firefox 62 is released.

#10

Hi, I confirm the above described behavior with the following page

www.hasznaltauto.hu (this is a Hungarian used car market), just search for a car using the drop down boxes and open a few pages. If needed I can provide a few random links.

As I experienced if you have only a few pages open from this site, the memory usage increases quite slowly, but if you have e.g. 8 open pages, it eats up 2-4 GB in 30-60 minutes. Sometimes longer, sometimes shorter. It starts slowly and after the usage grows faster. Disabling kee immediately frees up memory without closing firefox. Re-enabling kee increases memory usage again.

I have tested with firefox 60.0.2 under windows 10 and opensuse tumbleweed using latest kee and keepass