Memory consumption issue

[Antranig Basman]

Thanks for the investigation, Giovanni. This is very useful data. We're 
currently in the middle of upgrading all the base libraries for the 
GPII, before which it isn't worth doing much profiling work. We expect 
the memory leakage issues to ease off a bit, and the resulting 
simplification of the code should make it easier to hunt down the 
remaining leak issues later in the year.

Cheers,

Antranig.

On 03/07/2015 02:44, Tirloni, Giovanni wrote:
> Hello,
>
>   While running load tests on the preferences server (for benchmark and
>   capacity planning purposes), the memory consumption of the node
>   processes started to climb very quickly. To work around it and keep the
>   tests going, we were restarting the processes when they reached
>   100-200MB (which coincided with CPU usage nearing 85-95%). At some
>   point the monitoring script was restarting processes every 10 seconds
>   and it did not seem worth it anymore. This was done using the latest
>   code from gpii/universal (2015-07-01), the dependencies specified in
>   the package.json file and node 0.10.36.
>
>   I deployed the same code on my desktop, primed the CouchDB database
>   with the testData and tried to reproduce the issue with a simple test
>   (curl http://localhost:8081/preferences/elaine). I ran a few tests,
>   varying the wait time between the requests to increase the pressure, as
>   shown below. You can see in the graph a few 30-second plateaus between
>   tests. I then left the preferences server sitting idle for 6 minutes,
>   and the memory usage stayed flat at around 700MB, there were no
>   connections active at that time. The attached file contains the CSV
>   data (in KB) and the generated chart.
>
> 19:55:30 - 19:56:30 - wait 500ms
> 19:57:00 - 19:58:00 - wait 250ms
> 19:58:30 - 19:59:30 - wait 100ms
> 20:00:00 - 20:01:00 - wait  50ms
> 20:01:30 - 20:02:30 - wait  25ms
> 20:03:00 - 20:05:00 - wait  10ms
> 20:05:00 - 20:11:00 - no activity
>
>   I tried my hand at tracing/profiling the code but unfortunately I'm
>   closer to a JS newbie than an expert and I couldn't discover anything
>   Earth shattering. I did notice the number of  array/string/closure
>   objects accounted for most of the memory used. I also collected heap
>   snapshots using the 'heapdump' library (three samples available here:
>   http://1drv.ms/1f6Rmfp)
>
> Regards,
> Giovanni
>
>
>
> _______________________________________________________
> fluid-work mailing list - fluid-work at lists.idrc.ocadu.ca
> To unsubscribe, change settings or access archives,
> see http://lists.idrc.ocad.ca/mailman/listinfo/fluid-work
>