I’ve recently gotten completely fed-up with my backup process (using duplicity<\/a>) in the way it gobbles up huge amounts of memory (6GB+ out of 8GB physical memory) when doing its daily backup. This annoyance is amplified by code in the display driver that bails out when there is intense memory pressure and exits out of my Wayland session (likely bug<\/a>).\u00a0 I’m documenting my solution for Fedora 26, as I haven’t found trivially-understandable articles elsewhere.<\/p>\n Duplicity is a quite a nice program, and I am quite thankful for its development. While I am unhappy with the way its resource usage scales with 200GB backups and limited available knowledge on best tunings for various scales greater and smaller, I have not found any superior backup solution. I just need to do something about its ability to gobble memory undeterred by the Linux kernel.<\/p>\n Intel video hardware is probably the best low-power graphics with “reasonable” Linux support in its open-source drivers, but unfortunately, some debilitating bugs are being triggered with\u00a0 the recent Wayland migration. Most recently, its eagerness to bail out (and kill the user session) under memory pressure has been a concern.<\/p>\n Resource management tools have been available on Linux (and other POSIX-style OSes) for quite some time, so why can’t you just do a “nice -n “19 and be done? Some notes suggest that systemd may be of help<\/a>.<\/p>\n TL;DR: We need to create a systemd slice and put the backup process systemd unit into that slice.<\/p>\n Create a slice by creating the file in \/etc\/systemd\/system\/. Here’s what mine looks like:<\/p>\n I’ve set MemoryHigh to 2G, which is supposed to mean that systemd will start preferentially swapping pages out from the slice, should tasks inside that slice use more thatn 2GB physical memory. Other pages suggested setting\u00a0 MemoryMax, but as I still wanted my backup to complete, and a re-run of the backup process might even demand more, I did not want the MemoryMax behavior of “kill processes with OOM if the max is reached”. MemoryAccounting=true is a necessary prerequisite for any memory tracking controls in that slice.<\/p>\n With the slice file created, then I just needed to add:<\/p>\n to the corresponding duplicity.service file. While I was there, I also added (Nice=19, IOSchedulingClass=3, IOSchedulingPriority=7).<\/p>\n To apply the changes:<\/p>\n When the task is running, you can verify that it’s running in your new slice.<\/p>\n So far, the resource limiting has prevented the abrupt session-exits that used to consistently coincide with the backup trigger, but I will update this if I find anything new.<\/p>\n Motivation I’ve recently gotten completely fed-up with my backup process (using duplicity) in the way it gobbles up huge amounts of memory (6GB+ out of 8GB physical memory) when doing its daily backup. This annoyance is amplified by code in … Continue reading Context<\/h2>\n
Solution<\/h2>\n
$ cat \/etc\/systemd\/system\/duplicity.slice \r\n[Unit]\r\nDescription=Duplicity resource-limited slice\r\nBefore=slices.target\r\n\r\n[Slice]\r\nMemoryAccounting=true\r\nMemoryHigh=2G<\/pre>\n
Slice=duplicity.slice<\/pre>\n
$ systemctl daemon-reload<\/pre>\n
$ systemd-cgls # list running services by cgroup slice\r\n\r\n$ systemd-cgtop # list running services sorted by resource usage<\/pre>\n
References<\/h2>\n
\n