[IDEA] Making best use of 512mb Pi with tmpfs

[JayBlanc]

THIS IS DEPRICATED, DO NOT TRY IT.

This will no longer have a beneficial effect with future changes to memory handling and swap space in the next version. I recommend reversing the changes if you made them.

---------------------
Having a recent Rev.B I've modified my xbian set up to make a bit better use of the memory available.

First, I've mounted /tmp as a tmpfs. The defaults are fine because half of hardware memory as the top limit appears okay for the intended use. (Deliberately leaving /var/tmp alone as that may have large files written to it.)

Add the following line to /etc/fstab

Code:

tmpfs   /tmp         tmpfs   nodev,nosuid    0    0

Stopping XBMC, I symlink ~/.xbmc/temp to /tmp/, ~/.xbmc/userdata/addon_data/cache and ~/.xbmc/userdata/addon_data/script.common.plugin.cache to /tmp/, then reboot.

Terminal

sudo /etc/init.d/xbmc stop
rm /home/xbian/.xbmc/temp
ln -s /tmp /home/xbian/.xbmc/temp
rm /home/xbian/.xbmc/userdata/addon_data/cache
ln -s /tmp /home/xbian/.xbmc/userdata/addon_data/cache
rm /home/xbian/.xbmc/userdata/addon_data/script.common.plugin.cache
ln -s /tmp /home/xbian/.xbmc/userdata/addon_data/script.common.plugin.cache

Then create cron.daily script that reaps old files from /tmp/ to try and prevent overflow on a device that is rarely rebooted.

Code:

#!/bin/sh
#/etc/cron.daily/reaptmp

find /tmp/ -type f -mtime +3 -exec rm {} \;

Remember to run chmod a+r on the file so it will run every day. Then reboot.

Benefit: Noticeable responsiveness improvements in addons that create and process temporary files. Particularly ones that process downloaded rss/xml files, or download subtitle files.

Drawbacks: Caches cleared on reboot. But reboots are hopefully uncommon for XBian.

If anything fills /tmp/ with data over ~190mb, it's going to fail. I've yet to see this happen, and I don't know of any xbmc addon that would do this. If you know of a addon that uses creates temporary files over ~190mb, you might want to ask the addon's creator to store these files in a cache in it's addon_data directory. .xbmc/temp/ and common_cache appear to be intended for small temporary files only. tmpfs can be expanded beyond free memory, because it uses swap, but it can never be set above total hardware memory due to memory allocation handling.

There is the minor possibility that this could cause an issue due to file name collisions with something else. I haven't seen this happen on xbian, but it might be worth making a init script that properly creates a unique directory name in /tmp/ and creates the symlinks on boot.

[Koenkk]

good idea, but like you already said, we need to make 100% this causes no failures. Therefore it would be nice if people can test this and post their results here.

[CurlyMo]

I would wait for all the things Matus is implementing because he is already working on better ram usage due to different tmpfs solutions.

[JayBlanc]

This seems to conflict with the zram improvements that are coming up, and needs to be reverted if you want to take advantage of them without the system bogging down. With zram swap loopback, I'm noticing a great improvement in responsiveness.

[mk01]

despite the changes which are comming, I still like the idea of putting xbmc ./temp and some other caching files into memory.

debian is finally implementing standard linux features (8 years old!!!) like tmpfs shm on it etc - as standard feature.

@JayBlanc specially for this purpose I would choose /run/shm, not /tmp and /tmp I would leave solid. It's ok for you if you knows you system, but general many things can go wrong. /tmp is defined in posix with no limit in size. linux is changing /tmp from permanent to clean on reboot storage, but still no size limit.

feel free to play with /etc/default/tmpfs file, everything is managed via init according to those params.

and if you use mount --bind instead of delete / create symlinks, it will be completely transparent

(fstab records)
/run/shm /home/xbian/.xbmc/temp none bind 0 0
/run/shm /home/xbian/.xbmc/userdata/addon_data/script.common.plugin.cache none bind 0 0

[namtih]

Perhaps another idea would be vmtouch:
http://forum.xbian.org/thread-193-post-1783.html#pid1783

I added it to my startup script combined with some paths/files like

Code:

#!/bin/sh

#Preload Files
#touch /home/xbian/.xbmc/userdata/Database/zzstart_`date +%F_%T`
vmtouch -t /home/xbian/.xbmc/userdata/Database
vmtouch -t /home/xbian/.xbmc/userdata/addon_data/script.common.plugin.cache
vmtouch -t /home/xbian/.xbmc/addons/plugin.video.youtube
vmtouch -t /home/xbian/.xbmc/addons/plugin.video.dokumonster
vmtouch -t /home/xbian/.xbmc/addons/plugin.video.imdb.trailers
vmtouch -t /home/xbian/.xbmc/addons/plugin.video.the.trailers
#vmtouch -t /home/xbian/.xbmc/addons/plugin.video.hdtrailers_net-master
vmtouch -t /home/xbian/.xbmc/addons/script.globalsearch
vmtouch -t /usr/local/share/xbmc
vmtouch -t /usr/local/lib/xbmc
vmtouch -t /usr/lib/arm-linux-gnueabihf/gconv/gconv-modules.cache
#vmtouch -t /usr/share/pyload
vmtouch -t /usr/share/python
vmtouch -t /usr/bin/python2.7
vmtouch -t /usr/lib/python2.7
#vmtouch -t /home/xbian/.xbmc/addons/skin.realaskadark/fonts
#vmtouch -t /home/xbian/.xbmc/addons/skin.realaskadark/media/overlays
#vmtouch -t /home/xbian/.xbmc/addons/skin.realaskadark/media/views
vmtouch -t /home/xbian/.xbmc/addons/skin.refocus.big/fonts
vmtouch -t /home/xbian/.xbmc/addons/skin.refocus.big/media/overlays
vmtouch -t /home/xbian/.xbmc/addons/skin.refocus.big/media/views
vmtouch -t /usr/local/share/downloaddaemon
vmtouch -t /usr/local/bin/DownloadDaemon
vmtouch -t /usr/sbin/openvpn
vmtouch -t /etc/downloaddaemon
vmtouch -t /etc/openvpn/ovpn
vmtouch -t /lib/arm-linux-gnueabihf
vmtouch -t /bin/ntfs-3g
#touch /run/zzstop_`date +%F_%T`
exit 0

[JayBlanc]

Just a note that putting .xbmc/temp into a tmpfs absolutely breaks things in rar playback, because rar files get copied there.

[mk01]

@JayBlanc, you don't have the expansion memory module installed, to provide 8GB of ram? (it is for B model only).

[f1vefour]

(23rd Jun, 2013 01:22 AM)mk01 Wrote:  @JayBlanc, you don't have the expansion memory module installed, to provide 8GB of ram? (it is for B model only).

You have a link for this?