SDcard and Flash memory errors
SD card corruption and eventual failure is common on embedded systems like the raspberry pi. I've realize there is a lack of consistent and reliable information on this. This post describes the problem and provides solutions.
This post has 3 parts: description of the problem, practical implications, and tricks to disable swap on the raspberry pi and related linux devices. People who just want a solution can skip to part 3.
In our lab we have a range of systems from expensive underwater robotics to small embedded IoT devices all of which use SD card or flash memory. There are modes of operation that are acceptable on a normal desktop or laptop that will gradually degrade and eventually destroy an embedded device after a few thousand cycles of writing to a given cell. Since the damage is gradual, it can easy to overlook the cause during setup and testing, and just blame it on "bad memory."
The root issue is that solid state memory wears out with successive changes to the contents, and this happens on a per-memory-cell basis. Modern SD and flash memory have sophisticated internal algorithms for "wear levelling" that try to reduce the impact and extend the life of the chips, but these cannot eliminate the problem totally, since it's intrinsic to the physics of the devices. Hard disks, by the way, can also wear out, but not generally as a function of the read-write behaviour. The key idea behind wear levelling it to change the physical location used for frequently-changing data, to spread the use.
[ Technical aside: As of 2011, 25nm MLC NAND lasted for about 3,000 write cycles. As of 2018 some sources suggest that single level cell NAND (SLC) can last for 10,000 cycles ]
For normal use, only a small amount of the device is written at a time and if this is moved around the device it can be many years before the flash memory storage wears out. Without wear levelling it's easy to write a malicious program that "burns through" a specific cell in just a few minutes (although OS cacheing policies also try to preclude this).
Note that there are differences between expensive SD cards and cheap ones. These differences include both the robustness of the memory elements themselves, their speed and also the nature of the wear levelling (e.g static vs dynamic).
Normal file IO will take many years to damage a modern (wear levelled) flash device. Swapping, however, can be much more destructive. The issue is that swapping is a mechanism used by the operating system to compensate for insufficient RAM (memory) and is depends on writing large amounts of data to "disk", and sometimes doing it a lot. This heavy usage could damage a flash device like an SD card in just a few months.
Note that swapping to a "slow device" will also make the system run slow, but on the other hand in can preclude a hard crash that could occur if the system runs out of memory.
The commands below require root (or prefix each command with sudo).
Turning off swapping
and also (to make it permanent):
update-rc.d dphys-swapfile remove
Turn off swapping temporarily
The following two lines work (or npt) depending on the kind of linxu distribution you have (i.e does it use systemd):
systemctl disable dphys-swapfile
Totally disable the swapping mechanism for keeps
apt-get remove dphys-swapfile
Aside from swapping, you can also make the entire filesystem read-only that preventing any writes (but also limiting the kinds of things you can do).