Problem
Occasional data loss or corruption occurred in some output files that were sent to the /nobackupnfs2 filesystem.
Status: Resolved
We made a number of configuration changes to the /nobackupnfs2 filesystem to address the problem. Subsequently, the problem has not recurred.
Background
Due to a server issue, any data sent to the /nobackupnfs2 filesystem within a certain period (30 minutes to several hours) before the server hung/rebooted was being held in memory and not written to disk. After the server rebooted, the data in memory was lost, resulting in data loss or corruption.
Normally, data is continuously written to disk, so rebooting the server causes minimal if any data loss; writes to the filesystem are held during the reboot and the data can resume writing when the server is back up.