If the hard drive runs out of space during the writing of a son entry, the file will be corrupted, at least for large entries. (Corrupted in this case means that the entry will simply terminate in the middle of the JSON, without the closing ---.) The expected behaviour is that the write fails entirely, leaving the previously written entry as the last entry.
I think this is because appends to files on linux are not atomic (in the sense of: "it either succeeds and writes the whole entry, or fails without writing anything") if they are above a certain size limit: https://www.notthewizard.com/2014/06/17/are-files-appends-really-atomic/. I'm not sure there is anything we can do about this, but we should probably at least document the problem somehow.
If the hard drive runs out of space during the writing of a
sonentry, the file will be corrupted, at least for large entries. (Corrupted in this case means that the entry will simply terminate in the middle of the JSON, without the closing---.) The expected behaviour is that the write fails entirely, leaving the previously written entry as the last entry.I think this is because appends to files on linux are not atomic (in the sense of: "it either succeeds and writes the whole entry, or fails without writing anything") if they are above a certain size limit: https://www.notthewizard.com/2014/06/17/are-files-appends-really-atomic/. I'm not sure there is anything we can do about this, but we should probably at least document the problem somehow.