littlefs VS ocaml-tar

Pointers correctly round-trip through unrelated pointer types, provided the alignment for both types is compatible.

At least one bit does look a bit questionable: lfs_mlist is treated as the common initial sequence of lfs_dir and lfs_file, even though it isn't, and common initial sequences only apply to union fields anyway. Example cast of struct dir * to struct lfs_mlist * (probably valid of itself, assuming the alignment is compatible): https://github.com/littlefs-project/littlefs/blob/c733d9ec57... then use struct dir as if it were actually a struct lfs_mlist: https://github.com/littlefs-project/littlefs/blob/c733d9ec57...

(There's other occurrences of the same kind of thing.)

Strictly speaking, I think this might be unfixable without a bunch of work, but so much stuff does this kind of operation that any compiler that doesn't do what you expect will have been fixed by now. (Assuming you're not one of those people who is going to pop up and tell us with a straight face that what we should expect is for the compiler to do absolutely anything - except, perhaps, for having it generate correct code, which would be defective behaviour that should be eliminated.) Maybe improve the odds by using lfs_mlist as the common initial sequence of both structs, and fingers crossed that the compiler considers the union rules to apply to this case too. Or compile with -fno-strict-aliasing.

Also, it looks like you're writing to the flash every second the engine is running. Do you have an idea of what the endurance of that might be? I'm not too familiar with this but supposedly the flash on the Pico is good for at least 100K program/erase cycles and Micropython uses LittleFS on RP2040 which does wear leveling. I looked for more official info and the rp2 port code backs that up with a note: "the flash requires the programming size to be aligned to 256 bytes". And the littlefs readme does say it does wear leveling and other good stuff.

Maybe first go and localize which assumption has been voided, i.e. what of the many places in littlefs that raise this error actually raise it? And... erm... what littlefs are you talking about? https://github.com/littlefs-project/littlefs doesn't seem to have a fs_close() function...

Rather easy to add LittleFS for file storage onto the NOR flash. https://github.com/littlefs-project/littlefs

I helped build a tar-backed filesystem for MirageOS[0]. It is definitely easiest to make it append-only. I did add in-place renaming, updates (of file content), pre-allocating a file and removing the last file.

We also scan the filesystem on boot and keep a mapping in-memory of file names to file metadata and block offsets, and update this memory representation when modifying the on-disk tar filesystem. If you have a lot of small files this is maybe not a great idea as you will be storing most in memory, though.

For the purpose we had in mind it worked fine: A content addressable mirror of package archives. The archives exist somewhere on the web and the package file has a link to the archive and a checksum. We can then download the package and store it by the checksum. This gives integrity that the tar filesystem does not offer. Removing the last file works great if you only have one download job. Otherwise if a download fails and it is not the latest file you can rename it to `failed/-` and do garbage collection in the filesystem at a later point.

[0]: https://github.com/mirage/ocaml-tar/blob/main/mirage/tar_mir...