![]() Those writes eventually result in three blocks in the lowest layer being modified. ![]() Let's say I update two files in the XFS layer. >Ahh OK, I think I see - since the block files are synced in full, you are always swapping blocks and doing ~1MB of writing no matter what. Manually syncing also lets me run `sync` and watch the `rclone sync` output, which gives me greater confidence that all the layers have synced successfully. So instead I chose to have the block files in a regular directory, and I make sure to `rclone sync` that directory to the WebDAV server when I make changes in the XFS layer. The directory containing the losetup block files could be `rclone mount`'d from the WebDAV server, but that would make the setup unavailable if I didn't have network access. The stack is XFS inside cryptsetup inside mdraid on top of losetup. Using 1920 as the number of sectors fixed that, though I'll probably just nuke the LUKS partition and rebuild it on top of dm_linear with 1953 sectors each. I discovered mdadm's behavior by inspecting the first two loopback devices and the RAID device in xxd. In my first attempt at replacing mdadm with dm_linear I used 1953 as the number of sectors, which led to garbage when decrypted with dm_crypt. One caveat is that my 1MB (actually 999936B) block devices have 1953 sectors (999936B / 512B) but mdadm had silently only used 1920 sectors from each. And `mdadm -stop` blocks for multiple minutes for some unexplained reason, whereas `dmcreate remove` is almost instantaneous. dev/loop1`, but it's all scripted anyway so it doesn't matter. Setting up the table input to `dmsetup create`'s stdin is more complicated than just `mdadm -build. dm_linear is an easier way than mdadm to concatenate the loopback devices into a single device. Of course, despite working on this for a week I only now discovered this. So any modification of files in the XFS layer eventually results in some of the 1MB blocks at the lowest layer being modified, and only those modified 1MB blocks need to be synced to the WebDAV server. In my case I created a LUKS device on top of this, then an XFS filesystem on the top of the LUKS device, then that XFS filesystem is my backup directory. From here the 10GB block device can be used as any other block device. Not only would metadata blocks use up a significant portion of the 1MB device size, but also I don't really need mdadm to "discover" this device automatically, and also the metadata superblock does not support 10000 devices anyway (the max is 2000 IIRC).Ĥ. `-build` means that mdadm does not store metadata blocks in the devices, unlike `-create` which does. ![]() `-level=linear` means the RAID device is just a concatenation of the underlying devices. This RAID device appears as a single block device of 10GB size. Create a RAID device over the 10000 loopback devices with `mdadm -build -level=linear`. Mount each file as a loopback block device using `losetup`.ģ. Create 10000 files, each of 1MB size, so that the total usage is 10GB.Ģ. So writing a file requires reuploading it, which is the same situation as S3.ġ. The server allows 10GB usage, but max file size is 250MB, and in any case WebDAV does not support partial writes. I use a WebDAV server for storing backups (Fastmail Files). This should, in theory, allow for out of order writes, existing partial object re-use, and file appends. Last year I discovered that S3 can do transactional I/O via multipart upload operations combined with the CopyObject operation. The sequential requirement for writes is the part that I've been mulling over whether or not it's actually required in S3. Modifying existing files will not be supported. Writes will only be supported to new files, and must be done sequentially. In the future, Mountpoint for Amazon S3 will support sequential writes, but with some limitations: Write operations (write, writev, pwrite, pwritev) are not currently supported. The SEMANTICS doc details what is and isn't supported from a POSIX filesystem API perspective, and this stands out: This is really interesting and something I've been thinking about for a while now.
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |