Merge pull request #57 from djs55/snapshot

snapshots: put the revert implementation details in a subsection

Author: djs55

features/snapshots/coalesce1.graffle

@@ -29,10 +29,10 @@ conform to an api (the SMAPI) which has operations including
 - vdi_snapshot: create a snapshot of a disk
 
 
-Example vhd implementation
-==========================
+File-based vhd implementation
+=============================
 
-The existing "EXT" and "NFS" Xapi SM plugins store disk data in
+The existing "EXT" and "NFS" file-based Xapi SM plugins store disk data in
 trees of .vhd files as in the following diagram:
 
 ![Relationship between VDIs and vhd files](vhd-trees.png)
@@ -41,6 +41,32 @@ From the XenAPI point of view, we have one current VDI and a set of snapshots,
 each taken at a different point in time. These VDIs correspond to leaf vhds in
 a tree stored on disk, where the non-leaf nodes contain all the shared blocks.
 
+The vhd files are always thinly-provisioned which means they only allocate new
+blocks on an as-needed basis. The snapshot leaf vhd files only contain vhd
+metadata and therefore are very small (a few KiB). The parent nodes containing
+the shared blocks only contain the shared blocks. The current leaf initially
+contains only the vhd metadata and therefore is very small (a few KiB) and will
+only grow when the VM writes blocks.
+
+File-based vhd implementations are a good choice if a "gold image" snapshot
+is going to be cloned lots of times.
+
+Block-based vhd implementation
+==============================
+
+The existing "LVM", "LVMoISCSI" and "LVMoHBA" block-based Xapi SM plugins store
+disk data in trees of .vhd files contained within LVM logical volumes:
+
+![Relationship between VDIs and LVs containing vhd data](lun-trees.png)
+
+Non-snapshot VDIs are always stored full size (a.k.a. thickly-provisioned).
+When parent nodes are created they are automatically shrunk to the minimum size
+needed to store the shared blocks. The LVs corresponding with snapshot VDIs
+only contain vhd metadata and by default consume 8MiB. Note: this is different
+to VDI.clones which are stored full size.
+
+Block-based vhd implementations are not a good choice if a "gold image" snapshot
+is going to be cloned lots of times, since each clone will be stored full size.
 
 Hypothetical LUN implementation
 ===============================
@@ -85,9 +111,46 @@ We have fields that help navigate the new objects: ```VM.snapshot_of```,
 and ```VDI.snapshot_of```. These, like you would expect, point to the
 relevant other objects.
 
+Deleting VM snapshots
+=====================
+
+When a snapshot is deleted Xapi calls the SM API `vdi_delete`. The Xapi SM
+plugins which use vhd format data do not reclaim space immediately; instead
+they mark the corresponding vhd leaf node as "hidden" and, at some point later,
+run a garbage collector process.
+
+The garbage collector will first determine whether a "coalesce" should happen i.e.
+whether any parent nodes have only one child i.e. the "shared" blocks are only
+"shared" with one other node. In the following example the snapshot delete leaves
+such a parent node and the coalesce process copies blocks from the redundant
+parent's only child into the parent:
+
+![We coalesce parent blocks into grand parent nodes](coalesce1.png)
+
+Note that if the vhd data is being stored in LVM, then the parent node will
+have had to be expanded to full size to accommodate the writes. Unfortunately
+this means the act of reclaiming space actually consumes space itself, which
+means it is important to never completely run out of space in such an SR.
+
+Once the blocks have been copied, we can now cut one of the parents out of the
+tree by relinking its children into their grandparent:
+
+![Relink children into grand parent](coalesce2.png)
+
+Finally the garbage collector can remove unused vhd files / LVM LVs:
+
+![Clean up](coalesce3.png)
+
 Reverting VM snapshots
 ======================
 
+The XenAPI call `VM.revert` overwrites the VM metadata with the snapshot VM
+metadata, deletes the current VDIs and replaces them with clones of the
+snapshot VDIs. Note there is no "vdi_revert" in the SMAPI.
+
+Revert implementation details
+-----------------------------
+
 This is the process by which we revert a VM to a snapshot. The
 first thing to notice is that there is some logic that is called
 from [message_forwarding.ml](https://github.com/xapi-project/xen-api/blob/ce6d3f276f0a56ef57ebcf10f45b0f478fd70322/ocaml/xapi/message_forwarding.ml#L1528),