[Openstack] Understanding ephemeral and persistant volumes

When you ask Nova to boot a VM, nova-compute will connect to Glance and "GET" the image file from Glance and save it on the its local filesystem in "/var/lib/nova/instances/_base".

If Glance is set to use Swift as its backend storage, then Glance will get that file from Swift (through the Proxy). If not, then it will stream the file from Glance's filesystem (check the variable "filesystem_store_datadir" in the file "glance-api.conf" to see what Glance is set to use as backend store). 

So by default the disk of an instance is basically stored on the local filesystem of the server where the instance is running (in "/var/lib/nova/instances/instance-0000000X/disk"), and it's called ephemeral because when you terminate the instance the entire directory "/var/lib/nova/instances/instance-0000000X" gets deleted and the virtual disk is gone, but the base image in the "_base" directory is not touched. 

If the virtual disk is using qcow2 then only the changes that occur from the baseline are captured in the virtual disk, so the disk grows as the instance is changed more. The benefit is that you can have five instances using the same base template without using five times the space on the local filesystem (read http://people.gnome.org/~markmc/qcow-image-format.html for more info about qcow2). 

Persistent volumes are virtual disks that you attach to a running instance using the nova-volume service. These virtual disks are actually LVM volumes exported over iSCSI by the nova-volume server. They are called persistent because they are not affected by an instance being terminated, or by a nova-compute server crashing. 

You could just start a new instance and re-attach that volume and get your data back. The nova-volume is using LVM + iSCSI but there are drivers/plugins for Nexenta (and Netapp will release its own soon), so there are enterprise grade options available.

Table 10.1. Flavor parameters

Column	Description
ID	A unique numeric ID.
Name	A descriptive name, such as xx.size_name, is conventional but not required, though some third-party tools may rely on it.
Memory_MB	Virtual machine memory in megabytes.
Disk	Virtual root disk size in gigabytes. This is an ephemeral disk the base image is copied into. You don't use it when you boot from a persistent volume. The "0" size is a special case that uses the native base image size as the size of the ephemeral root volume.
Ephemeral	Specifies the size of a secondary ephemeral data disk. This is an empty, unformatted disk and exists only for the life of the instance.

OpenStack Storage Concepts

Table 6.1, “OpenStack storage” explains the different storage concepts provided by OpenStack.

Table 6.1. OpenStack storage

	Ephemeral storage	Block storage	Object storage
Used to…	Run operating system and scratch space	Add additional persistent storage to a virtual machine (VM)	Store data, including VM images
Accessed through…	A file system	A block device that can be partitioned, formatted, and mounted (such as, /dev/vdc)	The REST API
Accessible from…	Within a VM	Within a VM	Anywhere
Managed by…	OpenStack Compute (nova)	OpenStack Block Storage (cinder)	OpenStack Object Storage (swift)
Persists until…	VM is terminated	Deleted by user	Deleted by user
Sizing determined by…	Administrator configuration of size settings, known as flavors	User specification in initial request	Amount of available physical storage
Example of typical usage…	10 GB first disk, 30 GB second disk	1 TB disk	10s of TBs of dataset storage