create-datablocks.markdown

layout
default

Adding a CEPH datablock to your Virtual Machine

We recommend you to run your operating system on a small SSD image and store your bulk data on Ceph datablock(s).

In this page we will show you how to create Ceph datablocks of customisable size for your bulk data and make it available to your Virtual Machine.

The steps you require to add a Ceph datablock to your Virtual Machine are:

1. Create a new empty datadisk
2. Add the new datadisk to the template
3. Mount the datadisk in the VM

Create a new empty datadisk

In this section we will be setting up an empty disk on ceph Datastore.

1. Login to the UI at the following URL: https://ui.hpccloud.surfsara.nl
2. From the menu at the left pane, select the Images page under Storage.
3. Click on the green [+] button on the top. A new window appears titled Create image.
4. Fill in the Create image form that popped up:
   • Name: give a name meaningful for your datablock, e.g. output data. You will use this name later in your Template.
   • Description (optional): give information about the datablock
   • Type: set to Generic storage datablock
   • Datastore: choose 106: ceph
   • Check This image is persistent checkbox. If you omit this, your data will not be saved on the disk once you shutdown the VM. You can also change this flag later from the image persistence properties.
   • On the Image location: group, choose radio button Empty disk image
   • Give it a Size in MB that is meaningful to you (e.g. insert 10000 to create a 10GB disk)
5. Click the green button Create on the form, to submit it.

NOTE:

A new image will show on the Images list, and it will keep in status LOCKED while it is being created. When it is created it will come to status READY. Then you still have to format and mount the disk.

Add the new datadisk to the template

In order to let you VM know about the new datablock, you need to add it to your VM's template by editing storage options.

1. On the UI, from the menu at the left pane select the VMs page under Templates.
2. Click on your template and then click on the blue Update button.
3. Select the Storage tab from the menu bar.
4. Click on the + button under Disk 0 (that will make a new Disk 1), and then choose the output data image you created as a second image.
5. Click the green button Update on the top, to submit it.

NOTE:

The datablock will be available to your VM the next time you instantiate it. If your VM is running then changes in the Template will not have any effect.

Mount the datadisk in the VM

In this section we show you how you can start using the new disk.

1. On the UI, from the menu at the left pane select the VMs page under Instances. Create a new VM by selecting the Template you created in the previous step.
2. Once the VM is in RUNNING state, ssh as root to the machine using your SSH-key.
3. Run fdisk -l and see that your new datablock is there (Disk /dev/vdb: 10.5 GB).
4. Mount the datadisk in the VM:

Warning:

In the following listing, the first mkdir and mkfs commands only need to be run once (and they do destroy everything on the image!). The mount command (fourth line) needs to be run every time you start the VM with that image. Alternatively, you can add this line to /etc/fstab to have it done automatically: /dev/vdb /data xfs defaults 0 0. For more information on fstab you can consult the Ubuntu Community Wiki.

sudo su -
mkdir /data  
mkfs -t xfs /dev/vdb  
mount /dev/vdb /data

Optionally you may apply the following commands (the one with mkdir -p onwards) in order to ask the operating system to perform some optimisations when using Ceph. Note that the directory /etc/rc.d/ may or may not exist, but the -p will make sure that it exists after the command is run.

mkdir -p /etc/rc.d/ && touch /etc/rc.d/rc.local
echo "echo 4096 > /sys/block/vdb/queue/read_ahead_kb" > /etc/rc.d/rc.local
chmod 755 /etc/rc.d/rc.local