entangling exuberance
talks   experiments   about  
Enterprise Infrastructure Series Part 1 - Bootstrap OpenVZ
Dec 31, 2011 • 5 minutes

We have been working with OpenVZ for almost 4 years now, and its just amazing that how this nice piece of technology not only provides 99% native virtualization but also wins on Disk IO. OpenVZ provides way more containers or OS Instances compared to any hypervisor.

Here is sample comparision

96 GB RAM,  24 Cores X5650  @ 2.67GHz

For KVM it can host around ~40 2 GB Hypervisor based virtual machines, After around ~30 machines you can see performance going down and if you have high Disk IO build machines, its gonna be slow - this is almost true with every type of hypervisor

For OpenVZ it can host around 100+ containers and depends on free memory - since containers will consume only actual memory and OpenVZ can still allocate more containers based on free memory.

So lets get started with setting up OpenVZ on CentOS 6.2 box.

Hardware summary: 24 Cores, 96 GB RAM, 2 TB Box

Step 1

Install CentOS 6 with customized LVM layout. Allocated 500 MB on /boot (/dev/sda1), created 3 Physical Volumes of 200 GB - (/dev/sda2), 1 TB (/dev/sda3) and 800 GB (/dev/sda4)

In first physical volume, create volumegroup and then 2 logical volumes for 145 GB (mounted on /) and (50 GB swap) - this will leave around 5 GB for future use and snapshots

In second physical volume, just create a volumegroup and logical volume for 500 GB - it should have mount point /vz

This should complete CentOS setup

Step 2

Install OpenVZ repo

wget http://download.openvz.org/openvz.repo

rpm --import  http://download.openvz.org/RPM-GPG-Key-OpenVZ

 

Search for suitable for kernel and OpenVZ utils

yum search all OpenVZ

yum search all vzkernel

 

Then I installed OpenVZ with this command

yum install vzkernel.x86_64 vzkernel-firmware.noarch vzctl.x86_64 vzquota.x86_64 vzctl-lib.x86_64

Step 3

After you have installed OpenVZ - you need to modify three configuration files

/boot/grub/grub.conf - Make sure that default points to OpenVZ kernel

# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/mapper/rootvg-rootlv
#          initrd /initrd-[generic-]version.img
#boot=/dev/sdc
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.32-042stab044.11)
        root (hd0,0)
        kernel /vmlinuz-2.6.32-042stab044.11 ro root=/dev/mapper/rootvg-rootlv rd_NO_LUKS rd_LVM_LV=rootvg/rootlv LANG=en_US.UTF-8 rd_NO_MD quiet rd_LVM_LV=rootvg/swaplv SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM
        initrd /initramfs-2.6.32-042stab044.11.img
title CentOS (2.6.32-220.2.1.el6.x86_64)
        root (hd0,0)
        kernel /vmlinuz-2.6.32-220.2.1.el6.x86_64 ro root=/dev/mapper/rootvg-rootlv rd_NO_LUKS rd_LVM_LV=rootvg/rootlv LANG=en_US.UTF-8 rd_NO_MD quiet rd_LVM_LV=rootvg/swaplv SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM
        initrd /initramfs-2.6.32-220.2.1.el6.x86_64.img
title CentOS (2.6.32-220.el6.x86_64)
        root (hd0,0)
        kernel /vmlinuz-2.6.32-220.el6.x86_64 ro root=/dev/mapper/rootvg-rootlv rd_NO_LUKS rd_LVM_LV=rootvg/rootlv LANG=en_US.UTF-8 rd_NO_MD quiet rd_LVM_LV=rootvg/swaplv SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM
        initrd /initramfs-2.6.32-220.el6.x86_64.img

Then my modified sysctl.conf looks like this, you can apply this as it is, or look for changes here -> http://wiki.openvz.org/Quick_installation#sysctl

/etc/sysctl.conf - You need to change various parameters, but if you have freshly install system, you can replace with the below code

## Sysctl.conf

# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled.  See sysctl(8) and
# sysctl.conf(5) for more details.

# Controls IP packet forwarding
net.ipv4.ip_forward = 1

# Controls IP v6 packet forwarding
net.ipv6.conf.default.forwarding = 1
net.ipv6.conf.all.forwarding = 1

# Controls ARP Proxy
net.ipv4.conf.default.proxy_arp = 0

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

# And all interfaces as well
net.ipv4.conf.all.rp_filter = 1

# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
# Enables the magic-sysrq key
kernel.sysrq = 1

# Controls whether core dumps will append the PID to the core filename.
# Useful for debugging multi-threaded applications.
kernel.core_uses_pid = 1

# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1

# Disable netfilter on bridges.
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0

# Controls the maximum size of a message, in bytes
kernel.msgmnb = 65536

# Controls the default maxmimum size of a mesage queue
kernel.msgmax = 65536

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 68719476736

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296

# We do not want all our interfaces to send redirects
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 0

/etc/sysconfig/selinux  - You need to change SELINUX to disabled
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted 

Step 4

Boot into OpenVZ kernel - and after reboot everything should be okay and your /vz mount point should contain vz files. Now you can start downloading OpenVZ precreated templates, and put them in /vz/template/cache - you can download vz templates from here http://wiki.openvz.org/Download/template/precreated

Important

Dont go with CentOS default partitioning layout, you should have enough space for your new volumegroups and logical volumes to make sure that you can take backup for VZ and other maintainance tasks.

 

 

 

Share