= Lun Aligning =

**Summary**: This post is about aligning disks in a virtual environment to prevent IO congestion. \\
**Date**: 10 November 2011 \\
**Refactor**: 3 January 2025: Checked links and formatting. Removed some old content. \\
{{tag>netapp vmware bash}}

For information about why you should keep LUNs aligned with your vmdks I'd like to refer to [[http://www.yellow-bricks.com/2010/04/08/aligning-your-vms-virtual-harddisks/|this article from yellow-bricks.com]]. It explains why alignment is important, but to sum it up: it comes down to performance. When the LUNs, vmdk and Guest OS blocks are not aligned the array will request more blocks than the VMs are actually asking for, causing more IO then required.
\\
Please note the following remarks:
# Windows Server 2008 and Windows Vista are aligned by default if the VM was created as Windows Server 2008/Windows Vista.
## LUNs less than 4GB in size will align to 64KB
## LUNs larger than 4GB will align to 1MB
# Citrix Servers are not supported because they remap the C-drive.
# Windows Dynamic Disks are not supported and will be corrupted if an alignment is performed, you will be warned with something like this: "partition type: unknown - 0x42". Do not align these partitions.
# Linux LVM volumes are not supported.
# If you align linux guests that use grub as their boot loader you'll have to reinstall grub
# Windows Server 2003 non-boot disks that have been added (d:, e:, etc) will need to be remapped in Computer Management. The drive letter will be lost on alignment.
# Because the virtual disks are being recreated you'll need enough free space on your volume/datastore.
# Aligning can only be done on VMs that are turned off (so downtime)
# Aligning is not supported on VMs that have snapshots or linked clones

= Performing Alignment =

Before you can actually start aligning you need to know which VMs should be aligned. In this article we use Windows native tools and NetApp tools. The NetApp tools can be obtained on several ways: First they are part of the [[http://now.netapp.com/NOW/download/software/sanhost_esx/ESX/|host utilities]] but they're also part of the [[vsc]], an extension you can install on your [[vcinstall|vCenter]]. I grabbed them from there and made them available on my ESX host:

<code>
[root@esx01 ~]# gunzip mbrtools.tar.gz
[root@esx01 ~]# tar -xf mbrtools.tar
[root@esx01 ~]# ll
total 3860
-rwxrwxrwx 1 root root   86949 Apr 28  2009 howto.pdf
-rw------- 1 root root    1629 Apr 13 21:53 ks.cfg
-rwxrwxrwx 1 root root  962284 Apr 16  2009 mbralign
-rwxrwxrwx 1 root root  134186 Apr 28  2009 mbralign1.2.pdf
-rwxrwxrwx 1 root root  771276 Oct  5  2008 mbrscan
-rw-r--r-- 1 root root 1960960 May 12 12:57 mbrtools.tar
-rwxrwxrwx 1 root root    1133 Apr 28  2009 readme.html
</code>

= Determine Which VMs Should Be Aligned =

I'll explain three ways to determine which VMs should be aligned. We'll use the NetApp tool mbrscan, we'll use the native Windows tools and we'll use a [[vspherepowercli|PowerCLI]] script.

== MBRscan ==

After obtaining the NetApp tools you can use the mbrscan tool:

# Navigate to the directory of the VM you'd like to check for alignment.
# If the VM is running, create an ESX snapshot.
# In this directory run /root/mbrscan <filename>-flat.vmdk (NOTE: you are checking the –flat.vmdk file for proper alignment)
# Check if your virtual disk file is misaligned.
# Delete the snapshot.
\\
This option can be used when scanning one VM or when scanning an entire host (while using the -all switch).

=== MBRScan Example ===

If VM is running:
<code>
[root@esx01 WINXP01]# /root/mbrscan WINXP01-flat.vmdk
--------------------
Failed to open WINXP01-flat.vmdk - [Device or resource busy]
--------------------
</code>

So create a snapshot and rerun:
<code>
[root@esx01 WINXP01]# /root/mbrscan WINXP01-flat.vmdk
--------------------
WINXP01-flat.vmdk p1 (NTFS)    lba:63  offset:32256    aligned:No
--------------------
</code>

> Note: You can run the tool using the "--all" switch which will scan all -flat.vmdk files that are on the host.

== Native Windows Tool MSInfo32 ==

You can check disks from within Windows:

* Run Start -> All Programs -> Accessories -> System Tools -> System Information (or Start -> Run -> msinfo32 ).
* Go to Components -> Storage -> Disks and check the value for Partition Starting Offset.
* The value listed here should be completely divided by 4096. If so, the disk is aligned.
* Check every disk in the list.
\\
For misaligned VMs, you typically will find that the VM is running with a default starting offset value of 32,256, which is not completely divisible by 4,096; hence, the partition is not aligned.
\\
This option can be used when you just want to know if one particular VM is aligned.

== PowerCLI ==

You can also use this powercli script to determine per vCenter/Datacenter/Folder/Cluster/Host which VM Disks are misaligned:

<code powershell>
# Original script: http://ict-freak.nl/2009/12/15/powercli-check-partition-alignment-windows-vms-only/
# Changed: Sjoerd Hooft - 10 Nov 2011

$myCol = @()

$timestamp = Get-Date -format "yyyyMMdd-HH.mm"
$cluster = "Production"
$csvfile = "D:\sjoerd\$timestamp-alignmentinfo-$cluster.csv"

#$vms = Get-Datacenter "The Netherlands" | get-vm | where {$_.PowerState -eq "PoweredOn" -and $_.Guest.OSFullName -match "Microsoft Windows*" } | Sort Name
$vms = Get-Cluster $cluster | get-vm | where {$_.PowerState -eq "PoweredOn" -and $_.Guest.OSFullName -match "Microsoft Windows*" } | Sort Name

Write-Host (get-date -uformat %I:%M:%S) "Please provide alternative authentication credentials for VMs" -ForegroundColor Green;
$seccredentials = $Host.UI.PromptForCredential("Please enter credentials", "Enter alternative credentials", "domain\sjoerd", "")

foreach($vm in $vms){
  Write-Host (get-date -uformat %I:%M:%S) "Now starting $vm"
  $wmi = $null
  # First use default credentials
  $wmi = get-wmiobject -class "Win32_DiskPartition" -namespace "root\CIMV2" -ComputerName $vm
  # Secondly, use the alternative credentials as supplied
  if ($wmi -eq $null){
  $wmi = get-wmiobject -class "Win32_DiskPartition" -namespace "root\CIMV2" -ComputerName $vm -Credential $seccredentials
  #$wmi
  }
  # Use default authentication, but if unsuccesfull try other credentials until succesfull
  $teller = 1
  while (($wmi -eq $null) -and ($teller -lt 4)){
     Write-Host (get-date -uformat %I:%M:%S) "Enter Credentials for $vm. This is try $teller, with a maximum of 3 tries." -Foregroundcolor Red
	 $wmi = get-wmiobject -class "Win32_DiskPartition" -namespace "root\CIMV2" -ComputerName $vm -Credential Administrator
	 $teller++
  }
  foreach ($objItem in $wmi){
    $Details = "" | Select-Object VMName, Partition, Status, StartingOffSet,Size
    if ($objItem.StartingOffset -eq $null){
	  Write-Host "Checking $vm"
	  $vmview = Get-VM $vm | Get-View
	  $Details.VMName = $vmview.Name
	  $Details.Status = "VM could not be analyzed"
	}
	else{
      # Divide StartingOffset by 4096 and round the number to zero decimals
	  # If both numbers are the same then the partition is aligned
	  $divoffset = ($objItem.StartingOffset / 4096)
	  if ([System.Math]::Round($divoffset,0) -eq $divoffset){
	    $Details.VMName = $objItem.SystemName
        $Details.Partition = $objItem.Name
        $Details.Status = "Partition aligned"
	    $Details.StartingOffSet = $objItem.StartingOffset
		$Details.Size = $objItem.Size
      }
      else{
        $Details.VMName = $objItem.SystemName
        $Details.Partition = $objItem.Name
        $Details.Status = "Partition NOT aligned"
	    $Details.StartingOffSet = $objItem.StartingOffset
		$Details.Size = $objItem.Size
      }
    }
  $myCol += $Details
  }
}
$myCol | Export-Csv -NoTypeInformation $csvfile
</code>

= Aligning VMDK Files =

For alignment we'll use the mbralign tool. To do so, follow these steps:

# Remove any ESX snapshots from the VM that is to be realigned.
# Shut down the VM.
# Navigate to the directory of the VM you'd like to realign.
# Run the mbralign tool by using the command /root/mbralign <filename>.vmdk (NOTE: you are aligning the .vmdk file).
# To verify the virtual disk is now aligned run the mbrscan tool again.
# You should now be able to power the VM on.
# If the VM uses the grub boot loader (linux), please see appendix A.
# If the VM boots properly and you are satisfied with the result, the backup files should be removed from the virtual machine
directory in the datastore. These files end in “-mbralign-backup” and can be removed from the Virtual Center Client datastore
browser or from the ESX service console.

== Aligning Example ==

Shutdown the VM and make sure no snapshots exist:
<code>
[root@esx01 WINXP01]# ll
total 6676992
-rw-r--r-- 1 root root      82109 Apr 12 16:20 vmware-45.log
-rw-r--r-- 1 root root      69145 Apr 12 16:20 vmware-46.log
-rw-r--r-- 1 root root      66815 Apr 12 16:20 vmware-47.log
-rw-r--r-- 1 root root      81427 Apr 12 16:20 vmware-48.log
-rw-r--r-- 1 root root      54797 Apr 12 16:20 vmware-49.log
-rw-r--r-- 1 root root      62878 Apr 12 16:20 vmware-50.log
-rw-r--r-- 1 root root      64727 Apr 12 20:33 vmware-51.log
-rw-r--r-- 1 root root      97232 May 12 14:29 vmware.log
-rw------- 1 root root 8589934592 May 12 14:29 WINXP01-flat.vmdk
-rw------- 1 root root       8684 Apr 13 22:33 WINXP01.nvram
-rw------- 1 root root        566 Apr 12 16:20 WINXP01.vmdk
-rw-r--r-- 1 root root         43 May 12 13:42 WINXP01.vmsd
-rwxr-xr-x 1 root root       2814 May 12 14:29 WINXP01.vmx
-rw-r--r-- 1 root root       1587 Apr 13 22:33 WINXP01.vmxf
</code>

Start the mbralign tool and confirm there are no snapshots:
<code>
[root@esx01 WINXP01]# /root/mbralign WINXP01.vmdk
 Part    Type          old LBA    New Start LBA      New End LBA     Length in KB
   P1      07               63               64         16755796          8377866

NOTICE:
This tool does not check for the existence of Virtual Machine snapshots or linked clones.
The use of this tool on a vmdk file that has a snapshot or linked clone associated with it
can result in unrecoverable data loss and/or data corruption.
Are you sure that no snapshots/linked clones exist for this vmdk? (y/n)y
Creating a backup of WINXP01.vmdk
Creating a backup of ./WINXP01-flat.vmdk
Creating a copy the Master Boot Record
Working on partition P1 (3): Starting to migrate blocks from 32256 to 32768.
12801 read ops in 6 sec.  59.82% read (15.74 mB/s).  59.82% written (15.74 mB/s)
---
12801 read ops in 2 sec.  98.88% read (46.99 mB/s).  98.88% written (46.99 mB/s)
Working on space not in any partition: Starting to migrate blocks.
100.00 percent complete.  100.00 percent written. .
Making adjustments to ./WINXP01-flat.vmdk.
Adjusting the descriptor file.

[root@esx01 WINXP01]# ll
total 15066688
-rw-r--r-- 1 root root      82109 Apr 12 16:20 vmware-45.log
-rw-r--r-- 1 root root      69145 Apr 12 16:20 vmware-46.log
-rw-r--r-- 1 root root      66815 Apr 12 16:20 vmware-47.log
-rw-r--r-- 1 root root      81427 Apr 12 16:20 vmware-48.log
-rw-r--r-- 1 root root      54797 Apr 12 16:20 vmware-49.log
-rw-r--r-- 1 root root      62878 Apr 12 16:20 vmware-50.log
-rw-r--r-- 1 root root      64727 Apr 12 20:33 vmware-51.log
-rw-r--r-- 1 root root      97232 May 12 14:29 vmware.log
-rw-r--r-- 1 root root 8589935104 May 12 14:38 WINXP01-flat.vmdk
-rw------- 1 root root 8589934592 May 12 14:29 WINXP01-flat.vmdk-mbralign-backup
-rw------- 1 root root       8684 Apr 13 22:33 WINXP01.nvram
-rw------- 1 root root        566 May 12 14:38 WINXP01.vmdk
-rw-r--r-- 1 root root        566 May 12 14:31 WINXP01.vmdk-mbralign-backup
-rw-r--r-- 1 root root         43 May 12 13:42 WINXP01.vmsd
-rwxr-xr-x 1 root root       2814 May 12 14:29 WINXP01.vmx
-rw-r--r-- 1 root root       1587 Apr 13 22:33 WINXP01.vmxf

[root@esx01 WINXP01]# /root/mbrscan WINXP01-flat.vmdk
--------------------
WINXP01-flat.vmdk p1 (NTFS)    lba:64  offset:32768    aligned:Yes
--------------------
</code>
As you can see, the vmdk file is aligned now and there are two backup files. The VM started successfully so the two backup files van be deleted.

> Note: You can run the mbralign utility with several options:
<code>
Options:
   --sparse          The resulting file will be sparse.  This option is
                      designed for NetApp NFS based datastores.
   --force           The default behavior is to skip properly aligned disks.
                      The --force option can be used to
                      override this. (Useful for migrating and making sparse).
   --bs=n            Use n as the block size (specified in kB).  n must be one
                      of (8, 16, 32, 64, 128, 1024).  The default is 8.
   --codeview         Show what would be done, then exit.  No changes are made
                      to the file.
   --quiet           Only print warnings and errors (no other status isi
                      printed to screen).
   --debug           Print debug comments.
   --help            Print usage.
   --thereAreNoVmSnapshots  Supcodess the warning message about snapshots and
                             answer YES to the question of whether to continue.
</code>

= Linux Systems Using GRUB =

The following steps are required to make linux machines bootable again:
# After realignment, boot from old vmdk or CD (iso)
# Use grub to fix stage1.5 and stage2

== Example Fixing MBR ==

<code>
grub>find /boot/grub/stage1
(hd0,0) [old, misaligned vmdk file]
(hd1,0) [new, aligned vmdk file]
grub>root (hd1,0)
grub>setup (hd1)
Checking if "/boot/grub/stage1" exists... yes
Checking if "/boot/grub/stage2" exists... yes
Checking if "/boot/grub/e2fs_stage1_5" exists... yes
Running "embed /boot/grub/e2fs_stage1_5 (hd1)"... 17 sectors are embedded.
succeeded
Running "install /boot/grub/stage1 (hd1) (hd1)1+17 p (hd1,0)/boot/grub/stage2 /boot/grub/menu.lst"...
succeeded
Done.
</code>

= Creating Aligned Disks =

To prevent the issue from happening you could also create new disks for new VMs with the partitions already aligned.

== OS Disks ==

Virtual disks to be used as the boot disk can be formatted with the correct offset at the time of creation by connecting the new virtual disk to a running VM before installing an operating system and manually setting the partition offset. For Windows guest operating systems, you might consider using an existing Windows Preinstall Environment boot CD or alternative tools like Bart’s PE CD. To set up the starting offset, follow these steps.
* Boot the VM with the WinPE CD.
* Select Start -> Run and enter DiskPart.
* Enter Select Disk0.
* Enter "create partition primary align=32"

== Data Disks ==

To format virtual disks to be used as the data disk with the correct offset at the time of creation, use DiskPart in the VM. Attach the data disk to the VM. Check that there is no data on the disk and follow these steps:
* Select Start -> Run.
* Enter diskpart.
* Enter list disk to determine the disk # for the new data disk.
* Enter select disk <disk_number> (for example, select disk 1).
* Enter "create partition primary align=32".
* Enter exit to exit the DiskPart utility.
* Format the data disk as you normally do

= Lun Alignment Script =

This script can be used to align VMDKs from the console on an ESX server:
Please note the following manual actions:
* Check for enough diskspace since the VMDK files get backupped
* VM should be running on the host where you run the script
* After alignment disk mappings other than C: are lost and have to be mapped again manually
* Remove backup files after manual check
  * For example: rm `ls *mbralign`

> Note: because of bugs in newer versions and a different approach you'll need the 5.1 version of the host utilities to use this script.

<code bash>
#!/bin/bash
########################################################################################################################
# Author : Sjoerd Hooft
# Date Initial Version: 27 December 2011
# Comments: sjoerd_warmetal_nl
#
# Description:
# Script to align the disks of a Virtual Machine
#
# Recommendations:
# The script is designed for a 120 column terminal.
# The running user must be root.
#
# Changes:
# Please comment on your changes to the script (your name and email address, line number, description):
# DATE - USERNAME - EMAILADDRESS - CHANGE DESCRIPTION
########################################################################################################################

# Debug option; uncomment for debugging
# set -x

# Script Variables
HOSTNAME_SHORT=`hostname -s`
BASEDIR=`dirname $0`
WHATAMI=`basename $0`
LOGFILE="$BASEDIR/$WHATAMI.log.$1"
DATE=`date +%Y%m%d`
# Highlight output on screen
BOLD=`tput bold`
BOLDOFF=`tput sgr0`
# Send all output to logfile; disable if screen output is needed
exec > $LOGFILE 2>&1

# Mail Variables
EMERGENCY="mobile@mijngsm.nl"
REPORT="sjoerd_warmetal.nl"
RELAY="10.10.10.10"

# Align Variables
align=/tmp/netapp/mbralign

# Write down the name of the Virtual Machine to align
inputvm=$1

if [ -z "$inputvm" ]; then
   echo
   echo "Usage $0 Script:"
   echo "--------------------------------------"
   echo "$BOLD $0 <NAME of VM> $BOLDOFF"
   echo "--------------------------------------"
   echo
   echo NAME of VM: Name of VM as found in vCenter, case sensitive
   echo
   exit
fi

functioncheckvm() {
vmxfile=`vmware-cmd -l | grep $inputvm`

checkvmexists=`vmware-cmd -l | grep $inputvm | wc -l`

if [ "$checkvmexists" != "1" ]; then
   echo VMexists is $checkvmexists
   echo VM $inputvm does not exist.
   /usr/local/bin/smtp_send.pl -t $EMERGENCY -s "VM $inputvm kan niet gevonden worden. Draait deze wel op deze host?" -f `hostname -s`@warmetal.nl -r $RELAY
   echo Is $inputvm located on this host? If not please vMotion the VM to this host.
   echo Exiting...
   exit
fi

snapshotexist=`vmware-cmd $vmxfile hassnapshot | cut -f3 -d" "`

if [ "$snapshotexist" != "0" ]; then
   echo Snapshotexists is $snapshotexist
   echo VM $inputvm has snapshot, exiting...
   exit
fi

echo VM $inputvm does esixt and has no snapshots. Continuing...

}

functionstopvm() {

vmware-cmd $vmxfile stop trysoft

stopstate=on
stopcounter=0
while [ "$stopstate" != "off" ]
do
  stopcounter=`expr $stopcounter + 1`
  stopstate=`vmware-cmd $vmxfile getstate | cut -f3 -d" "`
  echo Stopstate = $stopstate and Stopcounter = $stopcounter
  sleep 10
  if [ "$stopcounter" == "60" ]; then
     /usr/local/bin/smtp_send.pl -t $EMERGENCY -s "VM $inputvm stopt niet correct. Er zijn nog 10 minuten om de VM te stoppen." -f `hostname -s`@warmetal.nl -r $RELAY
  fi
  if [ "$stopcounter" == "120" ]; then
     Echo "VM $inputvm is niet correct gestopt. Het script wordt afgebroken."
     /usr/local/bin/smtp_send.pl -t $REPORT -s "VM $inputvm is niet correct gestopt. Het script wordt afgebroken." -f `hostname -s`@warmetal.nl -m "`cat $LOGFILE`" -r $RELAY
     exit
  fi
done

}

functionaligndisks() {

# Record time
echo start time is `date +%H%M`

for disks in $(cat $vmxfile | grep vmdk | grep -v vmfs | cut -f2 -d"\"" ); do
  dir=`dirname $vmxfile`
  echo $dir/$disks
  $align --thereAreNoVmSnapshots $dir/$disks
done

for disks in $(cat $vmxfile | grep vmdk | grep vmfs | cut -f2 -d"\"" ); do
  echo $disks
  $align --thereAreNoVmSnapshots $disks
done

# Record time
echo end time is `date +%H%M`

}

functionstartvm() {

vmware-cmd $vmxfile start trysoft

startstate=5
startcounter=0
while [ "$startstate" != "1" ]
do
  startcounter=`expr $startcounter + 1`
  startstate=`vmware-cmd $vmxfile gettoolslastactive | cut -f3 -d" "`
  echo Startstate = $startstate en startcounter = $startcounter
  sleep 10
  if [ "$startcounter" == "60" ]; then
     /usr/local/bin/smtp_send.pl -t $EMERGENCY -s "VM $inputvm start niet correct. Er zijn nog 10 minuten om de VM te starten." -f `hostname -s`@warmetal.nl -r $RELAY
  fi
  if [ "$startcounter" == "120" ]; then
     Echo "VM $inputvm is niet correct gestopt. Het script wordt afgebroken."
     /usr/local/bin/smtp_send.pl -t $REPORT -s "VM $inputvm is niet correct gestart. Het script wordt afgebroken." -f `hostname -s`@warmetal.nl -m "`cat $LOGFILE`" -r $RELAY
     exit
  fi
done
/usr/local/bin/smtp_send.pl -t $EMERGENCY -s "VM $inputvm is aligned en moet handmatig worden gecontroleerd." -f `hostname -s`@warmetal.nl -r $RELAY

}

functioncheckvm
functionstopvm
functionaligndisks
functionstartvm

/usr/local/bin/smtp_send.pl -t $REPORT -s "Alignment report $inputvm." -f `hostname -s`@warmetal.nl -m "`cat $LOGFILE`" -r $RELAY

exit
</code>

> Note: to use the smtp_send.pl to send email from the ESX service console you need to configure this [[esxemailserviceconsole|script]]

== Schedule Alignment with Cron ==

This script can be scheduled using [[cron]], this would be the way to schedule it on the 24th of February on 11 AM:
<code>
# Start alignment
0 11 24 2 * /usr/local/bin/alignment.sh vmname01
</code>

= Determining Alignment On NetApp Filer =

You can use a diag command to view the alignment for a specific LUN:
<code>
filer01b*> priv set diag
filer01b*> lun alignment show /vol/01B_SATA/01B204_SATA
        /vol/vol/01B_SATA/01B204_SATA
                Multiprotocol type: vmware
                Alignment: aligned
                Write alignment histogram percentage: 76, 0, 0, 0, 0, 0, 0, 0
                Read alignment histogram percentage: 95, 0, 0, 0, 0, 0, 0, 0
                Partial writes percentage: 19
                Partial reads percentage: 1
</code>
> Note: In a previous version of this article this was explained wrong. Although applications with specific write patterns can cause spikes in the misaligned sections the overall explanation was wrong.

See here the correct explanation, with thanks to an anonymous contribution on my previous website:

The 8 figures in the histogram row are not about the percentage of overall misalignment, they are about which 512 Byte "sector" of a 4k wafl block got hit the most. Obviously you want to have 100% at the first figure, and all zeroes on the rest. Such a histogram would tell you that everything is aligned perfectly. If you have the highest value on the 7th figure (or 6th?) its probably a misaligned XP or windows 2003 system accessing the LUN. if you have spikes on the 1st and the 7th value, then the LUN is probably a VMware datastore, with some aligned and some misaligned VMs.

How misaligned a LUN is would be determined by the percentage of I/O not in the first figure. However, misalignment is (with the exception of limited cases involving multiple partitions per LUN) a yes/no question, not a question of how much. Generally in a misaligned scenario you will see one or two of the buckets with 90%+ of the I/O. If all of the buckets have a significant percentage, it is more likely that you have an application with a specific write pattern (as is the case with some databases) and not a misaligned partition.

So, the example above is aligned and the example below is misaligned:

<code>
/vol/01B_FC_RDM2/01B52_XXXXXXXXXX
                Multiprotocol type: windows
                Alignment: misaligned
                Write alignment histogram percentage: 0, 4, 0, 0, 0, 0, 30, 0
                Read alignment histogram percentage: 0, 94, 0, 0, 0, 0, 0, 0
                Partial writes percentage: 63
                Partial reads percentage: 1
</code>