Scripting VM Recovery for DR or other mass restores

Our DR plan involves restoring a lot of VMware virtual machines to a new Vcetner. I wrote a script to be run on the Avamar utility node to automate it. One quirk about restoring from a replicated copy with this method is that when you replicate a VM backup to another grid it adds a tag to the end of the backup name so you can't just use a client name list. To make this script work you need to:

1. Put it and the other required files in /home/admin/vmware_restore_script
2. Go into the GUI and copy/paste all names of the backup names (clientname_avamartag) into a file called vmware_lookup_list 
3. create a file called vmlist containing all of the servers you want to restore. 

This script is also customized to our ESX host names and datastore names. You can set the number of hosts and datastores and it will loop through them to distribute the VMs and Hosts equally. 

It does a minor amount of error checking and logs to an output file so you can see what VMs did not successfully submit a restore.

If anyone knows how to get a backup list based on a domain from CLI when there are no clients registered on that grid (just replica copies) please let me know. That would fix the whole vm_lookup_list part of this.

Feel free to use this and modify as needed. I'd appreciate any feedback if you think there is something to make it better.

#!/bin/bash
## Christian Schmidt 4/6/2015
## cgschmidt@landolakes.com
## Script to bulk restore VMs @ DR. It must resides in /home/admin/vmware_restore_script/
## folder must have vm_lookup_list containing the full avamar name of the servers (cut and paste from the restore area in the GUI)
## folder must have vmlist containing a simple list of vmnames to be restored
## ESX hosts names must be formatted as DRVMhost(count) edit the count of ESX hosts by changing ESXHOSTCOUNT variable
## Datastores must be formatted as vol(count) edit the count of datastores by changing the DSCOUNT variable
## montitor /home/admin/vmware_restore_script/vmrestore.log for success or failure of particular backups

##Domain replicated backups exist in
DOMAIN=REPLICATE/ahavmrun.ent.lolcentral.com/vmware.ent.lolcentral.com/windows-prod
##fqdn of the Vcenter
TGTVCENTER=vmware2.ent.lolcentral.com
##VMware Datacenter
TGTDC="Atlanta"
##VMware folder to put the VM in
TGTFOLDER=avamar_restores
##number of esxhosts named DRVMhost#
ESXHOSTCOUNT=3
##number of datastores named vol#
DSCOUNT=2
##setting counters
ESXCOUNTER=0
DSCOUNTER=0

echo "============================================================================" >> /home/admin/vmware_restore_script/vmrestore.log
echo "Starting VMware restore script at $(date)" >> /home/admin/vmware_restore_script/vmrestore.log
echo "============================================================================" >> /home/admin/vmware_restore_script/vmrestore.log
for SHORTSERVER in $(cat /home/admin/vmware_restore_script/vmlist)
do
        if [[ $(grep -i -c $SHORTSERVER /home/admin/vmware_restore_script/vm_lookup_list) -eq 1 ]]
        then
                SERVER=$(grep -i $SHORTSERVER /home/admin/vmware_restore_script/vm_lookup_list)
        else
                echo "ERROR: failed to lookup full server name for" $SHORTSERVER >> /home/admin/vmware_restore_script/vmrestore.log
                continue
        fi
        LABELNUM=a
        LABELNUM=$(mccli backup show --domain=$DOMAIN --name=$SERVER | grep -E "CST|CDT" | awk '{print $4}' | sort -rn | head -1)
        if [[ $LABELNUM -gt 0 ]]
        then
                let ESXCOUNTER=$ESXCOUNTER+1
                let DSCOUNTER=$DSCOUNTER+1
                mccli backup restore --name=$SERVER --domain=$DOMAIN --plugin=3016 --labelNum=$LABELNUM --restore-vm-to=new --virtual-center-name=$TGTVCENTER --datacenter=$TGTDC --folder=$TGTFOLDER --dest-client-name=${SHORTSERVER} --esx-host-name=DRVMhost$ESXCOUNTER --datastore-name=vol$DSCOUNTER
                if [[ $? -eq 0 ]]
                then
                        echo "SUCCESS: $SERVER Restore started" >> /home/admin/vmware_restore_script/vmrestore.log
                else
                        echo "ERROR: $SERVER mccli restore command failed, mccli return code $?" >> /home/admin/vmware_restore_script/vmrestore.log
                        continue
                fi
        else
                echo "ERROR: Unable to lookup labelnum for " $SERVER >> /home/admin/vmware_restore_script/vmrestore.log
                continue
        fi
        if [[ $ESXCOUNTER -eq $ESXHOSTCOUNT ]]
        then
                ESXCOUNTER=0
        fi
        if [[ $DSCOUNTER -eq $DSCOUNT ]]
        then
                DSCOUNTER=0
        fi
done
echo "========================Script completed $(date)==============================" >> /home/admin/vmware_restore_script/vmrestore.log