Jul 10

Configuring vSwitches From an SSH Session on an ESXi Host

Most of the time I work with SMB sized customers. Most of them don’t want or need to know the ends and outs of their vSwitch configuration (or much of the underlying configuration at all) because they don’t intend on needing to change it. If they want to modify something they will send in a support request and one of us at Eagle will reply to them.

Because most SMB shops only have maybe 3 ESXi hosts doing all of this via an SSH session into the ESXi shell tends to be quicker and provide better overall documentation for clients. It also helps with support if we ever have to reload a host. We don’t have to pull down any special Virtual Appliances or walk people though configuring a usb drive to do something special. It’s very clear and easy for folks to understand.

Keeping it simple also helps get rid of errors and makes things much more consistent.  This script takes advantage of variables which are kept in the head of the script so that things are made simpler and so you don’t have to go hunting for all the places to change things.

In this post I look at the basic configuration file I use to prepare clients ESXi hosts for use. This script has been optimized for vSphere 5.0. Much of it is not backwards compatible although I left the commands in for 4.1 hosts in most cases. It’s also worth noting that I don’t have the actual script file up for download. You’ll need to put the parts in a file yourself.

Before we get started I should make it clear if you use anything discussed in this blog it is at your your own risk.

It should first be noted that any thing preceded by a # symbol is a comment.

First we start off with the standard stuff that you would find in the header of the file…


################################################################### 
# 
# 10-23-11 Script created by Tony Foster 
# 10-28-11 Fixed some typos, moved jumbo frames for mgmt to end so 
# V.1.1      there is no disconect of the SSH session. Also added 
#            Syslog setup. 
# 12-11-11 Changed the iSCSI detection settings. In old script 
# V.1.2      it would detect hardware iSCSI HBA's as well as 
#            VMware's software HBA. 
# 12-14-11 Added a network refresh before changing network bindings 
# V.1.3      in Section 09.02. Otherwise it may fail to config the 
#            management network accordingly. 
# 05-08-12 Changed the default syslog to SSL port to 1514. Built a 
# V.1.4      vMotion vSwitch Config. Started work on createing a 
#            vLAN build option ran into snag with arrays. 
#            Also changed out the esxcfg to esxcli commands. 
# 06-21-12 V.1.4 did not work correctly re-writing to make it work. 
# V.1.5      Added vlan support and updated to new command set. 
# 
# 
# 
# Install Script for vSphere 5.0 
# Eagle Software, Inc V.1.5 
# 
# (C) Copyright 2012 EAGLE Software, Inc. in Association with 
#   Anthony (Tony) Foster. All rights reserved. 
# 
###################################################################

This section contains all the variables. Comments are out to the side that describe what they do.  I strongly recommend you just change the values of the variables to match what you want them to do.


#################Document what you are building#################### 
## For host:        <Put the host name here> 
## File build date:    <Put the build date here> 
## Pound signs (#) represent comments. 
## 
## Enter information in the variables below to configure your host. 

## Settings: 
#                        Management Settings
MyMgtVswitch="vSwitch0"                 #Management vSwitch name (Optional)
MgtNicOne='vmnic0'                      #Management Nic one's identifier
MgtNicTwo='vmnic4'                      #Management Nic two's identifier
MgtVMk='vmk0'                           #Management Kernel identifier
MyConsoleName='Service Console'         #Name for the connection to the ESXi host (Optional)
MyMgtNetName='Management Network'       #Name for the management network (Optional)
MyMgtVLAN='0'                           #vLAN for the management network leave at 0 for default (optional)
MyMgtMTU='9000'                         #MTU size for management 

/*#                        vMotion config
MyvMotVswitch="vSwitch0"                #Management vSwitch name (Optional)
MyvMotName='vMotion'                    #Name for base vMotion Kernel
vMotNicOne='vmnic0'                     #Managment Nic one's identifier 
vMotNicTwo='vmnic4'                     #Managment Nic two's identifier
MyvMotOneIP='x.x.x.x'                   #IP for vMotion1
MyvMotTwoIP='x.x.x.x'                   #IP for vMotion2
MyvMotSubNet='255.255.255.0'            #Subnet mask for vMotion
MyvMotVMkOne='vmk3'                     #First VM Kernel for vMotion (Optional)
MyvMotVMkTwo='vmk4'                     #Second VM Kernel for vMotion (Optional)
MyVMotVLAN='0'                          #vLAN for the vMotion network leave at 0 for default (optional)
MyvMotPortCount='16'                    #vMotion vSwitch Port count
MyvMotMTU='9000'                        #MTU size for vMotion 

/*#                        Production Settings
MyProdVswitch="vSwitch1"                #Production vSwitch name (Optional)
ProdNicOne='vmnic1'                     #Production Nic one's identifier
ProdNicTwo='vmnic5'                     #Production Nic two's identifier
MyProdNetName='Production Network'      #Production network name (Optional)
MyProdVlanNum="0"                       #Production vLAN id 0 for none (Optional) 
MyVlanBaseName="vLAN"                   #Base name for any additional vLANs created IE vLAN 10
MyVlanArray="0"                         #String array of additional vLANs to add each seperated by a space set to 0 for none
MyProdMTU='9000'                        #MTU size for Production 

#                        iSCSI Settings
MyiSCSIVswitch="vSwitch2"               #iSCSI vSwitch name (Optional)
MyiSCSIKernName='iSCSI'                 #iSCSI base kernel Name
iSCSINicOne='vmnic2'                    #iSCSI Nic one's identifier
iSCSINicTwo='vmnic6'                    #iSCSI Nic two's identifier
MyiSCSINetName='iSCSI Network'          #iSCSI network name (Optional)
MyiSCSIOneIP='y.y.y.y'                  #IP for iSCSI1
MyiSCSITwoIP='y.y.y.y'                  #IP for iSCSI2
MyiSCSISubNet='255.255.255.0'           #Subnet mask for vMotion
MyiSCSIVMkOne='vmk1'                    #First VM Kernel for iSCSI (Optional)
MyiSCSIVMkTwo='vmk2'                    #Second VM Kernel for iSCSI (Optional)
MyiSCSIVLAN='0'                         #vLAN for the iSCSI network leave at 0 for default (optional)
MyiSCSIMTU='9000'                       #MTU size for iSCSI 

#                        Host Settings
EditMyHostFile='0'                      #Set the value to 1 to edit the host file
MyDomainName=".domain.com"              #Domain name of the hosts
MyHostBaseName="ESXi0"                  #The base form of the host name
MyHostIPBase="x.x.x."                   #The first 3 octets of the hosts IP address
MyHostBaseIP="51"                       #The LAST octet of the ip, this will be the first host
MyTotalNumOfHosts='3'                   #The number of hosts you want to put into the file
MyStartingHostNum='1'                   #The first host to start with

#                        vCenter Settings
MyvCenterName="vCenter"                 #The short name of the vCenter
MyvCenterIP="x.x.x.x"                   #The IP address of the vCenter

#                        General Settings (designed for EqualLogic Arrays)
MyiSCSIArrayOneIP='x.x.x.x'             #IP address of iSCSI array
MyNTPServerOne='time.nist.gov'          #Time Server one
MyNTPServerTwo=''                       #Time Server Two (Optional)
MySyslogServer="ssl://x.x.x.x:1514"     #Syslog server name or IP : port SSL 1514
MySyslogLocPath='/scratch/log'          #Local path to save logs to (Optional)
MyDumpServerIP='x.x.x.x'                #Dump Server ip address
MyDumpServerVMK='vmk0'                  #Dump Server vmKernel number

This very first section is just a safety check to make sure you’re not running vMotion across the production network. If you need this explained as to why this is a bad idea you may want to consult some other documentation before continuing with this blog.


# Section Error Checking 
#Create vMotion vSwitch if needed
if [ $MyvMotVswitch == $MyProdVswitch ];                        
#Warn if vMotion and production are on the same network
then
clear screen
echo ""; echo ""; echo ""
echo "STOP!!! This is not a supported configuration DO NOT CONTINUE"
echo "Consult VMware for information"
echo ""; echo ""; echo ""
else
clear screen
echo ""; echo ""; echo ""
echo "NO ERRORS FOUND in the configuration"
echo ""; echo ""; echo ""
fi

Now we start on the meat of the script.  This first part changes the naming of the management kernel and the management network. The naming scheme has always driven me nuts. Don’t call it a network if there is only one thing is using it.


# Section 00 
# Clean up port settings
vim-cmd hostsvc/net/portgroup_set --portgroup-name="$MyConsoleName" $MyMgtVswitch "Management Network"
vim-cmd hostsvc/net/portgroup_set --portgroup-name="$MyMgtNetName" $MyMgtVswitch "VM Network"


#Reverse the decision 
#vim-cmd hostsvc/net/portgroup_set --portgroup-name="VM Network" $MyMgtVswitch "Management Network" 
#vim-cmd hostsvc/net/portgroup_set --portgroup-name="Management Network" $MyMgtVswitch "Service Console"

 

The next part starts by creating a vSwitch for all of the other network segments we need to create. You will also note that we are sizing the switches accordingly. To many ports and we are wasting space to few and we can cause problems for our design.


# Section 01.01 
# create a vSwitch 
# the -p represents how many ports the vSwitch has 
# 8 ports are held back for internal use, so subtract 8 to get usable ports 
# valid values are 16, 32, 64, 128, 248, 504, 1016, 2040, 4088 
esxcli network vswitch standard add -v $MyProdVswitch -P 128   
#create a Production vSwitch
esxcli network vswitch standard add -v $MyiSCSIVswitch -P 32   #create an iSCSI vSwitch
echo "*******************************************************"
#Modify vSwitch0
if [ $MyMgtVswitch != $MyProdVswitch ];                        #dont shrink the ports if Prod and Management vSwitchs are together
then
vim-cmd hostsvc/net/vswitch_setnumports $MyMgtVswitch 32
echo "Updated management network port count"
fi
echo "*******************************************************"
#Create vMotion vSwitch if needed
if [ $MyMgtVswitch != $MyvMotVswitch ];                        #dont create the vSwitch if Management and vMot vSwitchs are the same
then
esxcli network vswitch standard add -v $MyvMotVswitch -P $MyvMotPortCount 
echo "Created vMotion vSwitch"
fi

In the following section we get all of the parts setup for our vSwitchs. This includes things like jumbo frames, nics, CDP, and the like.


# Section 02.01 
# set vSwitchs to use jumbo frames 
#esxcfg-vswitch -m 9000 $MyMgtVswitch 
# 
esxcli network vswitch standard set -m $MyProdMTU -v $MyProdVswitch
esxcli network vswitch standard set -m $MyiSCSIMTU -v $MyiSCSIVswitch
#Enable Jumbo Frames on vMotion vSwitch if needed
echo "*******************************************************"
if [ $MyMgtVswitch != $MyvMotVswitch ];                        #dont enable unless the vSwitch for Management and vMot vSwitchs are diff
then
esxcli network vswitch standard set -m $MyvMotMTU -v $MyvMotVswitch
echo "Set frame size for vMotion"
fi


# Section 02.02 
# turn on CDP #esxcfg-vswitch  -B both $MyvMotVswitch 
esxcli network vswitch standard set -c both -v $MyMgtVswitch
esxcli network vswitch standard set -c both -v $MyProdVswitch
esxcli network vswitch standard set -c both -v $MyiSCSIVswitch
esxcli network vswitch standard set -c both -v $MyvMotVswitch   


# Section 02.03 
# add a nic to a vSwitch 
# esxcfg-vswitch -L $vMotNicTwo $MyvMotVswitch 
esxcli network vswitch standard uplink add -v $MyProdVswitch -u $ProdNicOne 
esxcli network vswitch standard uplink add -v $MyProdVswitch -u $ProdNicTwo 
esxcli network vswitch standard uplink add -v $MyiSCSIVswitch -u $iSCSINicOne
esxcli network vswitch standard uplink add -v $MyiSCSIVswitch -u $iSCSINicTwo 
#Add nics if needed to vMotion vSwitch
echo "*******************************************************"
if [ $MyMgtVswitch != $MyvMotVswitch ];                        #dont add nics unless Management and vMot vSwitchs are differant
then
esxcli network vswitch standard uplink add -v $MyvMotVswitch -u $vMotNicOne   
esxcli network vswitch standard uplink add -v $MyvMotVswitch -u $vMotNicTwo   
echo "Added nics to vMotion vSwitch"
fi


# Section 02.04 
# add nic's to default vSwitch
esxcli network vswitch standard policy failover set -a "$ProdNicOne,$ProdNicTwo" -v $MyProdVswitch
esxcli network vswitch standard policy failover set -a "$iSCSINicOne,$iSCSINicTwo" -v $MyiSCSIVswitch
esxcli network vswitch standard policy failover set -a "$MgtNicOne,$MgtNicTwo" -v $MyMgtVswitch
echo "*******************************************************"
if [ $MyMgtVswitch != $MyvMotVswitch ];                        #dont add nics unless Management and vMot vSwitchs are differant
then
esxcli network vswitch standard policy failover set -a "$vMotNicOne,$vMotNicTwo" -v $MyvMotVswitch
echo "Added nics to vMotion vSwitch"
fi

Section 3 lets you turn on flow control if your hosts don’t already do so.


# Section 03.01 
# Turn on flow control if neccesary (optional) 
# on by default in most cases 
#ethtool --pause vmnic0 tx on rx on 
#ethtool --pause vmnic1 tx on rx on 
#ethtool --pause vmnic2 tx on rx on 
#ethtool --pause vmnic3 tx on rx on 
#ethtool --pause vmnic4 tx on rx on 
#ethtool --pause vmnic5 tx on rx on 
#ethtool --pause vmnic6 tx on rx on 
#ethtool --pause vmnic7 tx on rx on

The following section starts building the useable components of our network specifically Port Groups.


# Section 04.01 
# build port groups 
#esxcfg-vswitch -A "iSCSI2" $MyiSCSIVswitch 
esxcli network vswitch standard portgroup add -p "$MyProdNetName" -v $MyProdVswitch
esxcli network vswitch standard portgroup add -p "$MyiSCSINetName" -v $MyiSCSIVswitch
esxcli network vswitch standard portgroup add -p "$MyiSCSINetName 1" -v $MyiSCSIVswitch
esxcli network vswitch standard portgroup add -p "$MyiSCSINetName 2" -v $MyiSCSIVswitch

esxcli network vswitch standard portgroup add -p "$MyiSCSIKernName""1" -v $MyiSCSIVswitch
esxcli network vswitch standard portgroup add -p "$MyiSCSIKernName""2" -v $MyiSCSIVswitch
esxcli network vswitch standard portgroup add -p "$MyvMotName""1" -v $MyvMotVswitch   
esxcli network vswitch standard portgroup add -p "$MyvMotName""2" -v $MyvMotVswitch

In the next section we start building our VMkernels


# Section 05.02 
# Set portgroups for a specific vLAN 
#esxcfg-vswitch -v $MyProdVlanNum -p "$MyProdNetName" $MyProdVswitch 
esxcli network vswitch standard portgroup set -p "$MyMgtNetName" -v $MyMgtVLAN
esxcli network vswitch standard portgroup set -p "$MyProdNetName" -v $MyProdVlanNum
esxcli network vswitch standard portgroup set -p "$MyiSCSINetName" -v $MyiSCSIVLAN
esxcli network vswitch standard portgroup set -p "$MyiSCSINetName 1" -v $MyiSCSIVLAN

esxcli network vswitch standard portgroup set -p "$MyiSCSINetName 2" -v $MyiSCSIVLAN
esxcli network vswitch standard portgroup set -p "$MyiSCSIKernName""1" -v  $MyiSCSIVLAN
esxcli network vswitch standard portgroup set -p "$MyiSCSIKernName""2" -v  $MyiSCSIVLAN
esxcli network vswitch standard portgroup set -p "$MyvMotName""1" -v $MyVMotVLAN
esxcli network vswitch standard portgroup set -p "$MyvMotName""2" -v $MyVMotVLAN
sleep 5


# Section 05.03 
# Build Production vLANs 
#MyProdVswitch="vSwitch1"                #Production vSwitch name (Optional) 
#MyVlanBaseName="vLAN"                   #Base name for any additional vLANs created IE vLAN 10 
#MyVlanArray="12 14 92 105 75"           #String array of additional vLANs to add each seperated by a space set to 0 for none 
if [ "$MyVlanArray" != "0" ]; then
for MyVlanProc in $MyVlanArray; do
esxcli network vswitch standard portgroup add -p "$MyVlanBaseName ${MyVlanProc}" -v $MyProdVswitch
esxcli network vswitch standard portgroup set -p "$MyVlanBaseName ${MyVlanProc}" -v ${MyVlanProc}
done
fi
sleep 5

Now we’ll configure our VMkernels


# Section 06.01 
# Convert port group to vmKernel with Jumbo Frames 
#MyMgtMTU='9000'                         #MTU size for management 
#MyvMotMTU='9000'                        #MTU size for vMotion 
#MyProdMTU='9000'                        #MTU size for Production 
#MyiSCSIMTU='9000'                       #MTU size for iSCSI 
#esxcfg-vmknic -a -i $MyiSCSIOneIP -n $MyiSCSISubNet -m 9000 "iSCSI1" 
esxcli network ip interface add -m $MyiSCSIMTU -p "$MyiSCSIKernName""1"
esxcli network ip interface add -m $MyiSCSIMTU -p "$MyiSCSIKernName""2"
esxcli network ip interface add -m $MyvMotMTU -p "$MyvMotName""1"
esxcli network ip interface add -m $MyvMotMTU -p "$MyvMotName""2"


# Section 06.02 
# set vmKernel IP 
#esxcfg-vmknic -a -i $MyiSCSIOneIP -n $MyiSCSISubNet -m 9000 "iSCSI1" 
esxcli network ip interface ipv4 set -I $MyiSCSIOneIP -N $MyiSCSISubNet -i $MyiSCSIVMkOne -t static
esxcli network ip interface ipv4 set -I $MyiSCSITwoIP -N $MyiSCSISubNet -i $MyiSCSIVMkTwo -t static
esxcli network ip interface ipv4 set -I $MyvMotOneIP -N $MyvMotSubNet -i $MyvMotVMkOne -t static
esxcli network ip interface ipv4 set -I $MyvMotTwoIP -N $MyvMotSubNet -i $MyvMotVMkTwo -t static

Now we’ll start configuring our iSCSI network.


# Section 07.01 
# create iSCSI adapter 
esxcli iscsi software set --enabled=true


# Section 07.02 
# Verify that its on. 
# Turn on iSCSI http://kb.vmware.com/kb/1029301 
#esxcfg-swiscsi -e #esxcfg-swiscsi -q
esxcli iscsi software get
sleep 10


# Section 07.03 
# Refresh the network
vim-cmd hostsvc/net/refresh
 
# Section 07.04 
# list vmhba#
echo `esxcli iscsi adapter list | grep vmhba | grep 'iSCSI Software Adapter' |awk '{print $1}'`


# Section 07.05 
# save it to a variable 
# Choose one of the two options, comment out the other 
MyHBA=$( esxcli iscsi adapter list | grep vmhba | grep 'iSCSI Software Adapter' |awk '{print $1}')


# Section 08.01 
# Add iSCSI array to iSCSI discovery 
vmkiscsi-tool -D -a $MyiSCSIArrayOneIP $MyHBA
sleep 5


# Section 09.01 
# Put nics in correct binding orders 
#iSCSI 
esxcli network vswitch standard portgroup policy failover set -u -p "$MyiSCSINetName"
esxcli network vswitch standard portgroup policy failover set -a "$iSCSINicOne" -s "" -p "$MyiSCSIKernName""1"
esxcli network vswitch standard portgroup policy failover set -a "$iSCSINicTwo" -s "" -p "$MyiSCSIKernName""2"
esxcli network vswitch standard portgroup policy failover set -a "$iSCSINicTwo" -s "$iSCSINicOne" -p "$MyiSCSINetName 1"
esxcli network vswitch standard portgroup policy failover set -a "$iSCSINicOne" -s "$iSCSINicTwo" -p "$MyiSCSINetName 2"
#Management
esxcli network vswitch standard portgroup policy failover set -u -p "$MyMgtNetName"
esxcli network vswitch standard portgroup policy failover set -a "$MgtNicOne" -s "$MgtNicTwo" -p "$MyConsoleName"
#vMotion
esxcli network vswitch standard portgroup policy failover set -a "$vMotNicOne" -s "$vMotNicTwo" -p "$MyvMotName""1"
esxcli network vswitch standard portgroup policy failover set -a "$vMotNicTwo" -s "$vMotNicOne" -p "$MyvMotName""2"
sleep 5


# Section 09.02 
# Refresh the network
vim-cmd hostsvc/net/refresh


# Section 11.01 
# bind VMkernels to VMhba’s
esxcli iscsi networkportal add -A $MyHBA -n $MyiSCSIVMkOne
esxcli iscsi networkportal add -A $MyHBA -n $MyiSCSIVMkTwo
sleep 5


# Section 12.01 
# Rescan for new storage
esxcli iscsi adapter discovery rediscover -A $MyHBA

I’ve left out the configuration rules for setting up an EqualLogic array. Its another post for another time. When I write it I’ll link to it here. Here we set our vMkernel for VMotion.


# Section 14.01 
# Set the VMotion kernel for vMotion http://kb.vmware.com/1006989 
vim-cmd hostsvc/vmotion/vnic_set $MyvMotVMkOne vim-cmd hostsvc/vmotion/vnic_set $MyvMotVMkTwo

Now we’ll set our NTP settings. For these changes to take affect you will want to restart the ESXi host so it will re-read its time servers.


# Section 15.01 
# Set the time servers for NTP

# for Windows TS see http://kb.vmware.com/kb/1318 for details

# found the orginal idea for this at http://zenhat.org/2010/09/11/how-to-sample-kickstart-file-for-vmware-esxi-4-1/
cat >> 
/etc/ntp.conf 
<< EOF
server $MyNTPServerOne
server $MyNTPServerTwo
EOF

chkconfig ntpd on

The next part configures syslogging and dump logging.


# Section 16.01 
# Configure the Syslog service for vCenter 
# List config
esxcli system syslog config get

# Set the Syslog
esxcli system syslog config set --logdir="$MySyslogLocPath" --loghost="$MySyslogServer" --logdir-unique=true --default-rotate=8 --default-size=1024

# Reload syslog
esxcli system syslog reload

# List config
esxcli system syslog config get


# Section 16.02 
# Enable Firewall entries 
# Open the firewall for syslog 
vim-cmd hostsvc/firewall_enable_ruleset syslog


# Section 17.01 
# Configure remote dump host 
# List current dump configuration 
esxcli system coredump network get

# Configure dump server connection
esxcli system coredump network set --interface-name $MyDumpServerVMK --server-ipv4 $MyDumpServerIP --server-port 6500

# Enable network core dump
esxcli system coredump network set --enable true

# List current dump configuration
esxcli system coredump network get

In some cases its beneficial to register the ESXi host’s in the host file. This can help in some instances where DNS is not available to respond to name resolution requests. We put entries in the ESXi hosts in the following section.


# Section 18.02 
# Register the ESXi Hosts in the /etc/hosts file
if [ $EditMyHostFile -eq 1 ]; then
while [ $MyStartingHostNum -le $MyTotalNumOfHosts ]; do
#echo "Counting up: $MyStartingHostNum"
echo -e "$MyHostIPBase""$MyHostBaseIP     $MyHostBaseName""$MyStartingHostNum      $MyHostBaseName""$MyStartingHostNum""$MyDomainName" >> /etc/hosts
MyHostBaseIP=`expr $MyHostBaseIP + 1`
MyStartingHostNum=`expr $MyStartingHostNum + 1`
done
echo -e "$MyvCenterIP     $MyvCenterName      $MyvCenterName""$MyDomainName" >> /etc/hosts
fi

Now we just have a couple of more things left to do. Firs we are going to put the host in maintinance mode so its ready for a reboot of the host.


# Section 19.01 
# put the host in maintenance mode for a reboot 
vim-cmd hostsvc/maintenance_mode_enter

Once that’s done we’ll take care of the management configuration. We don’t want to change this till we get to the end. In some instances I’ve seen networks get cranky when we flip over to jumbo frames and make other changes to the management network. That’s why this is done at the end of the script.


# Section 20.01 
# set the management vSwitch to use jumbo frames 
#MyMgtMTU='9000'                         #MTU size for management 
#MgtVMk='vmk0'                           #Management Kernel identifier 
esxcli network vswitch standard set -m $MyMgtMTU -v "$MyMgtVswitch"
esxcli network ip interface set -m $MyMgtMTU -i $MgtVMk

Now on to the very last thing we will do to our ESXi host. I expect that you didn’t type all of this stuff in while sitting at the console of your ESXi host… I bet you SSH’ed into it and either used winscp or putty with some copy and paste magic and dropped all of this code on to your system. Let seal up that loop hole so SSH is disabled. It should be noted that this is not permanent and you will probably want to go in and set a permanent rule for the SSH service through the gui.


# Section 21.01
# Turn off SSH
vim-cmd hostsvc/disable_ssh
vim-cmd hostsvc/stop_ssh
vim-cmd hostsvc/net/refresh
sleep 5

And that is a basic script minus some small parts that I use to build ESXi hosts in a hurry. I’m sure there are ways to improve it so that its more operationally efficient and I’m sure it could be ported over to a KS file or power-shell or any other form you want to use. You could probably get really inventive about how to deploy this quickly.

Remember this is for ESXi 5.0 hosts. If you run this on hosts prior to 5.0 you may get unpredictable results. Many of the section have the 4.x commands in the header if you need them however it should be noted that they are not complete and will not result in a full configuration if run.

If you have questions or comments please let me know and I will try my best to answer them for you.

 

 

 

 

 

 

 

Permanent link to this article: https://www.wondernerd.net/configuring-vswitches-from-an-ssh-session-to-an-esxi-host/

Jun 26

The wheels on Willy Wonka’s bus go round and crossways…

While driving  back to my hotel this evening from a clients site I had an interesting thought… Now I wonder how it can be applied to virtualization and storage…

Here’s the thought:

I grew up in a little town in Kansas and I knew all the back roads and roads that didn’t exist and all the cool places to visit. Yet while I’m in St. Louis I don’t know all of this information. I don’t have the slightest. I’m dependent on google, my cell phone, and my GPS to figure out where I’m heading. I bet if I were to put you in my home town you would be the same way. I guess you can think of this as relational unawareness.

This unawareness in relation to where we are is so natural that everyone does it without even thinking about it. If you don’t believe me, get lost! Literally. Drive or walk to somewhere you’ve never been before. Use a different path or road to get back to a place you know where you’re at without using technology. It’s not easy.

This is engrained in human nature, why can’t we create storage and virtual systems that are this way by default? Mutually autonomous systems. Discrete systems that while interacting with one another are oblivious to things farther away from themselves. Yet very knowledgeable about their own surroundings. Very similar to a heard or swarm but unique in the concept that it becomes aware of its surroundings.

So instead of knowing about all of the systems in my cloud the system only knows about the ones it interacts with regularly. This can be thought of as a school bus (not short bus jokes please), it has lots of stops to pickup youth and one destination. The driver is very aware of who’s supposed to get on and where as well as many other situational aspects surrounding the transport of children. Yet at the same time if you asked the bus driver to drive to a different town to pickup one child that situational awareness is lost. The bus driver becomes inefficient.

Why can’t we build storage and clouds with this sort of logic? Move our data in relation to how it interacts with other data, we’re not talking defraging, that’s linear, this is non-linear, like the Wonkavator in Charlie and the Chocolate Factory. It goes up and down, and sideways, and slant ways… you get the idea.

This same thing applies to our clouds… lets move them so that our services are floating around next to the things they use. This gets really cool when you start putting desktops and end user apps in the mix. What would our clouds look like? What becomes the center of the cloud. The users or the data? Would it be the same in all clouds? Would it stay the same in a cloud or would it change?

Imagine a hybrid cloud… where do the edges of the cloud meet? What do those edges look like. How do the services above or below or to the side impact that fuzzy gray area? When is it public cloud when is it private? The questions are absolutely endless.

You can say this nonsense or that’s how the cloud already is (it’s not). That’s fine and you are entitled to your opinions. As am I. And I will close with a quote from Roald Dahl – “A little nonsense now and then is relished by the wisest men.”

Think big, you may wind up with your head in the clouds!

Tony

Permanent link to this article: https://www.wondernerd.net/the-wheels-on-willy-wonkas-bus-go-round-and-round/

Jun 04

VeeamZIP Released Today

Today Veeam released something awesome! Veeam 6.1. Normally I don’t blog about products Eagle,  the company I work for, doesn’t currently sell. This is a special case. Veeam has released something that I am excited to talk about.

Rick Vanover  (@RickVanover) invited me to a sneak peak of Veeam 6.1 last Friday. During this peak I got to see many of the new features of Veeam 6.1. The one that really excited me was VeeamZIP.

What was the first part that excited me about VeeamZIP? The fact that its free! Yeup that’s right free. You get VeeamZIP as part of the free version of Veeam 6.1.

What else excited me about this release? Simply what you can do with VeeamZIP. What’s that? You can extract your VM’s from your virtual environment.

Why is that a big deal? You can do the following:

  • Archive your VM’s for later use
  • Move them via Sneaker Net from one site to another.
  • Distribute the VM’s quickly.

So what’s this mean for me?

It means I can do a lot. Now I can build VM’s for my clients at my facility and bring them to my clients on a usb device and they have very few hoops to jump through to use it.  I can grab a base copy of important VM configurations and save them off into an archive so if I need to reconstruct the system before my SE hosed an environment.

The possibilities are endless. Its worth checking out Veeam 6.1 just for VeeamZIP alone.

Hate to cut this post short but I’m at a clients site and its time to get the day started!

 

Cheers,

 

Tony

Permanent link to this article: https://www.wondernerd.net/veeamzip-released-today/

May 10

Storage in 3D or maybe 4D (multidimensional arrays)

4 Cubes

Last night I had this strange dream where I was literally arranging the interior walls of a hotel on a very high floor. I’d grab a wall and drag it into place then another one. For some reason it seemed like my task was to create the most efficient use of space. This got me pondering this morning.

Why not use Software Defined Storage in a new and unique way like a multidimensional array. Building off my earlier post of elevator to the cloud, it might be possible to make storage much more efficient. Instead of just two dimensional storage lets make it multidimensional.

If you look at how to most systems (and storage admins) work with storage its all in a 2D construct. There is a page and we can slide it around in a 2D plane. We can’t stack anything on top of it or define any additional dimensions to that page.  Its all stuck in a flat structure.

With the advent of cloud and software defined storage we are unchained from having to use flat constructs such as pages to build our data sets. Why because we’ve abstracted how we are storing data. We aren’t storing our data on a single disk device or storage array. Data can be anywhere. Accessed by different readers simultaneously or in sequence. And that read/write operation can happen along any axis or dimension in relatively any order.

My programming friends would probably recognize this c++ statement:

int storage[5][5][5][5];

The above statement defines a 4D array or 5^4 or 625 blocks. You can think of this if you like as being a cube 5 inches square with a time of 5 minutes (a forth dimension).

Now I can build data in a cloud to fit into each one of those blocks. I can then manipulate where those blocks are to form a highly efficient storage structure that represents anything I want.

I guess you could even pull out the calculus and define irregular shapes for your data (lets say a shape of a cloud). The only reason to use a non-cubed shape is if that structure made your data more efficient to use.

The one thing that I’ve noticed while writing this post is that it could parallel the movie the Matrix at some point. My only question at that point is who is the Architect? Paul Maritz or someone else?

Cheers,

Tony

Permanent link to this article: https://www.wondernerd.net/storage-in-3d-or-maybe-4d-multidimensional-arrays/

Apr 30

My VMware a Presentation to VMware

On Thursday April 26, 2012 Scott Bajtos of VMware & Tony Foster of Eagle Software presented at the VMware company wide “All Hands” meeting.  The company wide meeting was held at  VMware Corporate Head Quarters in Palo Alto and was attended by ~15,000 VMware employees both physically and virtually. Scott & Tony presented on the impact of My VMware on the global VMware ecosystem (VMware, partners, customers,  and end users). We talked about how My VMware makes the process of getting business done simpler and efficient.

Some interesting stats about the My VMware project:

  • It’s one of the biggest internal projects undertaken at VMware
  • 400K+ customer accounts created as part of the project.
  • 40K+ unique logins per day
  • It utilizes several VMware products:
    • vSphere 5
    • vCenter Ops
    • Spring Framework
    • vFabric Products

Tony, Eagle, and VMware are very excited about the launch of My VMware and the benefits it will bring to our customers.

Cheers,

Tony

Permanent link to this article: https://www.wondernerd.net/my-vmware-presentation/

Mar 27

Doomsday Preparation for companies

Time for a random (sort of) dream post around disaster recovery. This post is aimed at those companies that see themselves as preppers. It’s not for the individual home owner this is for the business that wants to avoid a dooms day scenario.

Many companies have disaster recovery (DR) plans in place they may even have a co-lo site that the replicate their servers to. Here’s a thought to take it to the level of a Doomsday Prepper.  I’m sure you’ve seen the TV show’s on Discovery Channel and Nat. Geo. These shows document people who are preparing for a major natural disaster to strike.

One of the things I’ve always wanted to do is buy a missile base. My mother was a historian back in my home town and was the first person to publish an article about Ed Peden and his 20th Century Castle in the Flint Hills Independent many years ago.  Since then Ed has been selling missile bases. You can check out his site here:  http://www.missilebases.com/

This evening one of my friends sent me an email saying they saw Ed on Doomsday Preppers on Nat. Geo. So I figured it would be a good time to type this up and share it with ya’ll.

So what’s this crazy idea? Simple, buy a missile base or communication bunker here in Kansas and put your DR site in something designed to withstand a nuclear missile. There are several cool things about this idea.

First cool thing is the cooling. Because the facility is mostly buried, the temperature stays at a much cooler than most other sites you will find. This means you get to save on cooling costs.

Next there are communication lines to these sites. Many of them have good communication lines running right up to the building. It’s not going to be that hard to drop an OC24 line into one of these sites so you have all the speed you need.

Another great thing about these sites is limited access and great physical security. This is especially true for anyone dealing with information that needs to be secure. With a several ton front door I don’t think anyone is going to just jimmy the door, walk in, and take something.

All of this is great but here’s the other cool thing (besides having a decommissioned missile base) is you can have plenty of room for your knowledge workers to go to in the event of a disaster. You could take this even further with virtualization.

If you use VDI and all of your servers are virtual… you’ve got instant recovery. All you have to do is get out your zero clients and run your SRM fail over plan. Once you do that everything is ready to go.

You can take something like this and even expand on it. You could offer the co-lo and let other companies replicate to you. If you really wanted you could turn the site into a training/conference center. Then have the ability to practice full blown DR plans.

That’s my crazy off the wall idea for this evening. So now your company can be a Doomsday Prepper too.

Cheers,

Tony

Permanent link to this article: https://www.wondernerd.net/doomsday-preparation-for-companies/

Mar 24

Moving a vCenter from one site to another with a virtual distributed switch

After 3 weeks of planning to post this I finally get to put the finishing touches on it and get it posted.

It’s midnight here and I’m talking with my wife on the phone after wrapping up my install for the day and my professional services for one of our clients this evening… Strangely enough most of today revolved around networking. This evening it was doing some network switching and routing with physical switches. Earlier today it was working with distributed virtual switches.

These virtual distributed switches (vDS) are the topics of my post today. Today I was at a site preparing for an SRM deployment next week. It’s a moderate size site (for what I normally work with anyway) with about 10 ESXi hosts at each site and a high speed link between production and DR. The way the site was originally setup DR and Production ESXi hosts were each in their own Data Center within the same vCenter. They have vDS for all of the networks in the environment.

My task today was to split the two sites into separate vCenters. Yesterday we updated all of the hosts to ESXi 5.0, spent a lot of time watching paint dry, and talking about the new features of vSphere 5. So for the most part this sounds like a pretty easy task to split a site and create a vCenter at production and DR. There are some caveats on this that I want to share with the rest of you though.

We started by building a new vCenter in the DR cluster, relatively straight forward, get all the pieces installed and operating, create a datacenter, cluster, and you’re ready to add hosts. We then put one of the ESXi hosts from the DR cluster in maintenance mode and then dropped it out of the pre-existing production vCenter and add it to our new DR vCenter. We then recreate the dVswitch in our new datacenter. All of this is still fairly mundane. One of the engineers I’m working with hasn’t had a lot of exposure to VMware so we spent some time doing things manually instead of with scripts.  (It’s the important knowledge transfer customers are after.)

All of this goes well, and we have some hosts in our cluster. It’s time to bring over the vCenter we created earlier in the day. If you were doing this on just standard vSwitches you would just shutdown the VM, browse the data store, and then bring it back up. No real issues there. Those who have spent time in datastores know this routine well, I’m sure (if not let me know and I’ll blog about it).

That won’t quite work with vDS’s and a vCenter… How do you move it over across a vDS in different vCenter’s? Let’s start with what happens when you shut down your DR vCenter, your vDS’s disappear from your list of networks you can plug VM’s into. Why are vDS’s not listed for your network adapters, you may ask? Because the vDS is a construct of the vCenter with hidden standard vSwitch’s on the ESXi hosts. This is why the vDS keeps working but you can’t change networking if the vCenter goes off line.

If you try to move the vCenter from one cluster to another you will probably see a message about unable to find any valid adapters when you click the drop down for your network adapters in your VM. What’s an IT director supposed to do? How do you get your vCenter into the cluster you just built?

To start, we will want to work with the vCenter where everything is working (in my case, on the production side with two clusters). With that done, pick one of the ESXi hosts that’s already in the cluster for the DR site. Now you want to drop one of your network adapters (You built for redundancy right?) from the vDS for your management network.  Then you are going to create a standard vSwitch. Call it temp and use the vNic you dropped off the vDS management switch.  Now shut down your DR vCenter running on the production side. Move it over to the DR ESXi host we just moved. Now edit settings on the VM and hook it to the temp vSwitch. Power on the DR vCenter. Edit settings for the DR vCenter on last time and flip the network over to the management vSwitch. Poof, you’re done, it’s moved. Now remove the temp vSwitch and add the vNic back into the management vDS.

For those who want the steps formatted a bit differently:

  1. Build your vCenter for the DR site on a DR ESXi host.
  2. Add a different ESXi host (We’ll call it ESXi2) to your DR vCenter that you just built
  3. Create your vDS for the DR environment (I’m assuming you’ve already done this otherwise you probably wouldn’t be reading this article).
  4. Remove 1 vNic from your DR management network (because you built it with at least 2 vNics).
  5. Create a temp vSwitch (if management has a vLAN make sure to tag it on the vSwitch).
  6. Shutdown the DR vCenter VM.
  7. Add it to the inventory on to ESXi2 by browsing the datastore on the ESXi host.
  8. Edit setting for the VM and change the network adapter to point at the Temp vSwitch.
  9. Power on the DR vCenter VM.
  10. It will come up and now you will see all your vDS’s become available.
  11. Edit settings for the DR vCenter VM and move the network connection back to the original Management vDS. (You may get an error message at this point, you should be able to click ok and continue on.)
  12. Now remove the temp vSwitch.
  13. Add the vNic from the previous step back into the management vSwitch (so you are redundant again).
  14. Then you’re done.

Now I know some of my friends, maybe even you, will say this is a hard way to move a VM around. Just create a standard vSwitch and then import it into a vDS. That’s actually not the point of this post. This is just an alternative way to skin the same cat. It’s not the easiest way to do it, it may not even be the proper way to do this, but it’s a way to do it.

Some of you will also say why would you put VMware management on a vDS. In vSphere 5 it’s recommended by VMware in their partner training (as of this writing). Those of us who have been around the block know the dangers of this and if you don’t you probably shouldn’t be considering a vDS for your management.

Anyway the above is one more way to move the vCenter when working with vDS.

Tony

Permanent link to this article: https://www.wondernerd.net/moving-a-vcenter-from-one-site-to-another-with-a-virtual-distributed-switch/

Feb 22

Lenten Traveling Tips…

Happy Ash Wednesday to all,

Today for Christians, we start the Lenten season (http://www.usccb.org/prayer-and-worship/liturgical-resources/lent/) with Ash Wednesday. This is a time of preparation. It’s about 40 days in length. This time is no coincidence, how long were the Israelites wandering in the desert when they fled from Egypt? How long did Christ spend in the desert? (The list goes on) the answer is 40 days!

Today I’m going to share some of my tips for traveling for work during the Lenten season, or just about any other time of year.

First: It’s Holy Week (the week prior to Easter, starting with Palm Sunday and Christ’s triumphant entry into Jerusalem and ending with the Easter Sunday) or it’s the weekend or another holy day of obligation. (BTW Ash Wednesday is not a Holy Day of Obligation in the USA but is the most attended non-holy day mass.) What to do? What to do? I’d recommend googleing around where you are working and see if there are any Catholic Churches nearby. Most parishes have websites now that list their mass schedule. Find out when mass is and make some time to go to church. Most of the time I find I can go to mass then go back to work and be more productive than I was before I left.

Second: Offer it up. There are several days that we as Catholics are called fast (Ash Wednesday). This isn’t the easiest thing when you have clients you are taking out for lunch or have a dinner meeting. We are also called to abstain from meat on some days, such as Friday’s during Lent. How can someone do this while taking care of their clients? I’d start by recommending that you look for sea food options on the menu, if none exist look for the vegetarian options.  Both are great alternatives. But what about fasting? Simple, order the small portion or only eat some of it and get the rest to go. And if you happen to be at a meeting where food is catered in and there are no non-meat selections politely decline and explain that you practice your faith and that during this time many don’t eat meat.

Third: What about the ashes on my forehead. I can’t go to a customer’s site like that. My first question is it bothering you or the customer? The ashes (or just about any other religious material) give you the opportunity to share your faith. My favorite reference to this is the song this little light of mine. Let it shine, let it shine. It can make things awkward but once you show you are willing to share about it I’ve found it makes things go so much smoother and you gain their trust. You will also often find that many others in the organization are wearing their ashes as well. (I strongly discourage this) If the ashes are such a distraction that neither you nor the client can work, wash them off. They are an outward symbol of your faith.

Those are three of my tips for traveling and fostering your faith. Now I’m off to mass.

 

Make it a happy day,

Tony

Permanent link to this article: https://www.wondernerd.net/lenten-traveling-tips/

Feb 16

Program the elevator to the cloud

I don’t know how many of you took programming in college (or were lucky enough to take it in high school). Do you remember having the task of programming elevators for optimal performance? You know given X elevators and Y floors you need to figure out the best place to put the elevators and the best way to move them around. I had this experience when I was learning C++.

Elevator

The storage nerds know that it could be compared to positioning heads on a set of hard drives. When I got back to my room today following VMware PEX it hit me. This will be coming back into play soon but probably not like many think. I foresee it coming into play with the cloud. It won’t be for storage anymore it will be efficiently positioning your work load in the cloud.

Where should my app be in the cloud to give my users what they need when they need it? I might have great performance in one place but because of other constraints my consumers of the cloud don’t get the benefits. However if I position the load in a slightly different location all the sudden my users will be able to gain immense performance benefits.

Why?  Because we moved the elevator to a better floor. It might not seem like this flys in what we have been taught. Right now everyone is racing to the cloud. They just want to get there. Right now it seems to me it’s who can provide the best spec’s leaving the cloud. Not who can provide the best end user experience from their cloud.

I think in 2 to 3 years this will be the position many are clouds are trying to find. The cloud that positions its workloads in relation to its consumers is the one that’s going to drive the market space. It will not be the one that may be fast but cannot effectively position its self to allow consumers to have the best experience.

So I predict that you will see a rebirth of this logic in the near future as cloud providers position themselves and their clouds so that even though it’s not the fastest cloud it’s the best cloud for what you need as a consumer of cloud services. Just ask Otis!

 

Make it a happy day,

Tony

Permanent link to this article: https://www.wondernerd.net/program-the-elevator-to-the-cloud/

Feb 16

Testing Centers at VMworld

So this is my first VMware blog post. I’ve had two major blocks to blogging. One I’ve not had time. Two working for a VAR a lot of the things I work on are on NDA or my employer wants to protect as they see the material as something of value which they can leverage. Take it or leave it that’s how it is.

For today’s topic I don’t have either of those constraints. VMware Partner Exchange (PEX) is over and my flight doesn’t leave till the morning. And my topic has nothing to do with any value proposition or bottom line.

What’s the topic about you might ask. I’m going to look at exam testing for VMware certs. Something that I felt was lacking at VMworld 2011 in Las Vegas last year was the testing. If you were there and wanted to take the opportunity to sit an exam you couldn’t because Pearson View did not have a testing center at VMworld 2011.

I found this an annoyance for two reasons. First I live in the middle of Kansas and if you look on a map you’ll see that there are limited testing centers that you can go to take a VMware exam. For me to take an exam I’d have to drive several hours to get to a testing center. The second part is finding the time to do it. I like many in the IT industry are busy. We can just say “ya know I’m going to go take my VCP on Monday.” It just doesn’t work for many of us. However VMworld is big and it’s been on the schedule for a while. It’s a perfect time to sit an exam.

I’m sure there are at least a few of you out there that would agree with this. So why are there no tests at VMworld? This week at PEX I got the chance to have lunch with a lot of VMware people. There is a very straight forward answer to this. Because it’s a hassle for both VMware and Pearson View, that and too few people were taking advantage of it.

Before someone says that’s not true it’s just that (insert company here) is not making money on it. There’s not a lot of money to be made by offering the exams at VMworld. VMware has to consume a room to put the testing center in it has to meet specific requirements. They also have guarantee a given number of tests are taken during the hours of operation. It’s not all that beneficial for Person View either. They have to divert staff and equipment to run a temporary testing center.

I bet someone out there is going to say, “way to tow the line Tony.” For those saying that I’d like to ask what certification(s) are you working on and where do you plan to do your testing? I have a feeling that many people stop at the VCP. This is probably because of where you work in the industry. Most people are not required to seek higher level certifications like the VCAP-DCD, VCAP-DCA, or the ever cherished VCDX. Why? Most likely because while the certifications are valuable they don’t help a lot of companies generate revenue so they don’t require it. The exception to this VMware Partners, it helps them distinguish themselves, as well as VMware having confidence in the certified person having the ability to perform with in the range of their certification.

For these reasons it makes a lot of sense that VMware doesn’t offer testing at VMworld any longer. I do want to give them props for offering it at PEX though. Why? Because this really gives partners who are busy the opportunity to fit it into their schedule. It also means that consumers are benefiting from more intelligent highly trained folks who have taken the time and put forth the effort to attend a partner event and show they know there stuff.

So in the end, even though it’s disappointing to some (myself included) that there is not a testing center at VMworld sometimes it’s not what works best for the individual its what’s best for the community.  If you don’t like that then think of it this way, there can be one less session at VMworld to help you do your job better because it’s being taken up by a testing center that only a few people are taking advantage of.

Make it a happy day,

Tony

For additional reference see my post on the VMworld 2011 site: http://www.vmworld.com/thread/5085

Permanent link to this article: https://www.wondernerd.net/testing-centers-at-vmworld-2/