Qwinsta Rwinsta - How to find Remote Desktop (Terminal Services) Sessions

I recently had to RDP into a Windows 2003 server to make a quick change.  Unfortunately, i was met with the old "exceeded the maximum connections" prompt:







To find out who's logged in, run the following command:
 qwinsta /server:<server name or IP address>

To disconnect the session run the following command:
rwinsta <ID> /server:<server name or IP address>

Using the output of the previous command, insert the <ID> of the user you wish to disconnect

vCenter Update Manager - Check New Notifications - Queued

I recently had HEAPS of Check new notifications tasks in the Queued state.

To resolve this, just restart the VMware vCenter Update Manager Service.

 

That's it.  The task completed successfully the next run.

VMware Converter Standalone 5.0 - Slow transfer rates

I've been noticing drastically slower conversion times using Converter 5.0 vs 4.3.  It turns out, Converter 5.0 encrypts the data stream using SSL by default.   This increased the security, but decreased transfer performance.  If SSL is not required, it can be disabled quite easily.

After installing Converter 5.0, locate the converter-worker.xml file: 

Windows 7 and Windows Server 2008 (R2):
C:\ProgramData\VMware\VMware vCenter Converter Standalone 
 
Windows XP and Windows Server 2003:
%ALLUSERSPROFILE%\VMware\VMware vCenter Converter Standalone

Make a copy of the converter-worker.xml file for safe keeping.  Then use Notepad to make the following change:

<nfc>
 <readTimeoutMs>120000</readTimeoutMs>
<useSsl>true</useSsl>
<!-- Delay is specified in milliseconds, -1 denotes the default.

TO:

<nfc>
<readTimeoutMs>120000</readTimeoutMs>
<useSsl>false</useSsl>
<!-- Delay is specified in milliseconds, -1 denotes the default.

Save the file, then restart the VMware vCenter Converter Standalone Worker service.  Enjoy the new found speed!

VMWare Converter: Converted VM fails to Boot

Recently, I performed a P2V of a Windows 7 laptop.  The destination was an ESX 4.1 host.  Per best practices,  I installed VMware Converter 5.0 within the source machine.  I used the Convert Machine wizard, and took the defaults to minimize the complexity of the conversion.  The conversion completed successfully. 

However, upon start up, I was "greeted" with a Blue Screen and later the following option:

I chose the repair option and patiently waited over an hour...

Upon closer inspection, I noticed the boot disk was setup for IDE.  The disk must be converted from IDE to SCSI for best performance.  In addition, the VM may fail to boot because the guest OS does not support the driver.  I ssh'd into the ESX host and used vi to open up the VMs .vmdk file.  I then changed the adapter from IDE to lsilogic by changing the following:

ddb.geometry.sectors = "63"

ddb.adapterType = "ide"

ddb.geometry.biosCylinders = "19457"

 

TO...

ddb.geometry.sectors = "63"

ddb.adapterType = "lsilogic"

ddb.geometry.biosCylinders = "19457"

 

I then removed the Hard disk (IDE (0:0)) from the virtual machine. (DO NOT Delete the disk)  I then added the Hard Disk back in using SCSI (0:0).   The LSI Logic SAS controller was added automatically.

After that change, the VM came up fine.  I just had to perform the usual post p2v clean up.

The CD-Rom complained about the IDE config being incorrect, but a quick Remove and Re-add took care of that.

How to kill a hung Windows Service

Yesterday, I was unable to stop the  vCenter Update Manager service.  It was hung at "stopping".

To kill the service, get the name of the offending service from Server Manager. In my case, it was vmware-ufad-vci:

Open a command prompt and get the PID for the hung service by running the following command:

sc queryex <service name>

Then kill the appropriate PID:
taskkill /PID <PID> /F

That's it!

Dell ExtPart.exe: Extending your C:\ Drive

I’ve been getting quite a few requests to extend the system partition on older Windows 2003 VMs. Over time, all of the Microsoft patches, application data etc, have filled up these drives.

This is not a problem with Windows 2008s native “Extend Volume” function. However, for Windows 2003 boxes, I’ve been using Dell ExtPart to extend these partitions. This process can be done on the fly with no down time. (Although, I typically perform a reboot to confirm it comes back up)

Dell Extpart can be located here: http://www.dell.com/support/drivers/us/en/04/DriverDetails?DriverId=R64398&FileId=2731129714&DriverName=Dell%20Basic%20Disk%20Expansion%2C%20v.1.0.4%2C%20A01

Confirm you have a good backup prior to proceeding!

Steps:
1. Confirm you have enough disk space to accommodate the increase in disk size in your datastore.  Increase the size of the hard drive you wish to extend from the vSphere Client.
2. Copy extpart.exe. to the temp directory. Extract.
3. Confirm that the newly allocated space is available in Disk Management.
4. From the command prompt, run extpart.exe.  Select the volume you wish to extend, and the size you wish to increase it by.  In this example we're extending the drive by 5GB (5x1024 = 5120)

5. Confirm the volume has been extended to the appropriate size.
6. Reboot the server to confirm that it comes back up clean.

That's IT!

VMware vCenter Server Appliance: Error: VPXD must be stopped to perform this operation.

I've been playing with the VMware vCenter Server Appliance and stumbled across the following error while setting up the vCenter Database:

Error: VPXD must be stopped to perform this operation.

To resolve this issue, ssh into the vCenter Server Appliance and run the following command:

vCenter Server Appliance:~ # chkconfig
after.local              off
apache2                  off
atftpd                   off
auditd                   on
.
### actual output cut..
.
.
vmware-inventoryservice  off
vmware-netdumper         on
vmware-rbd-watchdog      off
vmware-sps               off
vmware-tools             on
vmware-vpxd              on
vsphere-client           on
xinetd                   off
ypbind                   off

Confirm that the vmware-vpxd service is "on".  Then, run the following:
vCenter Server Appliance:~ # chkconfig vmware-vpxd off

Perform the needed function, then run the following to restart the vpxd service:
vCenter Server Appliance:~ # chkconfig vmware-vpxd on

Slow Read Performance from Storage Arrays - Delayed Ack

I recently stood up an ESX host on 10GbE.   The ESX host is using the Cisco Nexus 1000v connected to a Nexus 5K.  Storage is an Equallogic PS6010, also using 10GbE.

I ran IOMeter tests against this new setup and was disappointed with the poor read performance.  For consistency, the OpenPerformanceTest.icf config file from the following post was used:

http://communities.vmware.com/thread/73745

The IOPs for the Max Throughput -100%Read test was showing 584.   That's considerably slower than our existing production environment using a 1GbE infrastructure.

To resolve this issue, DelayedAck was disabled on the ESX host. TCP Delayed Ack groups together several ACK responses into a single reply. Delayed Ack can be disabled at the Discovery Address level (preferred), OR globally for the vmhba used for iSCSI.

Discovery Address Level - Select the iSCSI server under the Dynamic Discovery tab for your iSCSI vmhba.  Then, uncheck Inherit from Parent, and DelayedAck.  Reboot Host.
Globally - Uncheck DelayedAck in Advanced Setting for the vmhba used for iscsi. Reboot Host.

After making this change, I ran the same test and and was pleasantly surprised by an IOPs score of 22,354!!

Slow SSH Login - Fix

I was having slow login times on one of my newly built ESX hosts.  Other hosts on the same network were logging in without delay.  However, this particular host was taking 10-15 seconds.

In my particular case, it was a DNS configuration issue on the host.  Confirm that both the Preferred and Alternate DNS servers are still responding to requests.

After entering valid DNS Server IPs, I was getting lighting fast logins.

Error 15: Could not find file

After patching one of my ESX hosts, it became inaccessible after the reboot.  I DRAC'd into the server and found the following error:

Error 15: Could not find file

running the vdf -ha command showed that /boot was nearly full.  This host was upgraded from ESX 3.5 to 4.1 and was never "cleaned up"

Go into troubleshooting mode and run the following command to removed the old ESX 3.5 files:

cleanup-esx3 –f

The ESX host was then rebooted, and came back up without incident. 

The .dvsData folder

After deploying the Cisco Nexus 1000v and moving VMs onto it, I noticed a new folder called .dvsData in each of the datastores.  

Within the .dvsData folder are files with information regarding the port state of each VM using a dvport on the Cisco Nexus 1000v.  The .dvsData folder is located on the same datastore that the VM is registered.  In the event of a VMware High Availability (HA) event, the destination ESX host pulls the dvPort state of the VM from the .dvsData folder.

In addition, each ESX host, has a local database that contains vDS information.  (/etc/vmware/dvsdata.db) This allows the vDS to continue to run in the event vCenter is unavailable.

When deleting the .dvsData folder, confirm there are no registered VMs within the datastore.  Then proceed with the maintenance.

Configuring the Cisco Discovery Protocol (CDP) on ESX

By default, CDP on ESX(i) 3.5-5.0 is set to listen.  This allows ESX to read CDP information from attached Cisco Devices.  To listen and advertise vSwitch information,  the following must be performed on the ESX host.

1. Gather the current vSwitch configuration on the host by running the following command:

esxcfg-vswitch -l

2. To set the CDP status for a vSwitch, run the following.  Use the appropriate vswitch name for your environment:

esxcfg-vswitch -B both vSwitch1

3. Confirm the change by running the following:
esxcfg-vswitch -b vSwitch1

**Update Feb 15th 2013**
The esxcli command to configure CDP on an ESXi host with a standard switch is:
esxcli network vswitch standard set -c both -v vSwitch0

**Update Jan 27th 2017**
Commands for setting up CDP on a standard switch through powerCLI

$esxcli = Get-EsxCli -VMHost myhost.mycompany.com

$esxcli.network.vswitch.standard.set("both","9000","vSwitch1")

Confirm by running the following:

$esxcli.network.vswitch.standard.list()

EqualLogic Multipathing Extension Module (MEM)

For those of you using Equallogic Arrays and VMware Software iSCSI, I highly recommend the EqualLogic Multipathing Extension Module (MEM). 

The standard method of configuring SW iSCSI with 2 or more adapters is time consuming and tedious at best.  (create vSwitch, create vmkernels for each adapter, Override vSwitch Failover order, console in to set MTU to 9000, bind vmk's to the iSCSI initiator ect...)

The EqualLogic Multipathing Extension Module comes with a setup script which performs the configuration of SW iSCSI, in addition to installing the MEM. 

The SW iSCSI configuration is a breeze using the setup.pl script.  I ran it using vSphere CLI 4.1 from a local Windows host.   Run the script, enter in the host specific info (vmnics used, IP's, netmask, sw iscsi, etc..)  and the script does the rest.  It even enables sw iSCSI and enters the EqualLogic Group IP for you.

After the configuration, run the same setup script with "--install" to install the Equallogic Multipathing Extension Module.  After the successful install, and a reboot, I saw a noticable reduction in latency to the storage.

Unable to install the Cisco Nexus 1000V - Virtual Ethernet Module (VEM)

The Cisco Nexus 1000V is a software implementation of a Nexus switch.  It's comprised of 2 main components:

1. Virtual Supervisor Module (VSM) - In our environment, the VSM is made up of 2 VMs (Primary and Secondary)
2. Virtual Ethernet Module (VEM) - This component is installed onto each host.  It's similar to a linecard.

I was attempting to install the VEM using VMware Update Manager (VUM).  Unfortunately, I was unable to successfully complete the install,  it kept on erroring out. I confirmed that VUM was configured successfully, restarted the VUM service etc..

It turns out, VMware High Availability (HA) must be disabled to complete the VEM install.  After disabling HA at the cluster level,  I was able to successfully install the VEM on all 4 ESX hosts in this particular cluster.

How to change the VLAN ID of your ESX Service Console:

Need to change the VLAN ID of your Service Console?  Have you ever forgotten to enter the VLAN ID for your Service Console during the installation process?  If so, there's no need to reinstall, just console in and change it using the following commands.

Run the following command to see the current vSwitch configuration.  Make note of the virtual switch and the name of the Service console.  (typically vSwitch0 and "Service Console")
esxcfg-vswitch -l

Issue the following command to change the VLAN ID.   X is the desired VLAN:
esxcfg-vswitch vSwitch0 -v X -p "Service Console"

Verify the vSwtich configuration:
esxcfg-vswitch -l

Restart the network service:
service network restart