VMware 6.0 Notes



How to generate crash dump for hung virtual machine

1. When the Guest OS is Hung or unresponsive, Login to the vCenter Server or ESXi host using vSphere Client or vSphere Web Client

2. Take the Snapshot of the Virtual Machine which is hung or unresponsive state. Which will create .vmsn file in the virtual machine directory

[pic]

[pic]

Alternative to the snapshot, You can place the virtual Machine in Suspend State  by Right-Click the hung Virtual Machine -> Power -> Suspend. Which will create .vmss file in the virtual machine directory.

[pic]

3. Select the .vmsn or .vmss file and download to  your local machine where you are going to  save the Vmss2core utility.

4. Download the Vmss2core  from VMware Fling Site and place the downloaded file (vmss2core_win.exe)  in your local drive (C:\ or D:\). Ensure  your Windows machine is installed with Visual Studio 2008 Service Pack 1 runtime. Place both the Vmss2core file and downloaded .vmsn or .vmss file in same location.

c:\>vmss2core_win.exe -M VC-1-Snapshot3.vmsn

[pic]

What the basic troubleshooting steps in case of HA agent install failed on hosts in HA cluster?

1. Check for some network issues

2. Check the DNS is configured properly

3. Check the VMware HA agent status in ESXi host by using below commands

service vmware-aam status

4. Check the networks are properly configured and named exactly as other hosts in the cluster. otherwise, you will get the below errors while installing or reconfiguring HA agent.

5. Check HA related ports are open in firewall to allow for the communication

Incoming port: TCP/UDP 8042-8045

Outgoing port: TCP/UDP 2050-2250

6. First try to restart /stop/start the VMware HA agent on the affected host using the below commands. In addition u can also try to restart vpxa and management agent in the Host.

service vmware-aam restart

service vmware-aam stop

service vmware-aam start

7. Right Click the affected host and click on “Reconfigure for VMWare HA” to re-install the HA agent that particular host.

8. Remove the affected host from the cluster. Removing ESX host from the cluster will not be allowed untill that host is put into maintenance mode.

9.Alternative solution for 3 step is, Go to cluster settings and uncheck the VMware HA in to turnoff the HA in that cluster and re-enable the VMware HA to get the agent installed.

10. For further troubleshooting , review the HA logs under /Var/log/VMware/aam directory.

Troubleshooting purple screen of death.

When a PSOD occurs, the first thing you want to do is note the information displayed on the screen.

The PSOD message consists of the ESX version and build, the exception type, register dump, what was running on each CPU at the time of the crash, back-trace, server up-time, error messages and memory core dump info.

Once server reboot we need to find vmkernel-zdump in root directory

We can use vmkdump utility to extract the logs from vmkernel file

In ESXi we use below dump utility

esxcfg-dumppart -L vmkernel-zdump-filename

To extract the log file from a vmkernel-zdump file:

• Find the vmkernel-zdump file in the /root/ or /var/core/ directory:

# ls /root/vmkernel* /var/core/vmkernel*

/var/core/vmkernel-zdump-073108.09.16.1

• Use the vmkdump or esxcfg-dumppart utility to extract the log. For example:

# vmkdump -l /var/core/vmkernel-zdump-073108.09.16.1

created file vmkernel-log.1

# esxcfg-dumppart -L /var/core/vmkernel-zdump-073108.09.16.1

created file vmkernel-log.1

Deleting snapshot failed with error: A general system error occurred: concurrent access.

1. Veeam backup is attempting to access the Virtual disk at the same time.

Unable to delete the snapshot due to the locked file

a) Remove the vmdk file from backup appliance

b) Right-click the backup appliance virtual machine and click Edit Settings.

c) Check if the affected virtual machine's hard disk is mounted on the backup appliance virtual machine.

d) If the hard disk is mounted, select the hard disk and select the Remove from virtual machine option.

e) Warning: Do not select Delete option as it will result in data loss.

f) Create a new snapshot of the affected virtual machine and then Perform a Delete All from the snapshot manager to consolidate all snapshots

Adding an ESXi host to vCenter Server fails with the error: Host cannot be added to the vCenter as there are not enough Virtual Center Agent Licenses

To add ESXi hosts to vCenter Server, you must license the ESXi hosts with:

✓ vSphere Essentials

✓ vSphere Standard,

✓ Enterprise

✓ Enterprise Plus

Unable to login to Vcenter server

1. By restarting Vcenter server(vpxa) agent able to login successfully.

VMware Virtual Center Server service fails to start with the error: Assert Failed

✓ Take a backup of the vCenter Server journal directory, located at C:/Program Data/VMWare/VirtualCenter Server/journal to an alternate location.

✓ Delete the contents in the folder.

✓ Start the VMware Virtual Center Server service.

Creating a quiesced snapshot in Windows 2008 and Windows 2008 R2 fails with the error: Snapshot guest failed

✓ Power off the virtual machine.

✓ Right-click the virtual machine and click Edit Settings.

✓ Click the Options tab.

✓ Click General > Configuration parameters.

✓ Set the snapshot.redoNotWithParent parameter to false. This ensures that the snapshots are stored in the same location as the virtual machine configuration files.

✓ Verify if the changes are applied.

Power on the virtual machine

What happens to the VMs if a standalone host is taken to maintenance mode?

a) In case of standalone servers , VMware recommends that VMs should be powered off before putting the server in maintenance mode

b) If we put the standalone host in maintenance mode without powering off the VMs, it will remain in the ‘entering maintenance mode’ state until the VMs are all shutdown

c) When all the VMs are powered down, the host status changes to ‘under maintenance’

ESXi host is in disconnected state.

• Try to ping the ESXi host, this ensure that host is reachable.

• Ensure you can login to the VM's remotely, this ensure that VM's in the hosts are working fine.

• Try to login to ESXi host using vsphere client, this ensure that hostd service running fine. if this step fail restart the hostd service by login into ESX host using putty.

• service hostd restart

• If above three steps are working fine then issue would be vpxa and vpxd services.

• First try to restart vpxa service in the host if that did not resolve the issue try restarting vCenter server service.

Vcenter service restarting generates vmotion tasks when DRS set to automatic

To place a cluster in manual mode:

1. Right-click the DRS cluster and click Edit Settings.

2. Click VMware DRS.

3. Select Manual and click OK.

Migration of VM in fully automated DRS cluster failed

This issue may occur if any virtual machines in the cluster are in Standby mode. 

 

To resolve this issue, power on or power off the virtual machines to remove them from Standby mode.

Performance issue on VM

o Delete old snapshots.

o Consolidate snapshots.

o Check and Upgrade VMware tools.

o Verify that you are running most recent version of VMware product.

o Review the virtual machine's virtual hardware settings and verify that you have provided enough resources to the virtual machine.

o Ensure that any antivirus software installed on the host is configured to exclude the virtual machine files from active scanning. Install antivirus software inside the virtual machine for proper virus protection

o Check the storage sub-system on the host and verify that it is configured for optimal performance.

o Verify that there are enough free resources on the host to satisfy the requirements of the virtual machine

o Disable the CPU power management features on the host.

o Verify that host networking issues are not impacting the performance of the virtual machine.

o Verify that the host operating system is working properly and is in a healthy state.

VM network connection issue.

• Ensure that the Port Group name(s) associated with the virtual machine's network adapter(s) exists in your vSwitch or Virtual Distributed Switch and is spelled correctly.

• Ensure that the virtual machine has no underlying issues with storage or it is not in resource contention, as this might result in networking issues with the virtual machine.

• Verify that the virtual network adapter is present and connected.

• Verify that the networking within the virtual machine's guest operating system is correct.

• Verify that the TCP/IP stack is functioning correctly.

• If this virtual machine was converted from a physical system, verify that there are no hidden network adapters present.

• Verify that the vSwitch has enough ports for the virtual machine.

• Verify that the virtual machine's IPSec configuration is configured correctly and that it is not corrupted.

• Verify that the virtual machine is configured with two vNICs to eliminate a NIC or a physical configuration issue. To isolate a possible issue:

VM does not power on when there is high CPU reservation

1. Right-click on the virtual machine and click Edit Settings.

2. On the Virtual Machine Properties dialog, click the Resources tab.

3. Select CPU from list under the Settings column.

4. Use the slider or the Reservation text box to lower the CPU reservation value. 

To resolve this issue from the service console:

1. Establish an SSH session to your ESX host

2. Log in as root.

3. Run the following command to list the location of the configuration files for the virtual machines registered on that ESX host:

vim-cmd /vmsvc/getallvms

 

4. Record the location of the .vmx file (configuration file) for the virtual machine you are troubleshooting.

/vmfs/volumes/46b2f3eb-ced4c7d8-c1d2-111122223333//.vmx

5. If the virtual machine is not registered on the ESX host and you want to search its configuration file, run the following command and press Enter:

find /vmfs -name "*.vmx”

6. Make a backup copy of the .vmx file.

7. Open the .vmx file in a text editor and locate the line:

sched.cpu.min = "3230"

 

8. The value identified in the quotes exceeds the amount of physical CPU for one core. Lower the reservation by changing this value.

9. Save the file and exit the editor.

10. Power on the virtual machine.

Powering on a virtual machine fails with the error: memoryAllocation.reservation

virtual machines in the cluster has a memory reservation that is greater than the memory limit set on that same virtual machine.

 To resolve this issue, check the memory reservations and memory limits for all virtual machines in the cluster to ensure that no virtual machine in the cluster has a memory reservation that is greater than its memory limit.

EVC mode mismatch causes Virtual Machine migration issues

Power off the virtual machine.

Right-click the virtual machine and click Edit Settings > Options > CPUID Mask > Advanced.

Click Reset All to Default to reset the CPUID Mask.

Click OK > OK, then power on the virtual machine.

The virtual machine now shows the correct EVC mode

Snapshot consolidation failed maximum consolidate retries was exceeded

To set the parameter snapshot.asyncConsolidate.forceSync to TRUE using the vSphere client:

✓ Shut down the virtual machine.

✓ Right-click the virtual machine and click Edit settings.

✓ Click the Options tab.

✓ Under Advanced, right-click General

✓ Click Configuration Parameters, then click Add Row.

✓ In the left pane, add this parameter:

✓ snapshot.asyncConsolidate.forceSync

✓ In the right pane, add this value:

✓ TRUE

✓ Click OK to save your change, and power on the virtual machine.

Taking a snapshot fails with the error: Failed to take a memory snapshot, since the virtual machine is configured with independent disks

To change the options for the virtual disk(s):

Open the vSphere Client.

Right-click the virtual machine and click Edit Settings.

Find the affected virtual disk(s) and deselect the Independent option.

Click OK to apply and save the changes to the virtual machine configuration.

Consolidating virtual machine snapshots fails with the error: Change tracking target file already exists

To move the CBT files and consolidate snapshots:

Navigate to the virtual machine folder using this command:

cd /vmfs/volumes/datastore/virtual_machine/

List the contents of the directory using the ls command and look for .ctk files.

Create a temporary directory for the CBT files.

For example:

mkdir temp

Move the CBT files to this directory with this command:

mv *-ctk.vmdk temp/

Run the snapshot consolidation again.

Unable to take a quiesced VMware snapshot of a virtual machine

When using the VSS or SYNC driver within a virtual machine, the tools attempt to quiesce all I/O, such as storage and memory, within the virtual machine.

Troubleshooting issues when creating or committing snapshots in VMware ESXi

➢ Verify that the snapshot information in the Snapshot Manager is correct

➢ Verify that the snapshot maximum file size is supported by the datastore

➢ Ensure that the virtual disk is of a file type that supports snapshots.

➢ Verify that the datastore is not full and that there is sufficient space to commit all the snapshots.

➢ Ensure that there are not more than 32 levels of snapshots.

➢ Ensure that virtual machine files are not locked by any process,

➢ Ensure that no third-party software, such as a virtual backup software, is currently accessing the snapshot files.

Virtual machines are orphaned after rebooting the ESXi host on which they reside

✓ Power off the virtual machine.

✓ Use vCenter Server to un-register the affected virtual machine from the inventory and register it back again.

✓ Power on the virtual machine

✓ Repeat this procedure for all affected virtual machines.

Cannot create quiesced snapshots if backup or system recovery applications are installed

Remove the VSS component from VMware tool.

Adding an existing virtual disk to a virtual machine with snapshots fails with the error: An internal error occurred in the vSphere Client

To prevent this issue from occurring, remove all snapshots before copying, moving, or performing a vMotion on the virtual machine.

From the ESX/ESXi host command-line, navigate to the folder where the virtual machine vmdk files reside and run this command to recreate the .vmdk file from the snapshot file:

# vmkfstools -i -.vmdk .vmdk

For example, for VM test1 with 1 snapshot, run the command:

# vmkfstools -i test1-00001.vmdk test1.vmdk

Unable to save snapshot file" Error Occurs When Taking a Snapshot

One possible reason that you might encounter this error is that you have attempted to take a snapshot and save it to a VMware VMFS partition that does not have enough free space to take the snapshot.

Performing a vMotion freezes at 78% with the error: msg.snapshot.powerOff.prompt

Power off the virtual machine on the ESXi host.

Make a backup of the .vmx file.

Locate and change this configuration parameter from:

snapshot. action = "prompt"

To:

snapshot. action = "keep"

Power on the virtual machine and attempt to vMotion again.

Creating a virtual machine snapshot fails with the error: The attempted operation cannot be performed in the current state (Powered Off)

This issue may occur if snapshot. Current in the .vmsd metadata file points to a non-existent snapshot UID. If snapshot.current points to a non-existant snapshot UID, attempting to create a snapshot causes the virtual machine to power off.

In the following example, snapshot. Current points to 38. This issue occurs if none of the snapshots have a UID of 38:

root@bshp020 my-vm1]# cat my-vm1.vmsd

snapshot.lastUID = "39"

snapshot.numSnapshots = "0"

snapshot.current = "38"

snapshot0.uid = "39"

snapshot0.filename = "my-vm1-Snapshot39.vmsn"

snapshot0.displayName = "Consolidate Helper"

snapshot0.description = "Helper snapshot for online consolidate."

snapshot0.createTimeHigh = "274900"

snapshot0.createTimeLow = "-753698745"

snapshot0.numDisks = "1"

snapshot0.disk0.fileName = "my-vm1.vmdk"

snapshot0.disk0.node = "scsi0:0"

snapshot.needConsolidate = "FALSE"

To resolve this issue, perform one of these options:

If the virtual machine does not have any snapshots, delete the existing .vmsd file. The file is recreated the next time a snapshot is created or the next time the virtual machine is powered ON.

If the virtual machine has existing snapshots:

Modify snapshot. Current in the .vmsd so that it points to an existing snapshot.

When the virtual machine is up and running, commit all snapshots, power off the virtual machine, then delete the virtual machine's .vmsd file. The file is recreated the next time a snapshot is created or the next time the virtual machine is powered ON.

Reverting a snapshot fails with a permission denied error

➢ Change the user permission to power user from Vcenter console.

Powering on the virtual machine with snapshots fails

This issue occurs if the last snapshot file of the virtual machine is corrupted.

To resolve this issue, create a clone of the disk using an earlier snapshot and replace the existing disk with the new disk.

✓ To replace the existing disk with a new cloned disk:

✓ Provision space on another datastore.

✓ Create a folder on the new datastore with the same name as the virtual machine

✓ Run this command to clone the disk from the previous snapshot:

✓ # vmkfstools -i /vmfs/volumes/datastore1/virtualmachine/virtualmachine-0000003.vmdk /vmfs/volumes/datastore2/virtualmachine/virtualmachine.vmdk

✓ Edit the virtual machine settings and remove the existing disk.

✓ Add the new disk to the virtual machine.

✓ Power on the virtual machine.

✓ Remove the old files in the original datastore.

✓ Perform a Storage vMotion to move the disk to the original datastore.

[pic]

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download