This morning, one vms backup failed to snapshot it, and vmwaretool was out of date. try to migration it to other host failed again. First thought was VMwaretool issue, upgrade it, reboot the machine, still not working. Here is exactly what was happened.

Symptoms

•When you attempt a snapshot for a Virtual machine using Snapshot Manager, or VMware Consolidated Backup, or a third party backup solution such as vRanger, you receive these errors in vCenter

Cannot create snapshot
Operation not allowed in current state

•On observing the hostd.log file on the host, you see these messages:

[2010-09-12 07:43:03.404 F64A8B90 error 'vm:/vmfs/volumes/4acc704a-2b59540f-116b-0017a477fc18/VMNAME/VMNAME.vmx'] Invalid transition requested (VM_STATE_ON_SHUTTING_DOWN -> VM_STATE_CREATE_SNAPSHOT): Invalid state

•The virtual machine, vCenter and the host appear to be running fine, and yet the error persists.

Resolution

This issue occurs if the vCenter management agents have stopped causing the snapshot attempt to move in to the ON_SHUTTING_DOWN state.

To resolve this issue restart the VMware agents:
A.Ensure that the Virtual machine startup / shutdown with host feature is disabled before this step.

1.In the inventory, display the host where the virtual machine is located.
2.Select the host and click the Configuration tab.
3.Click Virtual Machine Startup/Shutdown, and click Properties

4.un-Select Allow virtual machines to start and stop automatically with the system.

http://deinoscloud.wordpress.com/2010/05/17/vsphere-virtual-machine-startup-and-shutdown-behavior/

B.Restart the management agents on the host. For more information, see Restarting the Management agents on an ESX or ESXi Server (1003490).

To restart the management agents on ESXi:
1.Connect to the console of your ESXi host.
2.Press F2 to customize the system.
3.Login as root.
4.Using the Up/Down arrows navigate to Restart Management Agents.

Note: In ESXi 4.1 and ESXi 5.0, this option is available under Troubleshooting Options .

5.Press Enter.
6.Press F11 to restart the services.
7.When the service has been restarted, press Enter.
8.Press Esc to logout of the system.
Note: For further information on restarting the management service on an ESXi host, see Service mgmt-vmware restart may not restart hostd (1005566).

Restarting the Management agents on ESX
To restart the management agents on ESX host:
1.Log in to your ESX host as root from either an SSH session or directly from the console.
2.Type service mgmt-vmware restart.

Caution: Ensure Automatic Startup/Shutdown of virtual machines is disabled before running this command or you risk rebooting the virtual machines. For more information, see Restarting hostd (mgmt-vmware) on ESX Server Hosts Restarts Hosted Virtual Machines Where Virtual Machine Startup/Shutdown is Enabled (1003312) and Determining whether virtual machines are configured to autostart (1000163).

3.Press Enter.
4.Type service vmware-vpxa restart.
5.Press Enter.
6.Type logout and press Enter to disconnect from the ESX host.
If this process is successful, it appears as:
 
[root@server]# service mgmt-vmware restart
Stopping VMware ESX Server Management services:
VMware ESX Server Host Agent Watchdog [ OK ]
VMware ESX Server Host Agent [ OK ]
Starting VMware ESX Server Management services:
VMware ESX Server Host Agent (background) [ OK ]
Availability report startup (background) [ OK ]
[root@server]# service vmware-vpxa restart
Stopping vmware-vpxa: [ OK ]
Starting vmware-vpxa: [ OK ]
[root@server]#
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003490

C.On vCenter Server, restart the vCenter and Management Web services. For more information, see Stopping, starting, or restarting vCenter services (1003895).

To stop a vCenter Server service:

1.Log in as an administrator to the server that is running vCenter Server.
2.Click Start > Run.
3.Type services.msc
4.Press Enter.
5.From the list of services right-click on the service you wish to stop. For example, VMware VirtualCenter Server.
6.Click Stop and wait for the service to complete stopping.
Note: When you stop the VMware VirtualCenter Server service, all connected users are disconnected from their session.

Starting vCenter Server services
To start a vCenter Server service if it has stopped:

1.Log in as an administrator to the server that is running vCenter Server.
2.Click Start > Run.
3.Type services.msc
4.Press Enter.
5.From the list of services right-click on the service you wish to start. For example, VMware VirtualCenter Server.
6.Click Start and wait for the service startup to complete.
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003895

D.Attempt the snapshot again. 

E.The operation succeeds and the corresponding log entry for the snapshot attempt reflects a transition from ON to CREATE SNAPSHOT instead of ON_SHUTTING_DOWN.
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1027707

If in case you have ctk file exist within the VM, here is the other solution:
1 Shut down the VM, remove it from the inventory, (DON'T delete from disk -  this would probably also solve the snapshot issue, but definitely not in a way you would be happy with),
2 Delete the ctk file (this is used for changed block tracking and will be re-created during the next backup/snapshot)
3 then add the VM to the inventory again by right clicking the vmx file.
4 Once this is done, try to set the virtual CDROM dirve to "Client Device".