One of the flex10 module failed and got replacement from HP. It taking a long time to update its firmware, detail story as below.

Our environment:
2 HP 7000 chassis in one domain for the virtual connector. (Primary chassis OA1 and secondary chassis OA2)

Issue:
Port 1 failed to communicate from OA2 interconnect Bay1. Called HP and have the flex10 module replaced.

The replacement come with firmware 2.2 and all our other bays running firmware 3.30. update is needed for the new replacement.

First Try (with HP vcsu V1.5.4):
Replaced the new one to OA2 bay1 and try update it, failed for the following reason:
[Thu Nov 10 14:47:35 2011] (7628)[VcsuCore.cpp:2092] : Primary VCM found in enclosure USE11695LX, bay 1
[Thu Nov 10 14:47:35 2011] (7628)[VcsuUpdate.cpp:1568] : Deleting FC firmware images from the Enet modules
[Thu Nov 10 14:47:46 2011] (7628)[VcsuUpdate.cpp:1579] : VCFC image deletion from bay 1 failed
[Thu Nov 10 14:47:56 2011] (7628)[Sftp.cpp:57] : Deleting remote file: /images/archives/vcfc2/SWBD82/v6.1.0_uth5
[Thu Nov 10 14:47:56 2011] (7628)[Sftp.cpp:63] : sftp_unlink failed, errorno = 0
[Thu Nov 10 14:47:56 2011] (7628)[Sftp.cpp:1115] : Delete file failed
[Thu Nov 10 14:47:56 2011] (7628)[VcsuUpdate.cpp:1604] : VCFC2 image deletion from bay 2 failed
[Thu Nov 10 14:47:56 2011] (7628)[VcsuProgress.cpp:156] : Starting Activation
[Thu Nov 10 14:47:56 2011] (7628)[VcsuProgress.cpp:796] : Activation order is ODD EVEN
[Thu Nov 10 14:47:56 2011] (7628)[VcsuProgress.cpp:801] : Activation for single bay update
[Thu Nov 10 14:47:56 2011] (7628)[VcsuProgress.cpp:819] : Primary bay number passed for -b option. Exiting activation process since update was not success
[Thu Nov 10 14:47:56 2011] (7628)[VcsuUpdate.cpp:653] : Firmware update failed on one or more modules

Second Try (with HP vcsu V1.6.0)
I thought that maybe is the version of vcsu is too old for this, so download the new one and try to update it again. Failed! Here is the log:
[Thu Nov 10 15:38:34 2011] (5836)[VcsuCore.cpp:1955] : Primary VCM found in enclosure USE11695LX, bay 1
[Thu Nov 10 15:38:44 2011] (5836)[VcsuSftp.cpp:563] : SFTP open failed for /images/archives/vcmConfig.tgz: errno = 11, Resource temporarily unavailable
[Thu Nov 10 15:38:44 2011] (5836)[VcsuCore.cpp:4143] : Download of VCM config file failed
[Thu Nov 10 15:38:44 2011] (5836)[VcsuInit.cpp:891] : Failed to read generation numbers from VCM
[Thu Nov 10 15:38:44 2011] (5836)[AuthenticateUser.cpp:135] : authenticateVCM failed

Third and fourth try was soft/hard rest the module and it still cannot update.

CAlled HP for support and been told that try to put the new module to the OA2 and the bay other than Bay1. it could be the Active Bay and cuase the update failed. ( what is that about ???) Anyway, take the word and give it fifth try. and this time even worse.
[Fri Nov 11 09:56:37 2011] (3492)[VcsuCore.cpp:2126] : Module not present in bay 8
[Fri Nov 11 09:56:38 2011] (3492)[VcsuCore.cpp:1146] : Domain configuration is present
[Fri Nov 11 09:56:38 2011] (3492)[VcsuCore.cpp:1146] : Domain configuration is present
[Fri Nov 11 09:56:39 2011] (3492)[VcsuCore.cpp:1146] : Domain configuration is present
[Fri Nov 11 09:56:40 2011] (3492)[VcsuCore.cpp:1146] : Domain configuration is present
[Fri Nov 11 09:56:41 2011] (3492)[VcsuCore.cpp:1176] : This is a subordinate enclosure
[Fri Nov 11 09:56:41 2011] (3492)[VcsuInit.cpp:744] : Primary OA is not in this enclosure. Getting primary OA IP
[Fri Nov 11 09:56:52 2011] (3492)[VcsuInit.cpp:765] : VCM user authenticated successfully
[Fri Nov 11 09:57:03 2011] (3492)[VcsuInit.cpp:778] : Unable to get primary OA IP

Well, I guess I need to make another call to HP, and it come out with this scenario:

Place the new module to the OA1 empty bay and try it. finally, it worked. But it something scary? what if I do not have empty space in the primary chassis? and also, the HP support would suggest to use vcsu 1.5.4 instead of 1.6.0.

Lesson learned from HP, it is not that smart for the chassis blade system and there is a potential to stop all service in this kind of environment. I wish they can have more stable and smart way to deal with this. maybe next 2 version of the firmware or vcsu can solve it. What a pain!!!