Thursday, February 22, 2018

How to reset iLO login password from within ESXi Shell

This is something could be useful if you are unable to recall the HP iLO login password.

Follow these steps to reset iLO password from esxi host
Create a file named reset_ilo_pwd.xml in tmp folder under root with the below info and add the new password in the password section:

<ribcl VERSION=”2.0″>
<login USER_LOGIN=”Administrator” PASSWORD=”password“>
<user_INFO MODE=”write”>
<mod_USER USER_LOGIN=”Administrator”>
<password value=”YOUR-NEW-PASSWORD”/>
</mod_USER>
</user_INFO>
</login>
</ribcl>

Alternatively you can create this file by copying these lines to notepad and then saving it as reset_ilo_pwd.xml file, now upload it to /tmp directory of your ESXi host using any ftp client like Winscp.

Now use the following command to commit the administrator password information from the file (reset_ilo_pwd.xml) to iLO using the below command

 
./hponcfg -f /tmp/reset_ilo_pwd.xml

This is it, now you should be able to login to iLO of your ESXi host using the password specified in the above xml file.

Additional Info: HPE offers support for the iLO features available on ProLiant servers with the HP Online iLO Configuration aka HPONCFG utility. HPONCFG can be used to set up and reconfigure the iLO (Integrated Lights-Out) management controller of a server.

HPONCFG is an online configuration tool used to set up and reconfigure iLO without requiring a reboot of the server operating system. The utility runs in a command-line mode and must be executed from an operating system command line on the local server. HPONCFG enables you to initially configure features exposed through the RBSU or iLO.
You can also use HPONCFG to reset the ILO from ESXi host and many other things.
Browse to /opt/hp/tools and execute the below command to reset the HP ILO configuration.

./hponcfg -r 

Before using HPONCFG, the iLO Management Interface Driver must be loaded on the server. HPONCFG displays a warning if the driver is not installed.
In case if its not already present on the server, you may download and install from following links: 
* HPE ESXi Utilities Offline Bundle for VMware vSphere 6.0 *

* HPE ESXi Utilities Offline Bundle for VMware vSphere 6.5 *


That's it... :)


Wednesday, February 7, 2018

How to reset HP iLO over ssh or remotely

Lately I faced this issue, where I was unable to login to iLO of one of my HP ProLiant server instead there was a ssh connection related error on login page.


I had seen this error during the iLO firmware upgrade in past but not any other time.

As I did upgrade this ProLiant server’s Firmware/Drivers last day using the latest HP Service Pack forProLiant (aka HP SPP) so, it had the latest iLO firmware ver 2.55 installed and I was sure everything was working as expected post SPP upgrade.

We tried few things to resolve the issue and eventually resting the iLO fixed the issue, as iLO web login was not available so we had to reset it via command line by connecting to iLO over ssh.

You would be able to access iLO over ssh only if you didn’t disable it during the initial iLO configuration (By default iLO ssh access is enabled).

The steps to reset the HP iLO over ssh are follows:-
  1. Open Putty (or any other ssh client) and connect to the iLO controller using its IP address or FQDN
  2. Login with iLO administrator credentails (or an account with equivalent rights)
  3. Once connected to iLO, issue the following command:
   cd /map1

Press Enter
  1. Then type
   Reset

Press Enter

Command and its output would be something like this on CLI:

hpiLO-> cd /map1

status=0
status_tag=COMMAND COMPLETED

hpiLO-> reset

status=0
status_tag=COMMAND COMPLETED

Resetting iLO.

CLI session stopped

The iLO controller will begin to reset itself. Resetting the controller fixes a lot of hung-up and freezes issues on the interface.

Note: In case of any issue, HPE always recommends upgrading the firmware on the iLO controller as the first thing and as it doesn’t require reboots or downtime so can be done anytime.

Additional resources:   

That's it... :)


Saturday, February 3, 2018

Extended LUN size but unable to increase the datastore size from vCenter

You might have seen this issue where in vCenter managed environment, you want to expand the datastore size and Storage team has confirmed that they have increased the respective LUN size and the same is reflecting in datastore properties under Device capacity however when you click on increase under Volume Properties, there is no extent available.


I had seen such issue in past but when adding additional extents, not when increasing datastore size.

vCenter Server calls a specific function to get all the available extents for that datastore. After getting extents, vCenter Server displays the extents as available if they meet these criteria filters:
  • LUNS are not used as datastores on that host or on any other host (with exceptions to force mounted volumes).
  • LUNS are not used as Raw Device Maps(RDMs) on that host or any other host.
vCenter Server applies these filters to stop any possible data corruption on already used LUNs by other host.
VMware recommends you to not to turn off the filters. Before making any changes to the LUN filters, contact the VMware Support team. 

When you connect directly to the ESX host, these filters do not exist and hence you can see the extents.
  
To work around this issue, you need to connect to the ESXi directly using 'root' credentials and then increase the volume size from there, once done then re-scan all ESxi hosts for storage. 
Once you do that the expended datastore would be visible to all other ESXi hosts as well. 

Related VMware KB article :- 1011754


That's it... :)


Conflicting VIBs error during ESXi upgrade using ISO image

You might have came across this error where, trying to upgrade the ESXi host using OEM provided Custom ESXi image or may be with VMware provided ESXi image and the upgrade got aborted with the CONFLICTING_VIBs ERROR.


In order to successfully complete the ESXi upgrade, one need to to do the following:

  • Connect to host over ssh using putty and run the following command find the exact name of vib

       # esxcli software vib list | grep “displayed conflicting vib”
  •  Once you identified the exact vib name, then just remove it by using following command

       #esxcli software vib remove -n “actual name of conflicting vib”

Note: Please do an impact analysis before you proceed with vib removal.

Now you boot up the host using ESXi ISO image and complete the upgrade (this time you shouldn’t see earlier previous conflicting vib error).


That's it... :)


Friday, January 5, 2018

Intel / AMD processor vulnerability: Meltdown-Spectre and VMware Esxi

Most of us would be aware about this by now...if not, there were serious security flaws named Meltdown and Spectre discovered in processors designed by Intel, AMD and ARM, these flaws could let attackers steal your sensitive data.

These flaws were discovered by security researchers at Google’s Project Zero in conjunction with academic and industry researchers from several countries. Combined they affect virtually every modern computer, including smartphones, tablets and PCs from all vendors and running almost any operating system like Windows, macOS and Linux etc.


The two ‘bugs’ stem from design flaws of microprocessors that have the potential to allow applications, malware, and JavaScript running in web browsers, to obtain information from the operating system kernel’s private memory areas.

So here you may think how would it affect the Vmware Esxi platform and the VMs running on it.

VMware has issued a Security Advisory (VMSA-2018-0002) for the same and according it, CPU data cache timing can be abused to efficiently leak information out of miss-speculated CPU execution, leading to (at worst) arbitrary virtual memory read vulnerabilities across local security boundaries in various contexts. (Speculative execution is an automatic and inherent CPU performance optimization used in all modern processors.) ESXi, Workstation and Fusion are vulnerable to Bounds Check Bypass and Branch Target Injection issues resulting from this vulnerability.

Result of exploitation may allow for information disclosure from one Virtual Machine to another Virtual Machine that is running on the same host.

To remediate the observed vulnerability (known variants of the Bounds Check Bypass and Branch Target Injection issues) in each in different versions of Esxi releases, one need to install a corresponding patch from the list.

VMware Patches for different  ESXi Versions:

  • ESXi 6.5 – ESXi650-201712101-SG,    There are new patches available,
  • ESXi 6.0 – ESXi600-201711101-SG,    Refer to following Advisory VMSA-2018-0004
  • ESXi 5.5 – ESXi550-201709101-SG *
* This patch has remediation against CVE-2017-5715 but not against CVE-2017-5753.

Downloads:  https://my.vmware.com/group/vmware/patch, Search with the patch name.

Whilst this will secure the risk of data leakage between virtual machines it will not mitigate against the risk of data leakage within individual virtual machines. To protect against this threat operating system specific security updates must be installed.
Microsoft has already released a patch on Jan 3rd, 2018 to fix this issue on systems running on Windows OSs.

Also Apply the applicable firmware update provided by your server/device manufacturer, Useful Link.

Note: It has been speculated that patching the flaws would cause performance hit. At this time, it’s still unclear what would be the degree of performance hit, currently the details available varying with the source of information.

Related Read: https://www.pcworld.com/article/3245606/security/intel-x86-cpu-kernel-bug-faq-how-it-affects-pc-mac.html
https://www.theverge.com/2018/1/4/16848976/how-to-protect-windows-pc-meltdown-security-flaw

That’s it…. 😊