The iDrac7 has some capabilities to update itself over the network. The easiest solution is probably using FTP and ftp.dell.com. However: This did not work for me. The server seems to have some problems at least when you are an iDrac. Logging in via workstation worked fine but it’s slow – maybe I was unlucky but I didn’t got it to work.
racadm didn’t worked:
/admin1-> racadm update -f Catalog.xml.gz -e ftp.dell.com/Catalog -t FTP ERROR: RAC1117 : Unable to perform the update from repository operation. Retry the command after two minutes. If the problem persists contact your service provider.
At first I thought about mirroring ftp.dell.com locally but the server seems far too huge (and slow).
The correct way to do it is Dell Repository Manager. While that likely solves all my problems it’s Windows only with no Linux alternative in sight. But I have no Windows around and I wanted something simple that you can somewhat automate…
Luckily a post on the linux-poweredge mailing list got the answer. Let’s see how far we come in making this work.
Getting the Updates via SUU ISO and sharing them with NFS on Linux
Turns out, all you need is the “Dell Server Update Utility, Windows 64 bit Format“. It’s a rather huge ISO (4,7GB) . I’ve never heard of that before and this looks like something to boot into and it upgrades your server firmware. But we have this Lifecycle Controller thing, that promises to do this without any boot order changes.
Downloading the ISO turned out to be rather slow and in the end introduced me to a new wundervoll tool called aria2 that supports multi-connections downloads.
# apt-get install aria2c
Downloading with 16 connections I got around 2-4Mbyte/s – before it crawled on 200-600kbit on a university machine that is well connected.
Here is the current link for the ISO.
Hopefully fast enough:
$ aria2c -x 16 https://downloads.dell.com/FOLDER03644175M/1/SUU-WIN64_16.04.200.19.ISO
Finally we have the ISO. The data we want is in the
repository folder and we can mount it via loop (or extract and pick updates) and share it via NFS (or CIFS/HTTP/TFTP) and finally we can point the iDrac7 updater to this share. I’m using NFS here.
# mkdir -p /mnt/iso # mount -t loop ./SUU-WIN64_16.04.200.19.ISO /mnt/iso
You have to install a NFS Server if you don’t already have one at hand.
# apt-get install nfs-kernel-server
Now we need to setup the share and configure access rights – here everyone has access and it’s read only. Good enough.
# /etc/exports /mnt/iso/repository *(ro,no_subtree_check)
Now we need to activate the new share:
# exportfs -a
Using the iDrac7 WebUI for installing updates
This seems to work very well. And the iDrac showed the updates almost instantly after checking for updates!. You can find the Idrac7 Firmware Update page below iDrac Settings in the menu. There you can also test if it works.
Settings that worked for me:
- Protocol: NFS
- IP Address: IP of share – Not sure if hostnames work here.
- Share name:
- Catalog Location:
/– By default it’s something like
- Catalog Name: I left it empty – you could put
Some quick look at the Updates itself: There is a
Catalog.xml file which seems to contain all the metadata and filenames. And compressed versions of the same file as well as
Catalog.xml.sign with a GPG-Signature – not sure where to get the Public Key for verification but it likely somewhere on the Dell website. I guess with some XML magic it’s possible to change something in
Catalog.xml and select updates but it doesn’t look like anyone ever anticipated that a human will ever look into it.
Some attempts trying to automating the download a new version of the SUU-ISO
Would be great if you could automate this somehow. There seems to be a RSS-Feed for Updates, but the items link only to the website of the driver download. But it’s something. I guess scraping dell.com could be an option but this looks not straight forward enough. Maybe there is some API but I couldn’t find anything. However even getting hands on the URL for the feed was not so simple. The servers from Dell were quite slow, guess I was unlucky – however here it is:
There I stopped. It’s probably possible to somehow extract something meaningful like the download URLs and checksums out of that page but it’s appears to be quite complex and will likely change. Following the RSS feed and manually downloading the file is good enough for me.
For updating to a new ISO just remount the loop-mount with the new ISO image:
# mount -t loop -o remount /mnt/iso ./SUU-WIN64_foobar.ISO
The iDrac7 web interface let’s you configure a lot of update related stuff, check the included documentation that hides below the help item. Testing the connection, checking for updates and installing them is quite straight forward. Setting up automatic updates and schedules as well as automatic reboot can be configured.
But if you have a lot of machines it’s not particularly pleasant and nobody got time for that. So something scriptable is required. Luckily it’s also possible to use
racadm for this. It’s also possible with other APIs. there is some documentation from Dell on the options. For Linux there are alternatives like Recite, a Python library from Dell that helps speaking WS-MAN with the server. Had no chance yet too look at it.
Applying the Updates with racadm
At least in theory it should work like this:
/admin1-> racadm update -f Catalog.xml -l 188.8.131.52:/mnt/iso/repository -t nfs RAC1118 : Update from repository operation has been initiated. Check the progress of the operation usi ng "racadm jobqueue view -i JID_623219728539
But this didn’t work always. I guess it’s my fault but I’m not sure where to look. The network is fine.
You can also just install a single update if you know the filename. Filesnames are on the support website as well as in
/admin1-> racadm update -f BIOS_MKCTM_WN64_2.5.2.EXE -l 184.108.40.206:/mnt/iso/repository .......Copying completed. RAC987: Firmware update job for BIOS_MKCTM_WN64_2.5.2.EXE is initiated. This firmware update job may take several minutes to complete depending on the component or firmware being updated. To view the progress of the job, use the "racadm jobqueue view" command.
Let the iDracs search and install new updates automatically
Time to configure all servers for automatic updates. Let’s check every month for updates and install them but don’t reboot the machine for updates! So critical updates are installed only on next reboot. Be careful the setting
-a 1 enables automatic reboots after updates. This is probably pretty unpleasant if you are not okay with random reboots.
/admin1-> racadm autoupdatescheduler create -l 220.127.116.11:/mnt/iso/repositories -time 22:30 -dom 1 -rp 12 RAC1041: Successfully configured the Automatic Update (autoupdate) feature settings.
/admin1-> racadm jobqueue view [Job ID=JID_623199511859] Job Name=Repository Update Status=Scheduled Start Time=[Next Reboot] Expiration Time=[Wed, 01 Jun 2016 22:35:00] Message=[RED061: The job is successfully scheduled.] ----------------------------------------------------------
So that’s it. But a word of warning: I basically have not much experience working with
racadm so be careful and verify with the official docs. There are likely easier ways to this with only CLI tools on Linux. Please tell me :)
Finally here is the help output from
racadm help autoupdatescheduler that is very useful for scheduling:
/admin1-> racadm help autoupdatescheduler autoupdate scheduler -- Automatic Platform Update of the devices on the server. ------------------------------------------------------------------------------ Valid options: -u : Username of the remote share that stores the Catalog file. In case of CIFS, domain name to be specified as domain/username. -p : Password of the remote share that stores the Catalog file. -l : Network share location that has the Catalog file. For more information on NFS or CIFS share, refer usage examples. -f : Catalog filename. If not specified, catalog.xml will be default. -ph : FTP proxy hostname. -pu : FTP proxy username. -pp : FTP proxy password. -po : FTP proxy port. -pt : FTP proxy type. -time : Time to schedule an autoupdate in HH:MM format. This option has to be specified. -dom : Day of Month to schedule an autoupdate. Valid values are 1-28, L(Last day) or '*' (default - any day). -wom : Week of Month to schedule an autoupdate. Valid values are 1-4, L(Last week) or '*' (default - any week). -dow : Day of Week to schedule an autoupdate. Valid values are sun-sat or mon,tue or '*' (default - any day). NOTE: The -dom, -wom, or -dow options should be included in the command for the autoupdate schedule. if -dom option is included in the command, then -wom and -dow options should not be included. if -wom option is included in the command, then only -dow option should be included. if -dom or -wom option is non-'*', then the autoupdate is scheduled for every month. if -dom and -wom options are '*'and -dow option is non-'*', then the autoupdate is scheduled for every week. if -dom, -wom, and -dow options are '*', then the autoupdate is scheduled to repeat each day. -rp : Repeat option has to be specified. if -dom is specified, then valid values for -rp are 1-12. if -wom is specified, then valid values for -rp are 1-52. if -dow is specified, then valid values for -rp are 1-366. -a : Apply Reboot(1-Yes, 0-No). This option has to be specified. ------------------------------------------------------------------------------ Usage examples: - To Enable/Disable autoupdate feature : racadm set lifecyclecontroller.lcattributes.autoupdate 1 racadm set lifecyclecontroller.lcattributes.autoupdate 0 - To configure autoupdate feature settings: - In case of CIFS, run the command: racadm autoupdatescheduler create -u Americas/admin -p pwd -l //18.104.22.168/share -f cat.xml -time 14:30 -wom 1 -dow sun -rp 1 -a 1 - In case of NFS, run the command: racadm autoupdatescheduler create -u nfsadmin -p nfspwd -l 22.214.171.124:/share -f cat.xml -time 14:30 -dom 1 -rp 5 -a 1 - In case of FTP, run the command: racadm autoupdatescheduler create -u ftpuser -p ftppwd -l ftp.test.com -f cat.xml.gz -ph 10.20.30.40 -pu padmin -pp ppwd -po 8080 -pt http -time 14:30 -dom 1 -rp 5 -a 1 - To view the autoupdate feature settings, run the command: racadm autoupdatescheduler view - To clear the autoupdate feature settings, run the command: racadm autoupdatescheduler clear For Help on configuring the properties of a group - racadm help set -----------------------------------------------------------------------