As I have worked with customers deploying Live Upgrade in their environments, several problems seem to surface over and over. With this blog article, I will try to collect these troubles, as well as suggest some workarounds. If this sounds like the beginnings of a Wiki, you would be right. At present, there is not enough material for one, so we will use this blog for the time being. I do expect new material to be posted on occasion, so if you wish to bookmark it for future reference, a permanent link can be found here.

Live Upgrade copies over ZFS root clone

This was introduced in Solaris 10 10/09 (u8) and the root of the problem is a duplicate entry in the source boot environments ICF configuration file. Prior to u8, a ZFS root file system was not included in /etc/vfstab, since the mount is implicit at boot time. Starting with u8, the root file system is included in /etc/vfstab, and when the boot environment is scanned to create the ICF file, a duplicate entry is recorded. Here's what the error looks like.

# lucreate -n s10u9-baseline
Checking GRUB menu...
System has findroot enabled GRUB
Analyzing system configuration.
Comparing source boot environment  file systems with the
file system(s) you specified for the new boot environment. Determining
which file systems should be in the new boot environment.
Updating boot environment description database on all BEs.
Updating system configuration files.
Creating configuration for boot environment .
Source boot environment is .
Creating boot environment .
Creating file systems on boot environment .
Creating  file system for  in zone  on .

The error indicator -----> /usr/lib/lu/lumkfs: test: unknown operator zfs

Populating file systems on boot environment .
Checking selection integrity.
Integrity check OK.
Populating contents of mount point .

This should not happen ------> Copying.

Ctrl-C and cleanup

If you weren't paying close attention, you might not even know this is an error. The symptoms are lucreate times that are way too long due to the extraneous copy, or the one that alerted me to the problem, the root file system is filling up - again thanks to a redundant copy.

This problem has already been identified and corrected, and a patch (121431-58 or later for x86, 121430-57 for SPARC) is available. Unfortunately, this patch has not yet made it into the Solaris 10 Recommended Patch Cluster. Applying the prerequisite patches from the latest cluster is a recommendation from the Live Upgrade Survival Guide blog, so an additional step will be required until the patch is included. Let's see how this works.

# patchadd -p | grep 121431
Patch: 121429-13 Obsoletes: Requires: 120236-01 121431-16 Incompatibles: Packages: SUNWluzone
Patch: 121431-54 Obsoletes: 121436-05 121438-02 Requires: Incompatibles: Packages: SUNWlucfg SUNWluu SUNWlur

# unzip 121431-58
# patchadd 121431-58
Validating patches...

Loading patches installed on the system...

Done!

Loading patches requested to install.

Done!

Checking patches that you specified for installation.

Done!


Approved patches will be installed in this order:

121431-58


Checking installed patches...
Executing prepatch script...
Installing patch packages...

Patch 121431-58 has been successfully installed.
See /var/sadm/patch/121431-58/log for details
Executing postpatch script...

Patch packages installed:
  SUNWlucfg
  SUNWlur
  SUNWluu

# lucreate -n s10u9-baseline
Checking GRUB menu...
System has findroot enabled GRUB
Analyzing system configuration.
INFORMATION: Unable to determine size or capacity of slice .
Comparing source boot environment  file systems with the
file system(s) you specified for the new boot environment. Determining
which file systems should be in the new boot environment.
INFORMATION: Unable to determine size or capacity of slice .
Updating boot environment description database on all BEs.
Updating system configuration files.
Creating configuration for boot environment .
Source boot environment is .
Creating boot environment .
Cloning file systems from boot environment  to create boot environment .
Creating snapshot for  on .
Creating clone for  on .
Setting canmount=noauto for  in zone  on .
Saving existing file  in top level dataset for BE  as //boot/grub/menu.lst.prev.
Saving existing file  in top level dataset for BE  as //boot/grub/menu.lst.prev.
Saving existing file  in top level dataset for BE  as //boot/grub/menu.lst.prev.
File  propagation successful
Copied GRUB menu from PBE to ABE
No entry for BE  in GRUB menu
Population of boot environment  successful.
Creation of boot environment  successful.

This time it took just a few seconds. A cursory examination of the offending ICF file (/etc/lu/ICF.3 in this case) shows that the duplicate root file system entry is now gone.

# cat /etc/lu/ICF.3
s10u8-baseline:-:/dev/zvol/dsk/panroot/swap:swap:8388608
s10u8-baseline:/:panroot/ROOT/s10u8-baseline:zfs:0
s10u8-baseline:/vbox:pandora/vbox:zfs:0
s10u8-baseline:/setup:pandora/setup:zfs:0
s10u8-baseline:/export:pandora/export:zfs:0
s10u8-baseline:/pandora:pandora:zfs:0
s10u8-baseline:/panroot:panroot:zfs:0
s10u8-baseline:/workshop:pandora/workshop:zfs:0
s10u8-baseline:/export/iso:pandora/iso:zfs:0
s10u8-baseline:/export/home:pandora/home:zfs:0
s10u8-baseline:/vbox/HardDisks:pandora/vbox/HardDisks:zfs:0
s10u8-baseline:/vbox/HardDisks/WinXP:pandora/vbox/HardDisks/WinXP:zfs:0

Solaris 10 9/10 introduces new autoregistration file

This one is actually mentioned in the Oracle Solaris 9/10 release notes. I know, I hate it when that happens too.

Here's what the "error" looks like.

# luupgrade -u -s /mnt -n s10u9-baseline

System has findroot enabled GRUB
No entry for BE  in GRUB menu
Copying failsafe kernel from media.
61364 blocks
miniroot filesystem is 
Mounting miniroot at 
ERROR:
        The auto registration file <> does not exist or incomplete.
        The auto registration file is mandatory for this upgrade.
        Use -k  argument along with luupgrade command.
        autoreg_file is path to auto registration information file.
        See sysidcfg(4) for a list of valid keywords for use in
        this file.

        The format of the file is as follows.

                oracle_user=xxxx
                oracle_pw=xxxx
                http_proxy_host=xxxx
                http_proxy_port=xxxx
                http_proxy_user=xxxx
                http_proxy_pw=xxxx

        For more details refer "Oracle Solaris 10 9/10 Installation
        Guide: Planning for Installation and Upgrade".

As with the previous problem, this is also easy to work around. Assuming that you don't want to use the auto-registration feature at upgrade time, create a file that contains just autoreg=disable and pass the filename on to luupgrade.

Here is an example.

# echo "autoreg=disable" > /var/tmp/no-autoreg
# luupgrade -u -s /mnt -k /var/tmp/no-autoreg -n s10u9-baseline
 
System has findroot enabled GRUB
No entry for BE  in GRUB menu
Copying failsafe kernel from media.
61364 blocks
miniroot filesystem is 
Mounting miniroot at 
#######################################################################
 NOTE: To improve products and services, Oracle Solaris communicates
 configuration data to Oracle after rebooting.

 You can register your version of Oracle Solaris to capture this data
 for your use, or the data is sent anonymously.

 For information about what configuration data is communicated and how
 to control this facility, see the Release Notes or
 www.oracle.com/goto/solarisautoreg.

 INFORMATION: After activated and booted into new BE ,
 Auto Registration happens automatically with the following Information

autoreg=disable
#######################################################################
Validating the contents of the media .
The media is a standard Solaris media.
The media contains an operating system upgrade image.
The media contains  version <10>.
Constructing upgrade profile to use.
Locating the operating system upgrade program.
Checking for existence of previously scheduled Live Upgrade requests.
Creating upgrade profile for BE .
Checking for GRUB menu on ABE .
Saving GRUB menu on ABE .
Checking for x86 boot partition on ABE.
Determining packages to install or upgrade for BE .
Performing the operating system upgrade of the BE .
CAUTION: Interrupting this process may leave the boot environment unstable
or unbootable.

The Live Upgrade operation now proceeds as expected. Once the system upgrade is complete, we can manually register the system. If you want to do a hands off registration during the upgrade, see the Oracle Solaris Auto Registration section of the Oracle Solaris Release Notes for instructions on how to do that.

Technocrati Tags: Oracle Solaris Patching Live Upgrade