[pca] Cluster, zones, noreboot.

[French, David]

I can't speak to Sun's Cluster software but here we use Veritas VCS and
do update on attach.  The advantage is not as much time as being able to
schedule the work.  For example, if you patch them all at once, they are
all down while this is being done.  If you migrate zones to another
system, leaving just the global zone on one of the systems, you can
patch that system.  Then when it has been rebooted and checked out, you
can migrate systems as needed using update on attach, until they have
all been updated.   Then patch the other node that has just a global and
roll back some of the zones to that box.

This may cause a 2-3 reboots for some zones depending on where they
live, but zones usually boot fast compared to actual HW.  Just to give
you an idea, a system needing a hundred patches may take > 1 hr to
patch.   An update on attach will run in a fraction of that time.  It
also allows you to schedule the time with  your customer instead of
requiring they all be down for hours at the same time.  I find people
are more apt to accept quick reboots and the short time for  an update
on attach than accept being down for an extended period while you patch
everyone.

But if all zones are related then having them down at the same time may
not be an issue and a parallel patch may be more acceptable.

Now, I will say this.  Before patching validate the current packages and
patches.  I mention this as I ran into an issue on one of my systems
(non cluster, but global and 4 container/zones) where the SUNWcsl
package was missing pkginfo under /var/sadm/pkg/SUNWcsl.   Not only did
it cause patch issues, but it also broke update on attach so much that
the only option I had was to rebuild the zones.  Sun (at the time)
wasn't much help.  The issue was deeper than just the pkginfo file but
also the version of that file and the files in the package used in
patching the zones.  The  patch utilities mangled them and I didn't
catch it before patching.

I mention this as the version you are running is in the period I was at
when there were issues with the pkg/patch utilities, so better to check
in advance.

	--Dave



> -----Original Message-----
> From: pca-bounces at lists.univie.ac.at [mailto:pca-
> bounces at lists.univie.ac.at] On Behalf Of David Stark
> Sent: Tuesday, July 13, 2010 6:09 AM
> To: PCA (Patch Check Advanced) Discussion
> Subject: Re: [pca] Cluster, zones, noreboot.
> 
> Ah, sorry Filip, my question wasn't very clear.
> 
> I'm wondering if the downtime for each zone would be much less using
> Update On Attach compared with just bringing the whole cluster down
and
> patching all the zones in parallel?
> 
> On 13/07/2010 13:56, Filip Francis wrote:
> >
> > Dave,
> >
> > If you want todo parallel patchen you need to edit
> /etc/patch/pdo.conf
> > file.
> > That will make parallel patching to work on several zones at the
same
> time
> > Filip
> >
> > On 07/13/10 14:28, David Stark wrote:
> >> Hi all.
> >>
> >> Thanks Filip, Glen - I'd forgotten all about Update on Attach.
> >>
> >> It looks like Update On Attach is part of kernel patch 137137-09,
> >> which we've actually got applied on some recently patched clusters.
> >> The zoneadm man page lists the '-u' option on those hosts too, so
it
> >> looks like this should be usable on the next patching run.
> >> Since we've already got Parallel patching for zones, I'm not sure
if
> >> this will really save a lot of down time, though - is a zone
> >> update-attach any faster than a normal patch run?
> >>
> >> Dave
> >>
> >> On 13/07/2010 13:05, Filip Francis wrote:
> >>> No unless you have a certain version off Solaris i think this is
> only
> >>> from version 10u7 or 10u8 that you have this option.
> >>> This is his problem.
> >>> The cluster will not do this for you it think this is scheduled in
> the
> >>> next release of sun cluster later this year
> >>> Filip
> >>>
> >>>
> >>> On 07/13/10 13:54, Glenn Satchell wrote:
> >>>> When a zone migrates back to a patched system, doesn't it
normally
> >>>> update itself as it starts up the first time?
> >>>>
> >>>> Have a look at the zoneadm man page, in particular the attach and
> >>>> detach sub-commands. I don't know for sure, but Sun Cluster may
be
> >>>> smart enough to do the right thing and use zoneadm attach -u to
> bring
> >>>> the zone up to date when it attaches to the patched system.
> >>>>
> >>>> Perhaps a quick chat with your local Sun SE to help plan things
> might
> >>>> be time well spent?
> >>>>
> >>>> regards,
> >>>> -glenn
> >>>>
> >>>> On 07/13/10 21:41, David Stark wrote:
> >>>>> On 13/07/2010 12:31, Filip Francis wrote:
> >>>>>> Hi there,
> >>>>>
> >>>>> Hi!
> >>>>>
> >>>>>> I have done already quit a few upgrades of sun clusters.
> >>>>>> Can you give me some more details on what version off cluster +
> >>>>>> version
> >>>>>> off os.
> >>>>>
> >>>>> Sure. It's Sun Cluster 3.2 on Solaris 10 5/08. Mainly 2 node
> clusters,
> >>>>> but there's a 3 node and a 6 node as well. Mainly T5220 machines
> >>>>> hooked
> >>>>> up to EMC Symmetrix storage. The hosts' / slices are all UFS,
all
> the
> >>>>> zones and data filesystems are ZFS.
> >>>>>
> >>>>>> Regards
> >>>>>> Filip
> >>>>>
> >>>>> Cheers!
> >>>>> Dave
> >>>>>
> >>>>>> On 07/13/10 11:52, David Stark wrote:
> >>>>>>> Hi List.
> >>>>>>>
> >>>>>>> A bit off-topic, but PCA's involved, so I'm going to push my
> luck.
> >>>>>>>
> >>>>>>> We've got a number of Sun Cluster installs using (lots of)
> failover
> >>>>>>> zones and with no LiveUpgrade alt. boot space set up, which we
> >>>>>>> need to
> >>>>>>> patch. If we were to patch the clusters node-by-node
(including
> the
> >>>>>>> kernel patches that don't like zones being booted when they're
> >>>>>>> applied), the zones would fail over between the nodes and
never
> get
> >>>>>>> patched themselves, and since kernel (and some other?) patches
> can't
> >>>>>>> be applied from inside zones we would end up in a situation
> where
> >>>>>>> the
> >>>>>>> zones' patch databases are out of sync with the Global zone. I
> know
> >>>>>>> from very bitter experience that this is a Bad Thing, so to
> avoid
> >>>>>>> that
> >>>>>>> we're currently bringing the clusters down to patch them. This
> >>>>>>> obviously isn't optimal - people expecting 100% uptime from
the
> >>>>>>> clusters are naturally a bit annoyed at having their
> applications
> >>>>>>> down
> >>>>>>> for several hours while we unleash the mighty PCA.
> >>>>>>>
> >>>>>>> So, to minimise downtime I'd like to apply the noreboot
> patches,
> >>>>>>> say,
> >>>>>>> the night before, and have a more minimal patch run with the
> >>>>>>> clusters
> >>>>>>> down. This brings me to the question:
> >>>>>>>
> >>>>>>> Has anyone ever had any problems with noreboot patches applied
> to
> >>>>>>> live
> >>>>>>> systems? Any weirdness at all? I've patched plenty of test
> >>>>>>> machines in
> >>>>>>> multi-user mode, but never busy production boxes - these are
> fairly
> >>>>>>> large Oracle and SAP environments for the most part.
> >>>>>>>
> >>>>>>> Anyone with experience patching Sun Cluster care to share any
> top
> >>>>>>> tips?
> >>>>>>>
> >>>>>>> Cheers!
> >>>>>>>
> >>>>>>> Dave
> >>>>>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>
> >>
> >>
> >>
> >
>