Solaris Zone in “shutting_down” state

One of the most annoying situations when administering  Solaris zones is when you try to reboot / halt a zone , the zone goes into a “shutting_down” state which is kind of a hung state where it is neither down nor up.

The only solution to get the zone out of the “shutting_down” state was to reboot the global which is not really good  because it brings down all the other local zones on the global.

On Solaris 10 10/08 (update 6) have observed that there may be a workaround for this issue.

The following are the steps that I followed from the global

zoneadm list -cv — This shows the affected local zone in “shutting_down”

ps -ef | grep zoneadmd — Find the PID for the zoneadmd corresponding to the affected zone

kill -9 PID — Kill the PID which we get from the previous command

zoneadm -z <zone> halt — Run the command to halt a zone with <zone> being affected zone name

The above command should complain about the zoneadmd for the zone being down and being restarted or something like that.

Disclaimer : This worked for me a couple of times on Update 6 but did not work for me on update 5 and below.

Hope this helps.

Install Solaris 10u6 x86 on HP BL460c

Problem:

Trying to install Solaris 10u6 x86 onto a HP BL460c server blade through the iLOv2 . The install errors out with “No Disks Found” message

Solution:

Download and install the HP SmartArray controller driver from HP site (CPQary3-2.0.0-solaris10-i386)

Procedure:

• Mount the ISO as a virtual cdrom from the iLO v2 web interface and boot the server blade
• When the install comes to the point where you are asked to choose what kind of install ( the prompt where it asks whether you want an interactive , desktop , text console etc)
• Choose the option 5 , something like “install 3rd party package/patch”,  and it asks you where to install the package / patch from
• From the iLO v2 interface , umount the Solaris 10 ISO and mount the image of the downloaded driver from HP and choose the option for searching all paths (cdrom , non-usb …etc)
• This will find and install the driver.
• Umount the image of the driver and mount the Solaris 10 ISO again.
• Continue with normal installation

Note:

Since most of our servers are SPARC based , did not want to take the pain to configure Jumpstart for a single server , but if Jumpstart is used for installation, addition of 3rd party packages / patches is easier.

Recommendation :

Choose the option 3, 4 from the installation menu and you get to choose the option to boot from ZFS. It has worked flawlessly for us. Your mileage may vary.

Migrate from EMC powerpath to MPxIO.

As always use at your own discretion. I do not take any responsibility for any screw-up/data loss and please take a good backup

To migrate from Powerpath to Solaris MPxIO, the following steps can be used .This has been working on Solaris 10 with zpools on powerpath devices

Note: Take care when executing “stmsboot -e” on servers like v880/v890 which has fibre connectivity for internal disks.

Note1: If “stmsboot -e” is run on a server which has zfs boot (Solaris 10 10/08), during boot a scary message is displayed on console which   can be safely ignored ( at least for us until now)

1. Remove the Powerpath package using pkgrm and reboot.
2. The machine should come up without Powerpath
3. Run “stmsboot -e -D fp” and follow prompts (Reboot required)

The system should come up with MPxIO enabled and the luns which are part of the zpools change their name from emcpowerXa to the MPxIO format.

If format does not show MPxIO multipathed disks … try adding the support for third party devices in the /kernel/drv/scsi_vhci.conf file. and rebooting the server again.

Raw Devices in Solaris Zones

To check if any LUNs are being used as raw devices within Solaris Zones, the following one-liner can be used from the global.

for i in `zoneadm list | grep -v global`; do echo $i; zonecfg -z $i info | grep -i match;done

(or)

cd /<zone-root>/root/dev/rdsk; ls

Note: the first one will only work if the zonecfg has been updated to include the raw device

Solaris Zones Update On Attach

Quick note to self:

On update on attach using Solaris zones

a) If the attach errors out saying some specific patches are not installed, add the patch numbers to

/usr/lib/brand/native/bad_patches

b) If instead of the -u (upgrade) option, the -f(force) option is specified, the following procedure can be used to update

• Detach the zone
• Remove the SUNWDetached.xml file
• Attach the zone using the -u option
• Verify the update happened by checking the update_log file at /var/sadm/install/logs

AIX – Increase FS size

The following is the procedure which can be used to configure a LUN which is allocated from SAN
and use it to increase the filesystem space .

Please use your own discretion, I will not be held responsible for any issues caused by trying out the commands given below

this command to find the VG name

lslv <lv-name>

This command can be used to find the free PPs available in the VG

lsvg <vg-name>

This command can be used to find LUNs which are not associated with any VG

lspv | grep -i none

If no free LUNs are available or if lsvg <vg-name> does not have the required space , its time to add LUNs to your system, once the LUNs are added

This command can be used to bring the storage into the system ( like Solaris devfsadm -Cv)

cfgmgr -v

This shows you the newly added LUNs

lspv | grep -i none

Add the new LUN to the required VG , the “-f” option may need to be used if this LUN was previously assigned to some other server and was part of a VG

extendvg <vg-name> <new-LUN>

check if the required space is available

lsvg <vg-name>

This command can be used to increase the filesystem space

chfs -a size=+<space-required>G <mount-point>

eg: This command will add 2G more to the filesystem on the mountpoint

chfs -a size=+2G /mount-point

Note :

There is a very good chance that you might see an error like this

0516-787 extendlv: Maximum allocation for logical volume<lv-name> is 1024 (or some number)

In case of an error like that, the LPs of a particular LV can be increased using the chlv command

chlv -x 2048 <lv-name>

where 2048= 1024 (existing) + 1024 (new)

Intro

I am an infrastructure consultant working with a MNC in Bangalore, India. My area of focus is the support of Datacenter infrastructure which includes system, network and security administration.

This blog is intended to be a central location to hold resources/links which I find useful and a quick reminder to stuff which I always tend to forget

So HELLO WORLD!

Btw, what is written here is my personal opinion on stuff and does not reflect the views of anybody else which includes among others the company which I work for, my family or my friends.