Network Time Protocol

By – Kunal Raykar

NETWORK TIME PROTOCOL Solaris 10

Network Time Protocol (NTP) is a networking protocol for clock synchronization between computer
systems.
On Solaris 10 ntp service is controlled by svcadm.
#svcs -a | grep ntp
online 13:09:43 svc:/network/ntp:default

The NTP servers advertise every 64 seconds, by means of a multicast address (224.0.1.1), that they are NTP servers. NTP server multicasts when the xntpd process starts. View the line that causes the system to act as an NTP server by typing the following:

myhost# grep broadcast /etc/inet/ntp.server
broadcast 224.0.1.1 ttl 4

Local NTP servers answer the multicast advertisements. The NTP client sends request packets to all the
NTP servers that it knows. Included in the request packet is the client’s local time. The NTP server replies
by inserting UTC time into the packet and then returns the packet to the client. The client compares its
original request time with its own time when it receives the response from the server. This allows the
client to determine how long the packet was in transit on the network.

The client uses the UTC time value from the NTP server after it receives several responses from the NTP
server. It can take up to five minutes for an NTP client to synchronize with an NTP server.
The /etc/inet/ntp.server file is a template for configuring an NTP server. Copy this file to
/etc/inet/ntp.conf, and edit it to meet your network’s requirements. When viewing the ntp.server file
contents, remember that an NTP server is also an NTP client.

Here we have 2 machine 10.0.4.61(server) and 10.0.4.62(client)
SERVER SIDE CONFIGURATION
Using an Undisciplined Local Clock

NTP servers can, but should not, use their own undisciplined local clock as an official, reliable time source.
To use an undisciplined local clock, complete the following steps:
Copy the /etc/inet/ntp.serverfile to the /etc/inet/ntp.conffile.

myhost# cp /etc/inet/ntp.server /etc/inet/ntp.conf

Open the /etc/inet/ntp.conf file for editing, and change the server IP address to 127.127.1.0, where the
number 1 represents the undisciplined local clock. Comment out the fudge keyword because special
configuration is not needed for the local reference clock.

myhost# vi /etc/inet/ntp.conf

Change:
server 127.127.XType.0 prefer
fudge 127.127.XType.0 stratum 0
to:
server 127.127.1.0 prefer
# fudge 127.127.XType.0 stratum 0

Note – Choices for XType are listed in the comments of the /etc/inet/ntp.server file.

3. Create a drift file as specified by the drift file /var/ntp/ntp.drift entry in the /etc/inet/ntp.conf file:

#touch /var/ntp/ntp.drift

NOTE:The units for the drift file are "PPM", or "parts per million". Your clock will drift due to fluctuations in the frequency oscillating the quartz crystal on your motherboard. A fluctuation of just 0.001% means losing or gaining about 1 second per day. NTP has finer grained control than that, so we look at errors of margin using 0.0001%. Thus:

1 PPM = 1 part per million = 1 microsecond per second = 3.6ms per hour = 86.4ms per day.
My drift file shows the value of "2.643" which means my clock is off by 2.643 parts per million, which means it's currently off at 228.3552ms per day.

CLIENT SIDE CONFIGURATION:
On client side copy /etc/inet/ntp.client to /etc/inet/ntp.conf
And modify the ntp.conf with following changes.
Change
multicastclient 224.0.1.1
to
server 10.0.4.61

Restart the service of ntp on both client and server
#svcadm restart ntp
NOTE: Sometime you have to take reboot of machine in order for changes to take effect. In such cases reboot the server first and then client.
This is not the recommended technique, but sometime it is required.

NTPQ Command :
ntpq is used to query ntp servers about status. The most commonly used option to ntpq is -p (“Print a list
of the peers known to the server as well as a summary of their state.”).
Here we have example:

bash-3.00#
bash-3.00# ntpq -p
remote refid st t when poll reach delay offset disp
==============================================================================
*10.0.4.61 LOCAL(0) 4 u 27 64 377 0.56 60.489 2.53
bash-3.00#

where:
remote: The address of the remote peer
refid:The reference ID (0.0.0.0 if the ref ID is unknown)
st: The stratum of the remote peer
t: The type of the peer (local, unicast, multicast or broadcast) when the last packet was received
poll:The polling interval in seconds
reach:The reachability register, in octal delay, offset, disp:The current estimated delay offset and dispersion of the peer, all in milliseconds.

Solaris UFS to ZFS

By Kunal Raykar

Requirement:
We need a physical disk matching to the current root hard disk size. If you don’t have spare disk, you remove the current mirror disk and use it for ZFS convert.
Assumptions:
Old disk: c1t0d0
New disk:c1t1d0
The new disk should formatted with SMI label and keep all the sectors in s0. EFI label is not supported for root pool.

bash-3.00# format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c1t0d0 <DEFAULT cyl 1563 alt 2 hd 255 sec 63>
/pci@0,0/pci15ad,1976@10/sd@0,0
1. c1t1d0 <DEFAULT cyl 1563 alt 2 hd 255 sec 63>
/pci@0,0/pci15ad,1976@10/sd@1,0
Specify disk (enter its number):

NOTE: You have ufs file system mounted on the old disk c1t0d0 and also has pool mounted on the old disk.Migration should be performed with data 0 loss.
The disk should be labled as the SMI(Sun Microsystem Inc.)ie vtoc, instead of the EFI(Extensible firmware
interface)

Conversion from EFI to SMI label:

bash-3.00#
bash-3.00# format -e c1t1d0
selecting c1t1d0
[disk formatted]
FORMAT MENU:
disk  - select a disk
type  - select (define) a disk type
partition  - select (define) a partition table
current  - describe the current disk
format  - format and analyze the disk
fdisk  - run the fdisk program
repair  - repair a defective sector
label  - write label to the disk
analyze  - surface analysis
defect  - defect list management
backup  - search for backup labels
verify  - read and display labels
save  - save new disk/partition definitions
inquiry  - show vendor, product and revision
scsi  - independent SCSI mode selects
cache  - enable, disable or query SCSI disk cache
volname  - set 8-character volume name
!<cmd>  - execute <cmd>, then return
quit
format> p
PARTITION MENU:
0  - change `0' partition
1  - change `1' partition
2  - change `2' partition
3  - change `3' partition
4  - change `4' partition
5  - change `5' partition
6  - change `6' partition
7  - change `7' partition
9  - change `9' partition
select - select a predefined table
modify - modify a predefined partition table
name  - name the current table
print  - display the current table
label  - write partition map and label to the disk
!<cmd> - execute <cmd>, then return
quit
partition> l
[0] SMI Label
[1] EFI Label
Specify Label type[0]: 0
Ready to label disk, continue? yes
partition>
partition>

Copy the vtoc from old disk to new disk.
bash-3.00# prtvtoc /dev/rdsk/c1t0d0s2 | fmthard –s - /dev/rdsk/c1t1d0s2

Creating rpool:
First create zpool with the name of rpool using the newly configured disk.

bash-3.00# zpool create rpool c1t1d0s0
bash-3.00#
bash-3.00#
bash-3.00# zfs list
NAME USED AVAIL REFER MOUNTPOINT
rpool 73.5K 11.8G 21K /rpool
bash-3.00#
Verify if you are having existing boot environment to name current boot environment,

bash-3.00# lustatus
ERROR: No boot environments are configured on this system
ERROR: cannot determine list of all boot environment names
bash-3.00#

Creating the new boot environment using rpool:
Now we can create a new boot environment using the newly configured zpool (i.e rpool) .
-c -- current boot environment name
-n -- new boot environment name
-p -- Pool name

bash-3.00# lucreate -c sol_ufs -n sol_zfs -p rpool
Checking GRUB menu...
Analyzing system configuration.
No name for current boot environment.
Current boot environment is named <sol_ufs>.
Creating initial configuration for primary boot environment <sol_ufs>.
The device </dev/dsk/c1t0d0s0> is not a root device for any boot environment; cannot get BE
ID.
PBE configuration successful: PBE name <sol_ufs> PBE Boot Device </dev/dsk/c1t0d0s0>.
Comparing source boot environment <sol_ufs> file systems with the file
system(s) you specified for the new boot environment. Determining which
file systems should be in the new boot environment.
Updating boot environment description database on all BEs.
Updating system configuration files.
The device </dev/dsk/c1t1d0s0> is not a root device for any boot environment; cannot get BE
ID.
Creating configuration for boot environment <sol_zfs>.
Source boot environment is <sol_ufs>.
Creating boot environment <sol_zfs>.
Creating file systems on boot environment <sol_zfs>.
Creating <zfs> file system for </> in zone <global> on <rpool/ROOT/sol_zfs>.
Populating file systems on boot environment <sol_zfs>.
Checking selection integrity.
Integrity check OK.
Populating contents of mount point </>.
Copying.
cp: cannot access //platform/i86pc/bootlst
Creating shared file system mount points.
Creating compare databases for boot environment <sol_zfs>.
Creating compare database for file system </>.
Updating compare databases on boot environment <sol_zfs>.
Making boot environment <sol_zfs> bootable.
Updating bootenv.rc on ABE <sol_zfs>.
File </boot/grub/menu.lst> propagation successful
Copied GRUB menu from PBE to ABE
No entry for BE <sol_zfs> in GRUB menu
Population of boot environment <sol_zfs> successful.
Creation of boot environment <sol_zfs> successful.
bash-3.00# 

Activating the new boot environment:
Once the lucreate is done,then activate the new boot environment.So that system will boot from new BE from next time onwards.
bash-3.00#
bash-3.00# luactivate sol_zfs
Generating boot-sign, partition and slice information for PBE <sol_ufs>
A Live Upgrade Sync operation will be performed on startup of boot environment <sol_zfs>.
Generating boot-sign for ABE <sol_zfs>
NOTE: File </etc/bootsign> not found in top level dataset for BE <sol_zfs>
Generating partition and slice information for ABE <sol_zfs>
Boot menu exists.
Generating multiboot menu entries for PBE.
Generating multiboot menu entries for ABE.
Disabling splashimage
Re-enabling splashimage
No more bootadm entries. Deletion of bootadm entries is complete.
GRUB menu default setting is unaffected
Done eliding bootadm entries.
**********************************************************************
The target boot environment has been activated. It will be used when you
reboot. NOTE: You MUST NOT USE the reboot, halt, or uadmin commands. You
MUST USE either the init or the shutdown command when you reboot. If you
do not use either init or shutdown, the system will not boot using the
target BE.
**********************************************************************
In case of a failure while booting to the target BE, the following process
needs to be followed to fallback to the currently working boot environment:
1. Boot from the Solaris failsafe or boot in Single User mode from Solaris
Install CD or Network.
2. Mount the Parent boot environment root slice to some directory (like
/mnt). You can use the following command to mount:
mount -Fufs /dev/dsk/c1t0d0s0 /mnt
3. Run <luactivate> utility with out any arguments from the Parent boot
environment root slice, as shown below:
/mnt/sbin/luactivate
4. luactivate, activates the previous working boot environment and
indicates the result.
5. Exit Single User mode and reboot the machine.
**********************************************************************
Modifying boot archive service
Propagating findroot GRUB for menu conversion.
File </etc/lu/installgrub.findroot> propagation successful
File </etc/lu/stage1.findroot> propagation successful
File </etc/lu/stage2.findroot> propagation successful
File </etc/lu/GRUB_capability> propagation successful
Deleting stale GRUB loader from all BEs.
File </etc/lu/installgrub.latest> deletion successful
File </etc/lu/stage1.latest> deletion successful
File </etc/lu/stage2.latest> deletion successful
Activation of boot environment <sol_zfs> successful.
bash-3.00#
bash-3.00#
Reboot the server using init 6 to boot from new boot environment:
bash-3.00#
bash-3.00# init 6
updating /platform/i86pc/boot_archive
propagating updated GRUB menu
Saving existing file </boot/grub/menu.lst> in top level dataset for BE <sol_zfs> as
<mount-point>//boot/grub/menu.lst.prev.
File </boot/grub/menu.lst> propagation successful
File </etc/lu/GRUB_backup_menu> propagation successful
File </etc/lu/menu.cksum> propagation successful
File </sbin/bootadm> propagation successful
bash-3.00#
bash-3.00#
bash-3.00# df -h
Filesystem size used avail capacity Mounted on
rpool/ROOT/sol_zfs 12G 4.7G 5.8G 46% /
/devices 0K 0K  0K 0% /devices
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 727M 392K 727M 1% /etc/svc/volatile
objfs 0K 0K 0K 0% /system/object
sharefs 0K 0K 0K 0% /etc/dfs/sharetab
/usr/lib/libc/libc_hwcap1.so.1
11G 4.7G 5.8G 46% /lib/libc.so.1
fd 0K 0K 0K 0% /dev/fd
swap 727M 44K 727M 1% /tmp
swap 727M 28K 727M 1% /var/run
rpool 12G 34K 5.8G 1% /rpool
/vol/dev/dsk/c0t0d0/sol_10_910_x86
2.0G 2.0G 0K 100% /cdrom/sol_10_910_x86
bash-3.00#
bash-3.00#
bash-3.00# lustatus
Boot Environment Is Active Active Can Copy
Name Complete Now On Reboot Delete Status
-------------------------- -------- ------ --------- ------ ----------sol_ufs yes no no yes  -sol_zfs yes yes yes no  -bash-3.00#
bash-3.00#

Now you can see the server is booted in ZFS.

bash-3.00#
bash-3.00# zfs list
NAME USED AVAIL REFER MOUNTPOINT
rpool 5.97G 5.78G 34.5K /rpool
rpool/ROOT 4.75G 5.78G 21K legacy
rpool/ROOT/sol_zfs 4.75G 5.78G 4.75G /
rpool/dump 512M 5.78G 512M  -rpool/swap 745M 6.50G 16K  
bash-3.00#
bash-3.00#
bash-3.00#
bash-3.00# zpool status
pool: rpool
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
c1t1d0s0 ONLINE 0 0 0
errors: No known data errors
bash-3.00#

You can remove the old boot environment using the below command

bash-3.00#
bash-3.00# ludelete -f sol_ufs
System has findroot enabled GRUB
Updating GRUB menu default setting
Changing GRUB menu default setting to <0>
Saving existing file </boot/grub/menu.lst> in top level dataset for BE <sol_zfs> as
<mount-point>//boot/grub/menu.lst.prev.
File </etc/lu/GRUB_backup_menu> propagation successful
Successfully deleted entry from GRUB menu
Determining the devices to be marked free.
Updating boot environment configuration database.
Updating boot environment description database on all BEs.
Updating all boot environment configuration databases.
Boot environment <sol_ufs> deleted.
bash-3.00#
bash-3.00#
bash-3.00# lustatus
Boot Environment Is Active Active Can Copy
Name Complete Now On Reboot Delete Status
-------------------------- -------- ------ --------- ------ ----------sol_zfs yes yes yes no  
bash-3.00#

IMPORTANT NOTE:
Old disk: c1t0d0
New disk: c1t1d0
Assumption: The configuration of the disk after the boot in which there /pool100 as zfs and d10 as the svm volume.
These are the mounted partition, which contain data.
In such case it’s better to mirror the both zfs and svm data. So the data loss will be 0.
For example
c1t0d0s6 => disk contain svm mirror this with c1t1d0s6 (d10)
c1t0d0s7 => disk contain zfs data mirror it with the c1t1d0s7 (/pool100)
Initiating the rpool mirroring:

bash-3.00# zpool attach -f rpool c1t1d0s0 c1t0d0s0
Please be sure to invoke installgrub(1M) to make 'c1t0d0s0' bootable.
Make sure to wait until resilver is done before rebooting.
bash-3.00#
bash-3.00#
bash-3.00# zpool status
pool: rpool
state: ONLINE
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scrub: resilver in progress for 0h0m, 10.90% done, 0h4m to go
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0   0 0
mirror-0 ONLINE 0 0 0
c1t1d0s0 ONLINE 0 0 0
c1t0d0s0 ONLINE 0 0 0 583M resilvered
errors: No known data errors
bash-3.00#
bash-3.00#

NOTE: Reboot the server after the syncing has been done between 2 legs of the mirror.

bash-3.00#
bash-3.00# init 6
updating /platform/i86pc/boot_archive
^[`propagating updated GRUB menu
File </boot/grub/menu.lst> propagation successful
File </etc/lu/GRUB_backup_menu> propagation successful
File </etc/lu/menu.cksum> propagation successful
File </sbin/bootadm> propagation successful
bash-3.00#

IPMP in Solaris 10

IPMP configuration using LINK BASED TECHNIQUE in Solaris 10

By Kunal Raykar

IPMP eliminates single network card failure and it ensures system will be always accessible via network.
The link based IPMP detects network errors by checking the "IFF_RUNNING" flag. Normally it doesn't
require any test IP address like probe based IPMP.
"/etc/default/mpathd" is file to configure the ipmp and the default value is 10 seconds for the failure
detection. In this file there is an option called “FAILBACK" to specify IP behavior when primary interface
recovered from the fault. "in.mpathd" is a daemon which handles IPMP (Internet Protocol Multi-
Pathing) operations.

We have Solaris ( hostname=solaris IP=10.0.4.61) machine with 2 NIC card.
e1000g0 => primary card
e1000g1 => secondary card

bash-3.00# dladm show-dev
e1000g0 link: up speed: 1000 Mbps duplex: full
e1000g1 link: unknown speed: 1000 Mbps duplex: full
bash-3.00#
bash-3.00# cat /etc/hosts
::1 localhost
127.0.0.1 localhost
10.0.4.61 solaris solaris.com loghost
10.0.4.61 solaris loghost
bash-3.00#
bash-3.00# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 10.0.4.61 netmask ffffff00 broadcast 10.0.4.255
ether 0:c:29:fb:38:68
bash-3.00#
Now plumb the secondary NIC card
bash-3.00#ifconfig e1000g1 plumb
bash-3.00# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 10.0.4.61 netmask ffffff00 broadcast 10.0.4.255
ether 0:c:29:fb:38:68
e1000g1: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4
inet 0.0.0.0 netmask 0
ether 0:c:29:fb:38:72
bash-3.00#

To configure Link based IPMP, create the hostname.<int name> files for the Active NIC and the standby NIC. In this
case we have /etc/hostname.e1000g0 and /etc/hostname.e1000g1
Edit the hostname.e1000g0 file and enter the following:

bash-3.00#vi /etc/hostname.e1000g0
solaris10 netmask + broadcast + group sol10-ipmp up

solaris is the hostname which should have a corresponding host entry in the /etc/hosts file.
sol10-ipmp is the name of the IPMP group.
Edit the hostname.e1000g1 and the following

bash-3.00# cat /etc/hostname.e1000g1
group sol10-ipmp up

Take a reboot of server.

bash-3.00# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index
1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 10.0.4.61 netmask ffffff00 broadcast 10.0.4.255
groupname sol10-ipmp
ether 0:c:29:fb:38:68
e1000g1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255
groupname sol10-ipmp
ether 0:c:29:fb:38:72
bash-3.00#

The standby NIC only has IPMP group configuration. This all is needed to setup Link based IPMP. When the server is
rebooted this configuration should take effect.
To test failover, you can do the hard way of pulling the cables or use if_mpadm command as follows:
To failover:

bash-3.00# if_mpadm -d e1000g0
-d is for detach
After detach you see that a virtual interface is created system to handle the IP. In this case e1000g1:1 is
the virtual interface

bash-3.00# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g0: flags=89000842<BROADCAST,RUNNING,MULTICAST,IPv4,NOFAILOVER,OFFLINE> mtu 0 index2
inet 0.0.0.0 netmask 0
groupname sol10-ipmp
ether 0:c:29:fb:38:68
e1000g1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255
groupname sol10-ipmp
ether 0:c:29:fb:38:72
e1000g1:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
inet 10.0.4.61 netmask ffffff00 broadcast 10.0.4.255
bash-3.00#

To failback
bash-3.00# if_mpadm -r e1000g0
-r is for reattach