[Milber09] Chapter 18. Linux on Power

Chapter 18. Linux on Power

This chapter provides an overview of systems performance when running Linux on Power (LoP).

18.1. Monitoring

AIX administrators will be happy to know that the nmon command works great with Linux. SystemTap, which conducts performance analysis by analyzing a running kernel, also runs on the platform. Two other popular tools, iostat and sar, are also available on Linux systems. While tools aren't the focus here, it's nice to know these options are available.

For our first monitoring example, let's inspect some basic Linux configuration files. The /proc file system is one you should use frequently, much more so than on AIX boxes because the information is simply more valuable in Linux. A lot of kernel and process information resides here, in the form of configuration files. One such file is cpuinfo:

Code View: Scroll / Show All

[root@172_29_140_173 proc]# more cpuinfo


processor : 0cpu : POWER5 (gr)clock : 1654.344000MHzrevision : 2.1 (pvr 003a 02001)

					    

It's easy to see from this file that you're using a POWER5 system.

Next, we can look at the release text file to determine the operating system level:

[root@172_29_140_173 etc]# more redhat-release
Red Hat Enterprise Linux Server release 5.2 (Tikanga)

This box is running Red Hat Enterprise Linux 5 (RHEL5) on a POWER5 partition.

18.2. Handy Linux Commands

One of my favorite Linux commands is top. This command provides realtime information quickly, in a character-based display, including the processes that are consuming the most CPU time.

Another useful command is free. It reports the total amount of free and used physical and swap memory:

[root@172_29_140_173 etc]# free

             total    used    free  shared  buffers  cached
Mem:       2073856 2057536  16320       0    440832 1389760
-/+ buffers/cache:  226944  1846912
Swap:            0       0        0

Still, my favorite of all Unix/Linux performance commands is vmstat. I love this old standby because, unlike other tools, vmstat provides quick-and-dirty information about all subsystems. Nothing fancy here:

Code View: Scroll / Show All

[root@172_29_140_173 etc]# vmstat 1

procs -----------memory---------- --swap-- ---io--- --system-- -----cpu------
 r b   swpd   free    buff  cache   si  so  bi   bo   in  cs    us sy id wa st
 0 0  16256 440832 1389760      0   0    1   1   23   11   0    1  98  0  0  0
 0 0  16384 440832 1389760      0   0    0   0  524   30   0    1  99  0  0  0
 0 0  16384 440832 1389760      0   0    0   0  536   16   0    1  99  0  0  0
 0 0  16320 440832 1389760      0   0    0   0  588   25   0    1  99  0  0  0
 0 0  16320 440832 1389760      0   0    0   0  628   12   0    1  99  0  0  0
 0 0  16320 440832 1389760      0   0    0  24  633   29   0    1  99  0  0  0
 0 0  16320 440832 1389760      0   0    0   0  578   18   0    1  99  0  0  0

					    

You will find that vmstat output on Linux differs a bit from what you see on AIX systems. Here's a quick description of what each field means:

• swpd — Amount of virtual memory being used

• free — Amount of idle memory

• buff — Amount of memory used as buffers

• cache — Amount of memory used as cache

18.3. Virtualization

Some administrators don't take full advantage of the PowerVM capabilities of Linux. If you administer LoP the same way you do on x86 boxes, you're doing yourself and your organization a major disservice. Some capabilities available on POWER systems include the following.

• Symmetric multithreading (SMT)

• Shared processor pool and uncapped partitions

From a CPU perspective, SMT is an important feature. It lets you maximize the use of instruction sets and in some cases increase CPU performance by 30 percent. SMT enables these improvements by supporting multithreading, a capability that's part of PowerVM and the POWER architecture. Multithreading enables two separate instruction streams to run concurrently on the same physical processor, with each thread appearing to run on its own independent logical processor. This feature is enabled by default.

Through the POWER architecture, you can create Linux and AIX partitions. When creating Linux partitions, you can "uncap" them, which means that the partitions will receive unused CPU cycles from the shared processor pool over and above their entitled capacity. Other than the number of cycles left in that shared processor pool, the only limitation is the number of virtual processors configured for the profile. I recommend uncapping partitions whenever possible to maximize all available CPU resources and increase performance. From a CPU perspective, I'd also take advantage of the capability to add CPU horsepower on the fly through a dynamic LPAR (DLPAR) operation.

18.4. Tuning

In Linux, the sysctl command changes kernel parameters. Be advised that the method you use to change parameters may depend on your distribution; for example, you can use the Powertweak tool with Novell SUSE Linux, but it isn't available with Red Hat. Because we're using Red Hat here, sysctl is the choice. Let's change some parameters.

One parameter that's changed frequently is SHMMAX, which is used to define the maximum size (in bytes) for a shared memory segment. In Oracle, you should set this value large enough for the largest System Global Area (SGA) size. Let's examine the default parameter:

# sysctl kernel.shmmax
kernel.shmmax = 268435456

In this case, the limit is set to 256 MB. Let's change this to 1 GB. To do so, use the vi command to display the /etc/sysctl.conf file. This is where you edit the value:

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 107374182

When you view the file again using sysctl, you can see the change:

# sysctl kernel.shmmax
kernel.shmmax = 107374182

To make the parameter work without a reboot, issue the sysctl command with the -p parameter.

On the memory side, parameters worth examining include SEMMSL, which controls the maximum number of semaphores per semaphore set; SEMMNI, which controls the maximum number of semaphore sets on the entire Linux system; and SEMMNS, which controls the maximum number of semaphores (no semaphore sets) on the entire Linux system.

Another important parameter is vm.nr.hugepages. The background here is that the POWER architecture supports page sizes of 4 KB and 16 MB. The default vm.nr.hugepages setting for LoP is 4 KB, which is too small for larger databases. To enable large pages, you need to change this parameter.

Let's first view the hugepage parameter — in this case, by looking at the proc/meminfo file.

# grep -i hugepages /proc/meminfo

HugePages_Total: 0
HugePages_Free:  0
HugePages_Rsvd:  0
Hugepagesize: 16384 kB

Now, let's allocate 130 large pages to support an SGA of approximately 2 GB:

# sysctl -w vm.nr_hugepages=130

Then view the meminfo file again:

[root@172_29_140_173 ~]# grep -i hugepages /proc/meminfo

HugePages_Total: 41
HugePages_Free:  41
HugePages_Rsvd:   0
Hugepagesize:  16384 kB

You can see that the system is already starting to use the hugepage parameter.

Entire books are dedicated to Linux performance tuning. Check your application recommendations to learn how kernel parameters should be configured in your environment.

Linux starts many daemons that usually aren't needed, including autofs, cups, nfslock, sendmail, and xfs. You should turn off anything that isn't explicitly required. You can accomplish this in several ways, but the chkconfig command is probably the best method.

As an example, let's shut down cups:

<[root@172_29_140_173 ~]# chkconfig --del cups

Now, let's make sure it's not around anymore:

#  chkconfig --list


service cups supports chkconfig, but is not referenced in any runlevel (run
 'chkconfig --add cups')