一点一滴努力,满仓满谷收成

1 为什么要介绍/使用HugePage?
Baidu一下，讲的很多，大体如下：
1)当开启HugePage的时候，HugePage是不会Swap的；
2)减少Page Table空间负载；
3)如OS内存非常的大,可以减少管理/访问内存的时间,减少了管理过程的复杂性；
2 对于Oracle,HugePage缺点
11g新特性 AMM就不能使用了，不过ASMM仍然可以继续使用。
3 配置流程
以下步骤以RHEL6.4为例。
1)首先要确保操作系统内核打开了如下项目.

[root@pg01 ~]# cat /usr/src/kernels/`uname -r`/.config|grep HUG
CONFIG_TRANSPARENT_HUGEPAGE=y
CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y

2) 设置memlock的限制，更改/etc/security/limits.conf加入下面的行

* soft memlock 60397977
* hard memlock 60397977

服务器64G内存,让它的值稍微比系统的物理内存小一些。
3) 检查memlock是否生效，要使用oracle的用户执行下面的操作

$ ulimit -l
60397977

4) 如果使用11g数据库，确认参数MEMORY_TARGET和MEMORY_MAX_TARGET已经设为0
5) 启动数据库，并运行Document 401749.1提供的脚本来计算应该分配多少HugePage页面。
hugepages_settings.sh

#!/bin/bash
#
# hugepages_settings.sh
#
# Linux bash script to compute values for the
# recommended HugePages/HugeTLB configuration
#
# Note: This script does calculation for all shared memory
# segments available when the script is run, no matter it
# is an Oracle RDBMS shared memory segment or not.
#
# This script is provided by Doc ID 401749.1 from My Oracle Support
# http://support.oracle.com

# Welcome text
echo ”
This script is provided by Doc ID 401749.1 from My Oracle Support
(http://support.oracle.com) where it is intended to compute values for
the recommended HugePages/HugeTLB configuration for the current shared
memory segments. Before proceeding with the execution please note following:
* For ASM instance, it needs to configure ASMM instead of AMM.
* The ‘pga_aggregate_target’ is outside the SGA and
you should accommodate this while calculating SGA size.
* In case you changes the DB SGA size,
as the new SGA will not fit in the previous HugePages configuration,
it had better disable the whole HugePages,
start the DB with new SGA size and run the script again.
And make sure that:
* Oracle Database instance(s) are up and running
* Oracle Database 11g Automatic Memory Management (AMM) is not setup
(See Doc ID 749851.1)
* The shared memory segments can be listed by command:
# ipcs -m

Press Enter to proceed…”

read

# Check for the kernel version
KERN=`uname -r | awk -F. ‘{ printf(“%d.%d\n”,$1,$2); }’`

# Find out the HugePage size
HPG_SZ=`grep Hugepagesize /proc/meminfo | awk ‘{print $2}’`
if [ -z “$HPG_SZ” ];then
echo “The hugepages may not be supported in the system where the script is being executed.”
exit 1
fi

# Initialize the counter
NUM_PG=0

# Cumulative number of pages required to handle the running shared memory segments
for SEG_BYTES in `ipcs -m | cut -c44-300 | awk ‘{print $1}’ | grep “[0-9][0-9]*”`
do
MIN_PG=`echo “$SEG_BYTES/($HPG_SZ*1024)” | bc -q`
if [ $MIN_PG -gt 0 ]; then
NUM_PG=`echo “$NUM_PG+$MIN_PG+1” | bc -q`
fi
done

RES_BYTES=`echo “$NUM_PG * $HPG_SZ * 1024” | bc -q`

# An SGA less than 100MB does not make sense
# Bail out if that is the case
if [ $RES_BYTES -lt 100000000 ]; then
echo “***********”
echo “** ERROR **”
echo “***********”
echo “Sorry! There are not enough total of shared memory segments allocated for
HugePages configuration. HugePages can only be used for shared memory segments
that you can list by command:

# ipcs -m

of a size that can match an Oracle Database SGA. Please make sure that:
* Oracle Database instance is up and running
* Oracle Database 11g Automatic Memory Management (AMM) is not configured”
exit 1
fi

# Finish with results
case $KERN in
‘2.2’) echo “Kernel version $KERN is not supported. Exiting.” ;;
‘2.4’) HUGETLB_POOL=`echo “$NUM_PG*$HPG_SZ/1024” | bc -q`;
echo “Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL” ;;
‘2.6’) echo “Recommended setting: vm.nr_hugepages = $NUM_PG” ;;
‘3.8’) echo “Recommended setting: vm.nr_hugepages = $NUM_PG” ;;
esac

# End

运行：

$ ./hugepages_settings.sh
…
Recommended setting: vm.nr_hugepages = 1496
$

6) 更改/etc/sysctl.conf，把上一步得到的值指定给vm.nr_hugepages参数

…
vm.nr_hugepages = 1496
…

7) 重启OS和数据库
8) 验证HugePage是否已启用

# grep HugePages /proc/meminfo
HugePages_Total: 1496
HugePages_Free: 485
HugePages_Rsvd: 446
HugePages_Surp: 0

Free不等于Total说明配置成功
4 PostgreSQL相应设置
PG数据库,huge_pages 默认为 try, 表示尝试使用hugepage, 如果失败则使用原来的内存管理方式。
看一下PG数据库用了多少内存

[root@pg01 ~]# cat $PGDATA/postmaster.pid
25359
/opt/pg944/data
1440985518
5432
/tmp
*
5432001 131076

可见主进程ID为：25359, 查看它的VmPeak.

[root@pg01 ~]# cat /proc/25359/status |grep -i vmpeak
VmPeak: 33698088 kB

这是内存峰值, 只要确保可用的hugepage总数比这个多就可以了.
从/proc/meminfo得知Hugepagesize=2MB, 所以33698088/2/1024=16454, 只要nr_hugepages比这个大就可以了。

vi /etc/sysctl.conf
vm.nr_hugepages=16500

5 本文主要内容来自以下Oracle note

Document 361323.1 HugePages on Linux: What It Is… and What It Is Not…
Document 401749.1 Shell Script to Calculate Values Recommended Linux HugePages / HugeTLB Configuration