Oracle Database 開啟 HugePage (Oracle Database Enable HugePage on Redhat 7)

1. Check Memory & PageSize

[oracle@test-odb1 ~]$ cat /proc/meminfo |grep MemTotal
===> MemTotal:       49291352 kB
[oracle@test-odb1 ~]$ getconf PAGESIZE
===> 4096
===> 記憶體管理過小 oracle 忙於找空間給 session user

2. Config Oracle memory_target & sga_target by Oracle
( Example: sga_target = 8G & sga_max_size = 10G )

[oracle@test-odb1 ~]$ sqlplus / as sysdba
SQL> alter system set memory_max_target=0 scope=spfile;
SQL> alter system set memory_target=0 scope=spfile;
SQL> alter system set sga_max_size =10240m scope=spfile;
SQL> alter system set sga_target=8192m scope=spfile;

SQL> show parameter sga
NAME                         TYPE        VALUE   
---------------------------- ----------- ------- 
allow_group_access_to_sga    boolean     FALSE   
lock_sga                     boolean     FALSE   
pre_page_sga                 boolean     TRUE    
sga_max_size                 big integer 10G     
sga_min_size                 big integer 0       
sga_target                   big integer 8G      
unified_audit_sga_queue_size integer     1048576 

3. Get hugepage value

[oracle@test-odb1 ~]$ cd /home/oracle/script
[oracle@test-odb1 ~]$ vi hugepages_setting.sh

#!/bin/bash
#
# hugepages_setting.sh
#
# Linux bash script to compute values for the
# recommended HugePages/HugeTLB configuration
#
# Note: This script does calculation for all shared memory
# segments available when the script is run, no matter it
# is an Oracle RDBMS shared memory segment or not.
# Check for the kernel version
KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`
# Find out the HugePage size
HPG_SZ=`grep Hugepagesize /proc/meminfo | awk {'print $2'}`
# Start from 1 pages to be on the safe side and guarantee 1 free HugePage
NUM_PG=1
# Cumulative number of pages required to handle the running shared memory segments
for SEG_BYTES in `ipcs -m | awk {'print $5'} | grep "[0-9][0-9]*"`
do
  MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q`
  if [ $MIN_PG -gt 0 ]; then
     NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q`
  fi
done
# Finish with results
case $KERN in
  '2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`;
         echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;;
  '2.6' | '3.8' | '3.10' | '4.1' | '4.14' | '4.18' ) echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
   *) echo "Unrecognized kernel version $KERN. Exiting." ;;
esac
# End

[oracle@test-odb1 ~]$ chmod u+x hugepages_setting.sh
[oracle@test-odb1 ~]$ ./hugepages_setting.sh
Recommended setting: vm.nr_hugepages = 5136

4. Edit /etc/sysctl.conf by root on RAC both
( 設定 vm.nr_hugepages 需用到 memory 5136 ( 5136 * 2048 需大於 sga_max_size) )
( 執行 sysctl -p 需要等一段時間 ，可以從 Step 5 看已經建立多少 HugePages_Total )

[root@test-odb1 ~]# vi /etc/sysctl.conf
# setup huge memory (5130 *2048 for sga_max_siz=10GB)
# default memory page =4k hugepage=2MB )
vm.nr_hugepages = 5136
#vm.hugetlb_shm_group = 54322

[root@test-odb1 ~]# sysctl -p
vm.nr_hugepages = 5136

5. Check meminfo

[root@test-odb1 ~]# grep Huge /proc/meminfo
AnonHugePages:   1366016 kB
HugePages_Total:    5132
HugePages_Free:     1034
HugePages_Rsvd:     1024
HugePages_Surp:        0
Hugepagesize:       2048 kB

6. Config /etc/security/limits.conf on RAC both
( 可設定 RAM 的 90% 給 memlock for Oracle )

Edit the memlock setting in the /etc/security/limits.conf file. The memlock setting is specified in KB, 
and the maximum locked memory limit should be set to at least 90 percent of the current RAM

#Add the following entries into /etc/security/limits.conf" script or "/etc/security/limits.d/99-grid-oracle-limits.conf"
oracle  hard  memlock  10510336
oracle  soft  memlock  10510336

7. Check Is hugepage enable or not

[root@test-odb1 ~]# cat /sys/kernel/mm/redhat_transparent_hugepage/enabled
[root@test-odb1 ~]# cat /sys/kernel/mm/redhat_transparent_hugepage/defrag
Or
[root@test-odb1 ~]# cat /sys/kernel/mm/transparent_hugepage/enabled
[root@test-odb1 ~]# cat /sys/kernel/mm/transparent_hugepage/defreg

if not [never]
[root@test-odb1 ~]# echo never > /sys/kernel/mm/transparent_hugepage/enabled

8. Disable Transparent HugePages on boot

Note:  Oracle highly recommends the use of standard HugePages

Doc ID 1557478.1
ALERT: Disable Transparent HugePages on SLES11, RHEL6, RHEL7, OL6, OL7, and UEK2 and above

How to disable transparent hugepages (THP) on Red Hat Enterprise Linux 7, 8

1. vim grub
[root@test-odb1 ~]# vim /etc/default/grub
GRUB_CMDLINE_LINUX="rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap ... transparent_hugepage=never"

2. backup grub.cfg
[root@test-odb1 ~]# ls /boot/grub2/grub.cfg
[root@test-odb1 ~]# cp /boot/grub2/grub.cfg /boot/grub2/grub.cfg.bak

3. rebuild grub.cfg
On BIOS-based machines: ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
On UEFI-based machines: ~]# grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg

4. reboot
[root@test-odb1 ~]# shutdown -r now

9. Config use_large_pages & Restart Database by Oracle

[oracle@test-odb1 ~]$ sqlplus / as sysdba

SQL> ALTER SYSTEM SET use_large_pages=only SCOPE=SPFILE;
SQL> exit

[oracle@test-odb1 ~]$ srvctl stop database -d testdb
[oracle@test-odb1 ~]$ srvctl start database -d testdb

10. Check HugePage again

[oracle@test-odb1 ~]$ cat /proc/meminfo |grep Huge
AnonHugePages:   1366016 kB
HugePages_Total:    5132
HugePages_Free:     1034
HugePages_Rsvd:     1024
HugePages_Surp:        0
Hugepagesize:       2048 kB

11. Check Oracle alert.log

2021-05-25T09:21:24.624334+08:00
  PAGESIZE  AVAILABLE_PAGES  EXPECTED_PAGES  ALLOCATED_PAGES  ERROR(s)
2021-05-25T09:21:24.624426+08:00
     2048K             5132            5122            5122        NONE
2021-05-25T09:21:24.624463+08:00
 Reason for not supporting certain system pagesizes:
2021-05-25T09:21:24.624498+08:00
  4K - Large pagesizes only