Search

Top 60 Oracle Blogs

Recent comments

hugepages

Kernel panic – not syncing: Out of memory and no killable processes

This is a quick post to give a solution (maybe not the best one as this was just quick troubleshooting) if, at boot, you see something like:
Trying to allocate 1041 pages for VMLINUZ [Linux=EFI, setup=0x111f, size=0x41108d01]
and then:
Kernel panic - not syncing: Out of memory and no killable processes
Pid: 1228 comm: kworker Not tainted 3.8.13-118.17.4.el6uek.x86_64 #2


IMG_E5543
If you do not see the messages, then you may need to remove the ‘quiet’ option of kernel and replace it by ‘nosplash’ – this is done from grub.

If you’re not using hugepages, you’re doing it wrong!

Well, there’s been a bit of a delay in with my planned testing of dbVisit Replicate and Oracle GoldenGate for zero-downtime upgrades. So, I’ll be (hopefully) getting back to that within a couple of weeks.

Meanwhile, I recently ran across a discussion on the Oracle OTN Community forums, asking about performance and hugepages configuration, here in the Oracle Database – General Questions Forum.

I think my answer bears repeating, so, here is a slightly modified version:

First, I’m going to take a strong position on hugepages. I’m going to go as far as to say, for any non-trivial SGA size, if you’re not using hugepages, you’re doing it wrong. There are three main points to consider.

Linux HugePages for Oracle on Amazon EC2

One of the optimizations available to us when running Oracle on Linux is huge page support. This feature of the Linux kernel enables processes to allocate memory pages of size 2M (instead of 4k). In addition, memory allocated using hugepages is pinned in physical memory. It cannot be swapped out.

It is now common practice to enable huge page support for Oracle databases with large SGAs (one rule of thumb is 8G). Without this feature, the SGA can be, and often is, paged out. Paging out portions of the SGA can result in disastrous consequences from a performance standpoint. There are a variety of load patterns that perform particularly poorly without hugepages. Running with large numbers of processes, sudden increases in processes (connection storms), and highly concurrent access of diverse sets of SGA pages all can bring an Oracle system without hugepages to its knees.

Configuring Linux Hugepages for Oracle Database Is Just Too Difficult! Isn’t It? Part – I.

Allocating hugepages for Oracle Database on Linux can be tricky. The following is a short list of some of the common problems associated with faulty attempts to get things properly configured:

  1. Insufficient Hugepages. You can be short just a single 2MB hugepage at instance startup and Oracle will silently fall back to no hugepages. For instance, if an instance needs 10,000 hugepages but there are only 9,999 available at startup Oracle will create non-hugepages IPC shared memory and the 9,999 (x 2MB) is just wasted memory.
    1. Insufficient hugepages is an even more difficult situation when booting with _enable_NUMA_support=TRUE as partial hugepages backing is possible.
  2. Improper Permissions. Both limits.conf(5) memlock and the shell ulimit –l must accommodate the desired amount of locked memory.

In general, list item 1 above has historically been the most difficult to deal with—especially on systems hosting several instances of Oracle. Since there is no way to determine whether an existing segment of shared memory is backed with hugepages, diagnostics are in short supply.

Oracle Database 11g Release 2 (11.2.0.2)
The fix for Oracle bugs 9195408 (unpublished) and 9931916 (published) is available in 11.2.0.2. In a sort of fast forward to the past, the Linux port now supports an initialization parameter to force the instance to use hugepages for all segments or fail to boot. I recall initialization parameters on Unix ports back in the early 1990s that did just that.

The initialization parameter is called use_large_pages and setting it to “only” results in the all or none scenario. This, by the way, addresses list item 1.1 above. That is, setting use_large_pages=only ensures an instance will not have some NUMA segments backed with hugepages and others without.