Search

Top 60 Oracle Blogs

Recent comments

11g Release 2

crsctl status resource – state details are really useful

A very short post about a cool new feature I noticed today. RAC 11.2 has moved a lot of commands previously having their own syntax into crsctl. One of the cool new things is the fact that crsctl status resource -t (“tabular”) reports state details. Here I could see that my lab environment had a stuck archiver. Other state details include information about the cluster time synchronisation daemon ctss, or ASM instances. Have a look at my 4 node cluster:

[oracle@rac11gr2node2 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
 ONLINE  ONLINE       rac11gr2node1                                
 ONLINE  ONLINE       rac11gr2node2                                
 ONLINE  ONLINE       rac11gr2node3                                
 ONLINE  INTERMEDIATE rac11gr2node4                                
ora.LISTENER.lsnr
 ONLINE  ONLINE       rac11gr2node1                                
 ONLINE  ONLINE       rac11gr2node2                                
 ONLINE  ONLINE       rac11gr2node3                                
 ONLINE  ONLINE       rac11gr2node4                                
ora.OCRVOTE.dg
 ONLINE  ONLINE       rac11gr2node1                                
 ONLINE  ONLINE       rac11gr2node2                                
 ONLINE  ONLINE       rac11gr2node3                                
 ONLINE  INTERMEDIATE rac11gr2node4                                
ora.asm
 ONLINE  ONLINE       rac11gr2node1            Started             
 ONLINE  ONLINE       rac11gr2node2                                
 ONLINE  ONLINE       rac11gr2node3                                
 ONLINE  INTERMEDIATE rac11gr2node4                                
ora.eons
 ONLINE  ONLINE       rac11gr2node1                                
 ONLINE  ONLINE       rac11gr2node2                                
 ONLINE  ONLINE       rac11gr2node3                                
 ONLINE  ONLINE       rac11gr2node4                                
ora.gsd
 OFFLINE OFFLINE      rac11gr2node1                                
 OFFLINE OFFLINE      rac11gr2node2                                
 OFFLINE OFFLINE      rac11gr2node3                                
 OFFLINE OFFLINE      rac11gr2node4                                
ora.net1.network
 ONLINE  ONLINE       rac11gr2node1                                
 ONLINE  ONLINE       rac11gr2node2                                
 ONLINE  ONLINE       rac11gr2node3                                
 ONLINE  ONLINE       rac11gr2node4                                
ora.ons
 ONLINE  ONLINE       rac11gr2node1                                
 ONLINE  ONLINE       rac11gr2node2                                
 ONLINE  ONLINE       rac11gr2node3                                
 ONLINE  ONLINE       rac11gr2node4                                
ora.registry.acfs
 ONLINE  ONLINE       rac11gr2node1                                
 ONLINE  ONLINE       rac11gr2node2                                
 ONLINE  ONLINE       rac11gr2node3                                
 ONLINE  ONLINE       rac11gr2node4                                
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
 1        ONLINE  ONLINE       rac11gr2node2                                
ora.LISTENER_SCAN2.lsnr
 1        ONLINE  ONLINE       rac11gr2node4                                
ora.LISTENER_SCAN3.lsnr
 1        ONLINE  ONLINE       rac11gr2node3                                
ora.oc4j
 1        OFFLINE OFFLINE                                                   
ora.poldb.db
 1        ONLINE  INTERMEDIATE rac11gr2node3            Stuck Archiver      
 2        ONLINE  INTERMEDIATE rac11gr2node4            Stuck Archiver      
ora.poldb.drcp.svc
 1        ONLINE  ONLINE       rac11gr2node3                                
 2        ONLINE  INTERMEDIATE rac11gr2node4                                
ora.poldb.nondrcp.svc
 1        ONLINE  INTERMEDIATE rac11gr2node4                                
ora.polstdby.db
 1        ONLINE  INTERMEDIATE rac11gr2node4            Stuck Archiver      
 2        OFFLINE OFFLINE                                                   
ora.prod.batchserv.svc
 1        ONLINE  ONLINE       rac11gr2node2                                
 2        ONLINE  ONLINE       rac11gr2node1                                
ora.prod.db
 1        ONLINE  ONLINE       rac11gr2node1            Open                
 2        ONLINE  ONLINE       rac11gr2node2                                
ora.prod.reporting.svc
 1        ONLINE  ONLINE       rac11gr2node2                                
 2        ONLINE  ONLINE       rac11gr2node1                                
ora.rac11gr2node1.vip
 1        ONLINE  ONLINE       rac11gr2node1                                
ora.rac11gr2node2.vip
 1        ONLINE  ONLINE       rac11gr2node2                                
ora.rac11gr2node3.vip
 1        ONLINE  ONLINE       rac11gr2node3                                
ora.rac11gr2node4.vip
 1        ONLINE  ONLINE       rac11gr2node4                                
ora.scan1.vip
 1        ONLINE  ONLINE       rac11gr2node2                                
ora.scan2.vip
 1        ONLINE  ONLINE       rac11gr2node4                                
ora.scan3.vip
 1        ONLINE  ONLINE       rac11gr2node3                                

Nice!

Display scheduler class for a process in linux

The ps command in the ways I use it most (ps -ef and ps auxwww) doesn’t display the scheduling class for a process. Oracle have cunningly released a patchset to update Grid Infrastructure that changes the scheduling class from the VKTM and LMSn ASM processes to “Timeshare” instead of Realtime.

So far so good, but I had no idea how to display the scheduling class of a process so some man page reading and Internet research were in order. After some digging around I found out that using the BSD command line syntax combined with the “–format” option does the trick. The difficult bit was in figuring out which format identifiers to use. All the information ps can get about a process are recorded in /proc/pid/stat. Parsing this with a keen eye however proves difficult due to the sheer number of fields in the file. So back to using ps (1).

Here’s the example. Before applying the workaround to the patch, Oracle ASM’s VKTM (virtual keeper of time) and LMSn (global cache services process) run with TS priority:

[oracle@rac11gr2node2 ~]$ ps ax --format uname,pid,ppid,tty,cmd,cls,pri,rtprio \
>| egrep "(vktm|lms)" | grep asm
grid      4296     1 ?        asm_vktm_+ASM2               TS  24      -
grid      4318     1 ?        asm_lms0_+ASM2               TS  24      -

After applying the workaround the scheduling class changed:

[oracle@rac11gr2node1 ~]$ ps ax --format uname,pid,ppid,tty,cmd,cls,pri,rtprio | egrep "(vktm|lms)" | grep asm
grid      2352     1 ?        asm_vktm_+ASM1               RR  41      1
grid      2374     1 ?        asm_lms0_+ASM1               RR  41      1

Notice how the cls field changed, and also that the rtprio is now populated. I have learned something new today.