Note that if some latch is held for a lot of time it’s not necessarily a problem. Only troubleshoot latch contention of anyone is actually waiting for that latch!

NAME KSLLWNAM KSLLWLBL Held Gets Held % Held ms Avg hold ms
———————————– —————————————- ——————– ———- ———- ——- ———– ———–
cache buffers chains kcbgtcr: fast path 1697 1697 16.97 1213.355 .715
cache buffers chains kcbgtcr: kslbegin excl 551 551 5.51 393.965 .715
cache buffers chains kcbrls: kslbegin 160 160 1.60 114.400 .715
resmgr group change latch kskrefreshattr 144 144 1.44 102.960 .715
cache buffers chains kcbgtcr: kslbegin shared 86 86 .86 61.490 .715
cache buffers chains kclcls_3 86 86 .86 61.490 .715
simulator lru latch kcbs_simulate: simulate set setid 85 85 .85 60.775 .715
redo copy kcrfw_redo_gen: nowait 71 71 .71 50.765 .715
cache buffers chains kcbgtcr ptr to buffer header 61 61 .61 43.615 .715
library cache kglpnp: child child 38 38 .38 27.170 .715
library cache kglpndl: child: after processing latch 37 37 .37 26.455 .715
library cache kglpndl: child: before processing latch 32 32 .32 22.880 .715
cache buffers lru chain kcbzgws 30 30 .30 21.450 .715
session idle bit ksupuc: set busy session ptr 27 27 .27 19.305 .715
client/application info kskirefrattrmap 26 26 .26 18.590 .715
ges resource hash list kjakcai: search for resp by resname 25 2 .25 17.875 8.938
cache buffers chains kcbgcur: kslbegin 19 19 .19 13.585 .715
ges resource hash list kjucvl: lock convert request 19 19 .19 13.585 .715
cache buffers chains kcbget: pin buffer 14 14 .14 10.010 .715
library cache pin kglpnp: child child 14 14 .14 10.010 .715
ASM map operation hash table kffmTranslate 14 14 .14 10.010 .715
gcs resource hash kjblocalfile_nolock 14 14 .14 10.010 .715
ASM map operation hash table kffmDoDone_1 13 13 .13 9.295 .715
process allocation ksufap: active procs 13 13 .13 9.295 .715
gcs resource hash kclzcl 11 11 .11 7.865 .715
row cache objects kqrpre: find obj 11 11 .11 7.865 .715
cache buffers chains kcbzib: finish free bufs 10 10 .10 7.150 .715
library cache lock kgllkdl: child: cleanup latch 9 9 .09 6.435 .715
session idle bit ksupuc: clear busy session ptr 9 9 .09 6.435 .715
library cache pin kglpndl child 8 8 .08 5.720 .715
cache buffers chains kcbnew: new latch again 8 8 .08 5.720 .715
library cache kglLockCursor 7 7 .07 5.005 .715
gcs resource hash kjbopen:affinity 7 7 .07 5.005 .715
gcs resource hash kjbcropen:affinity 7 7 .07 5.005 .715
library cache lock kgllkal: child: multiinstance latch 7 7 .07 5.005 .715
enqueue hash chains ksqgtl3 acquiring session 7 7 .07 5.005 .715
library cache kglobpn: child: latch 6 6 .06 4.290 .715
library cache kgllkdl: child: no lock handle latch 6 6 .06 4.290 .715
SQL memory manager workarea list la qesmmIRegisterWorkArea 6 6 .06 4.290 .715
client/application info kqp327i 6 6 .06 4.290 .715
shared pool simulator kglsim_unpin_simhp: fast path 6 6 .06 4.290 .715
enqueues ksqgel: create enqueue parent obj 6 6 .06 4.290 .715
ges resource hash list kjrref: find matched resource 5 5 .05 3.575 .715
name-service request queue kjxgnapq: process a request 5 5 .05 3.575 .715
SQL memory manager latch qesmmIDeamonCb 5 3 .05 3.575 1.192
cache buffers chains kcbchg: kslbegin: bufs not pinned 4 4 .04 2.860 .715
KCL gc element parent latch kclnfndnewm 4 4 .04 2.860 .715
cache buffers chains kcbbxsv 4 4 .04 2.860 .715
gcs partitioned table hash 27 4 4 .04 2.860 .715
ges process parent latch kjatioc 4 4 .04 2.860 .715
row cache objects kqreqd 4 4 .04 2.860 .715
ges resource hash list kjrmas1: lookup master node 3 3 .03 2.145 .715
ges process parent latch kjlalc: lock allocation from lmon 3 3 .03 2.145 .715
session allocation ksufap: active sessions 3 3 .03 2.145 .715

Hi Richard,

Can you paste the output?

I calculate the “Avg hold ms” by checking how fast does the GETS increase during the sampling loop. If each iteration of the V$LATCHHOLDER/X$KSUPRLAT scan takes 607 microseconds, then the max precision of the “Avg hold ms” I can report is 607 microsecond. So in reality the Avg hold ms may be even smaller. The only way to have better precision would be to sample faster, but my script is already sampling at the max speed (the bigger the V$PROCESS array is, the longer it takes to sample).

However, this is not a problem, the main column to look at is how busy a latch is in total (the Held %) and other columns are just extra supporting information.

Great script!

I have a question: any idea why I see (consistently) “.607″ for the “Avg hold ms”?

All other values in the output columns appear normal.

Regards – Richard

Even though your scripts are meant for single instance , it can be very well used for RAC instances as well as for parallel queries.
In fact, I regularly use your scripts for troubleshooting. These are what I do.

1. stored all the relevant scripts like snapper , latchprof, waitprof etc.. in a directory of one node [say node 1] of the cluster.
2. created tnsnames entry for all the other nodes from the node [node 1] where all the scripts are stored.
3. Log into the instances in node 1,find the issue in any node by using any of the views like gv$session or gv$session_wait or gv$session_event.
For example , this query gives an overview of any wait event in any of the instance:
select inst_id inst, sid, module, action, osuser, username, logon_time, event, seconds_in_wait sec from gv$session where username is not null and event not like ‘SQL*Net%’ and seconds_in_wait > 10 order by seconds_in_wait;

4. Create a remote sql session to affected instance using the tnsname created in step 2 from the directory of node 1 where all the scripts are stored in step 1.
And use the relevant script to get to the root of the issue.

In my opinion all your scripts are very much relevant for all environments with just a bit tweak here and there.

—Raj

But if you ask whether these scripts gather info from all RAC nodes, then no. It’s not needed either as latches aren’t global, they’re instance-local memory structures only…