As far as I know it should affect only SYSTEM tablespace, mainly because critical stuff – data dictionary is in there. In SYSAUX you don’t have such critical stuff but rather just some supporting things..

The top of the stack says these 711 samples (of 2920) were spent allocating memory (and not doing block checking)

The reason is that connect by queries are recursive and if you run a query with connect by level <100000, Oracle needs to keep allocating memory for cursor work heap.

I’ve blogged about it previously:

http://blog.tanelpoder.com/2008/06/08/generating-lots-of-rows-using-connect-by-safely/

You could run your test with the query I proposed there to elimiate the memory allocation issue.

Another reason why the block checking code may be less visible in your test case is that with bulk inserts the block checking can probably be done in a block at a time “bulk fashion” as well but for a tight PL/SQL loop it has to be done after each row insert separately.

The second stack with 211 samples is Solaris kernel pagefault code, which initializes zero-filled pages for your process when they’re touched the first time.

I have tested the example and works as you say with the associated problem. However, when the insert is done like this:

insert into t2 select rownum from dual connect by level <= 100000;

The output generated is (the two last samples):

221 samples with stack below
__________________
unix`hwblkclr
unix`pagezero
genunix`anon_zero
genunix`segvn_faultpage
genunix`segvn_fault
genunix`as_fault
unix`pagefault
unix`trap
unix`_cmntrap

711 samples with stack below
__________________
kghbshrt
kghalo
kghgex
kghfnd
kghprmalo
kghalp
kxsWorkHeapAllocPerm
qercbiPushState
qercbiFetch
qercoFetch
rwsfcd
insfch
insdrv
inscovexe
insExecStmtExecIniEngine
insexe
opiexe
kpoal8
opiodr
ttcpip
opitsk
opiino
opiodr
opidrv
sou2o
opimai_real
main
0xd7de4c

2920 Total samples captured

Obviously the way of the insert done makes sense, but why? Isn´t there block checking in direct-path-insert as the checking is done in the buffer cache?

Thank you for this wonderful blog Tanel!

Thanks again.

K.Senthil Murugan

I recommend reading this to understand Oracle and Unix touchpoint well:

http://www.scaleabilities.co.uk/book/scalingOracle8i.pdf (it’s free – and ignore that the name says 8i in title, the core concepts have remained the same)

And then get this book: http://antognini.ch/top/

Did you notice any significant differences in Oracle Wait Interface times or v$sesstat counters between the versions?

I would always start from these before looking into stack sampling. Also, dstacktrace samples only the processes currently on CPU, so if your process waits/sleeps on something, dstackprof doesn’t show any samples for those sleeps.

If you have already validated with Oracle Wait Interface data that your process is not waiting/sleeping majority of time, then can you post few of the bottom dstacktrace stack samples from both 11g and 10gR2?

Good that you did this post. i had similar issue with 10gr2 and 11g, where the inserts taking 3x times more cpu than 10gr2. Didn’t see any waits and DB time was equivalent to DB cpu. Ran the dtraceprof on 11g and noticed the following which didn’t occur in 10gr2. Any idea what these functions are? Wondering if this has anything to do with the 3x cpu usage.

`qesltcLoadIndexList
`qesltcLoadIndexes
`qerltcNoKdtBufferedInsRowCBK
`qerltcLoadStateMachine
`qerltcInsertValuesRop
‘qerltcFetch

Thanks.