Comments on: Advanced Oracle Troubleshooting Guide: When the wait interface is not enough [part 1]

By: Alter system kill session and ORA-00031: session marked for kill | Tanel Poder's blog: Core IT for Geeks and Pros

Thu, 13 Aug 2009 07:22:40 +0000

[...] target session to die – what is the wait event you are waiting on then? Every wait, except few bugs, should be instrumented in [...]

By: Tanel Poder

Tanel Poder — Sat, 10 Jan 2009 09:41:41 +0000

Hi Martin,

Yeah, starting from 11.1.0.7 (and probably 10.2.0.5 but I haven’t checked) there are new wait events related to external table access:

SQL> @sed “external table”

EVENT# EVENT_NAME PARAMETER1 PARAMETER2 PARAMETER3 ------ --------------------------------------------- ------------------------------ ------------------------------ ---------- 293 external table misc IO filectx iocode P3 291 external table read filectx file# size 292 external table write filectx file# size

Btw, I don’t have to ask, I *know* that it’s fun to read old (quality) blog entries on Saturday evenings! ;)

By: Martin Berger

Martin Berger — Fri, 09 Jan 2009 20:42:05 +0000

You should just have waited for 18 months or more and change the Release (or wait any longer for 10.2.0.5).
Now Preprocessor For External Tables is out and should solve all these problems (whereas it should be still checked how it’s shown in wait events).
Just for the records (I’m sure you know it): http://structureddata.org/2008/11/19/preprocessor-for-external-tables/
Before you ask: yes, it’s fun to read old blog entries on Saturday evening! ;-)

By: jiulu sun

jiulu sun — Mon, 08 Dec 2008 20:44:24 +0000

nice work, where can i find your article of “I will deal with more complex problems like what to do when the session is not reporting significant waits and is spinning heavily on CPU”, that seems exactly happened on my system.

By: My Blog is 1 year old! 8-) « Tanel Poder’s blog: Core IT for geeks and pros

Fri, 20 Jun 2008 17:05:46 +0000

[...] wrote my first real blog entry on 18th June 2007, about advanced Oracle troubleshooting in cases where conventional methods are [...]

By: Car

Car — Fri, 11 Apr 2008 14:53:25 +0000

Based on your review no 16, to whom SECONDS_IN_WAIT that accounted when not waiting at all (constantly on cpu) belongs ? How do you got that ? Please clarify ?

By: Log Buffer #60: a Carnival of the Vanities for DBAs · Steve Karam · The Oracle Alchemist

Fri, 31 Aug 2007 16:03:28 +0000

[...] I can’t resist going back to Oracle. Tanel Poder has posted an outstanding article on advanced Oracle troubleshooting when the wait interface is not enough. Consequently, this is his first post on the new blog, and I hope to see more great articles like [...]

By: Advanced Oracle Troubleshooting Guide, Part 2: No magic is needed, systematic approach will do « Tanel Poder’s blog: Core IT for geeks and pros

Sun, 26 Aug 2007 17:52:03 +0000

[...] is, even if the session statistics and wait interface are not enough (as I started blogging in my previous post in this [...]

By: Case Study: Statspack/AWR Latch Waits (Part 2) : Ardent Performance Computing

Case Study: Statspack/AWR Latch Waits (Part 2) : Ardent Performance Computing — Fri, 29 Jun 2007 16:21:59 +0000

[...] read a great quote from Tanel Poder’s blog recently… I think I’ll call it Tanil’s dictum: “lower-level instrumentation always has a better chance to know what’s really going on at [...]

By: tanelp

tanelp — Sun, 24 Jun 2007 12:19:53 +0000

This IS a bug as I have proven already. Oracle IS WAITING in a OS read syscall and the only correct state in V$SESSION_WAIT.STATE is WAITING, regardless of values in any other columns. This contradiction alone proves that Oracle behaves incorrectly.

Regarding what documentation says, the section about wait interface in Oracle has always been lacking concreteness, thus the myths about having to use the WAIT_TIME and such. Just the fact that something is documented, doesn’t make it a fact ;-)

The SECONDS_OF_WAIT is an unfortunate misnaming, it should really be called SECONDS_IN_STATE, as it’s just the time since last wait state change (e.g. from waiting to not-waiting or vice versa). It’s measured and incremented even if being constantly on CPU without any waiting at all. Every time LGWR runs, it calculates the time delta since last wait state change for each session and updates the fixed table under V$SESSION_WAIT.

Normally when you enter a wait, this wait state change timestamp is reset and SEQ# is incremented. What happens in my test case, however, is that both last SEQ# and SECONDS_IN_WAIT are not changed at all, their values persist (the latter continues to increment, is not zeroed).

I suspect this lack of instrumentation is because external tables use an ODCI cartridge for loading data, which in turn call “trusted” external sqlldr library functions (which however are linked in to oracle binary on Solaris). Perhaps noone wanted to change the generic sqlldr functions to include wait instrumentation, perhaps there are limitations altering sessions wait state from external library, perhaps it was just missed…

Note that the wait event should not have been a db file scattered nor sequential read as the external tables do not read from datafiles to buffer cache, they just invoke ODCI cartride, which invokes a method from object type (ORACLE_LOADER in our case) which in turn call a function in external library (qxxq_fetch in qxxqlib) for reading the data from text file using sqlloader functions. The returned data is presented as a rowsource to the consumer who invoked external table read. Now that I look into it, I don’t even see a wait event anything named “external” or ODCI related. So perhaps it’s a design flaw.