Thanks,

What you probably meant was v$session_wait.state (statE, not statUS) which shows the wait state, whether you’re waiting for something or not. The statE column is also in v$session starting from 10g, but they’re different. STATE shows whether you’re waiting/not on CPU or working/ON CPU, but STATUS shows whether the session is servicing a user request or not.

Regarding systemstate dumps, yes it’s possible to get “unlucky” so that when the dump is taken the session just happens to be doing something else than the problem operation… this is in case you don’t have a hang but just bad performance. That’s why you should take a few systemstate dumps to see if anything is changing over time. Note that systemstate dump is quite expensive, better use @sw/v$session_wait if just one session is slow and hanganalyze at level 3 if there seems to be a hang affecting multiple sessions.

I will post a little script in my blog for information gathering hopefully today, but what I usually gather first for a problem session is v$session_wait, v$sesstat and exec plan info from v$sql_plan.

I think this explanation answers why I don’t see the same address in the dump file.

I do see in dump , session info shows as last wait..however, I was checking the v$session status which was active.
From my observation, if the session is waiting
on anything in the database, the v$session status shows as active ( I might be wrong ).

So is it possible that during the dump duration when it was scannig this session, the session was doing some work?

Now about next time, when this happens, if I use sw.sql and v$sesstat, I think I should get more info on this issue.
However, do you forsee any other info to be collected – just based on what you have seen?
Your insight is greatly appreciated. Thanks,

That’s why I use my sw.sql script which masks the last event name and replaces it with “on cpu” when not waiting.

And systemstate dump confirms that as well, if you were waiting you’d see “waiting for …” in there instead of “last wait …”

I would look into v$sesstat counter of that session (with snapper for example) and see what counters are incrementing (especially whether the execute count is going up) and then see into the execution plan of the prevalent SQL_ID/sql_hash_value in v$session for that session. This may be caused by just a bad execution plan, but snapper would confirm it.

Info: This is what I have in my dump file for this latch free waiting session..I did a grep on address 380010668 in the entire dump file and I don’t see any reference to this address. So I am puzzled ..did I use wrong address?

*******************************************

SO: 2760886dd8, type: 4, owner: 276c962000, flag: INIT/-/-/0×00
(session) trans: 27ef7a0c08, creator: 276c962000, flag: (100041) USR/- BSY/-/-/-/-/-
DID: 0001-03F0-00000002, short-term DID: 0000-0000-00000000
txn branch: 0
oct: 2, prv: 0, sql: 27d892ab90, psql: 27d892ab90, user: 79/XYZUSER
O/S info: user: xxxeng04, term: , ospid: 25364, machine: xxxxap4p
program: exxxxeng@xxxxap4p (TNS V1-V3)
application name: yyyyyy4, hash value=1554640259
action name: Thread 25 (type T4: Object-Histo, hash value=1445968872
last wait for ‘latch free’ blocking sess=0×0 seq=30610 wait_time=204
address=380010668, number=58, tries=0
temporary object counter: 5

*******************************************

Yes, it is system state dump. I checked the session_wait for that particular session and it was latch_free event..latches were ‘cache buffers chains’ and ‘mostly latch-free SCN’. This session has been running almost about 16 hrs. Again at the system level db was normal. My db version is 9208 on SunOS 64 bit.I think I had seen an example on your site where you were showing how to track back to the latch blocking session.

Thanks,

Thanks,

How did you conclude that you had latching problem for that session?

From systemstate dump you can search for “latch:” or “latch free” (depending on db version and latch type) to find the waiters and latch address and then you can look the latch holder up by that latch address.

But before going there, tell me how exactly (what view, what columns) you concluded that there was a latch free problem in the system?

Today I had this issue of latch_free waits for one of the session. I have a database system dump using oradebug dump systemstats. I believe there is a way to locate the latch blocking session from looking at dump file. Can you please guide me how to do this?
I appreciate your help.

Thanks

How did you determine there were long waits for that latch? Did you look into v$session_wait.event perhaps *without* realizing that the state column was not waiting? Unfortunately Oracle doesn’t clear up the last wait event name when going back onto CPU, so that often misleads people (including me, that’s why I use a case statement to remove the wait event name when not waiting anymore in my sw.sql script)

Your tkprof script shows CPU time as big as elapsed time so there should be no significant waits going on (did you enable sql_trace with waits=>true – in that case tkprof should also report total time spent waiting for different events)