The procedure cache filled up again.
Luckily I was monitoring it at the time...
select DBName, ObjectName, sum(MemUsageKB) / 1024 MemMB, count(*) c
from master..monCachedProcedures
group by DBName, ObjectName
having sum(MemUsageKB) / 1024 > 10
order by 3
and it was only showing less than 2Gb usage
DBName MemMB count
tempdb 145 4862
DB3 178 597
DB2 334 1403
DB1 846 1803
I was monitoring it in a loop and the total space in monCachedProcedures started to decrease.
until I got the message
Msg 701, Level 17, State 3
Server 'UAT_SQL', Line 1
There is not enough procedure cache to run this procedure, trigger, or SQL batch.
Retry later, or ask your SA to reconfigure ASE with more procedure cache.
I ran
dbcc procbuf
and this only shows 257 objects in the procedure cache.
I ran
dbcc proc_cache(free_unused)
and this showed
Procedures in cache before free unused: 258
Procedures in cache after free unused: 212
We've had this a few times on 2 or our non-production servers but never on production.
All machines are the same expect Production runs in "process mode" and non-production runs in "thread mode". (We were re-testing thread mode as we had to turn it off as it was unstable).
It seems from this that monCachedProcedures doesn't show everything thats in the the procedure cache ?
Running select * from master..monProcedureCacheModuleUsage
shows
InstanceID ModuleID Active HWM NumPagesReused ModuleName
0 5 831 6308111 0 Execution
0 10 282868 1097270 1805401 Procedural Objects
0 1 14 236915 0 Parser
The HWM of the "Exectuion" module shows 6m pages - which is probably what fills up the procedure cache.
What populates the this ?
Will log this as an incident.