"Next key" and "infinity locks" are used for DOL locked tables under isolation level 3 - e.g. datarows (which sysobjects and other system tables are datarows locked). There should be a description of this in the P&T Locking manual or the Transact SQL Users Guide, but essentially what happens is if you do a query that affects (for example) all the "B*'s", ASE will lock the first "C" row to prevent another "B" from being added. Soooo....I expect that for some reason the transaction that is inserting into #temp is running in ISO 3 - perhaps to make sure the select portion is consistent. I would check to see if that is really necessary as a first step. Often times, this happens with java apps when someone calls a proc and the default java JDBC setting is chained mode...kinda different as it simply means that every DML starts a transaction, but if the connection is in a connection pool and someone didn't clean up after an iso3 query (via set), then the insert into #temp would be in a iso3 transaction. However, a pure insert into #foo would not have held the lock on sysobjects - it likely was a create table #foo sometime earlier (or select/into) - that is why I mentioned chain mode. The insert *might* be the current statement that was being run by the spid holding the lock, but the real cause was the DDL in tran (check to see if that is on as well). One other possibility is that the coder might have done something like:
if exists (select * from sysobjects where name like '#monoFun%')
insert into #monoFun select * from ...
That like clause in the IF statement would be an exact match for the type of query that would drive a "next key" lock under ISO 3.....and would be what I would suspect. Still, not sure why it would be inside a transaction unless in chained mode, etc.......