Hi,
In the application why not do a query timeout? You will have to catch the exception and move the application forward. setQueryTimeout(seconds)
Is the stored procedure on the ASE?
Is the stored procedure dyanamic and made from the application?
Why the stored procedure on the ASE is still taking so long would be something to look at.
Maybe the dba on the ASE should do some monitoring on the stored procedure and tune it if the procedure is located on the ASE.
Thanks,
Dawn Kim
Dawn y
Your suggestion of query timeout is excellent.
In the original post I do not see any mention of stored procedure on server side !
So it seems to be a case of execution of dynamic SQL getting out of control !
Khoa Tran
There is a global variable @@spid to get spid for each connection. You need to get it before you start your big query execution. Use it with caution for killing the spid later if needed.
Please do not use sp_who or select from master..sysprocesses because these can aggravate the situation if CPU usage is already high.
HTH
Avinash
SP2 will be released in September.
There are new features in this release?
Hi Jeff,
After execute the batch, I got the issue and these are the results to your questions:
You also said that there wasn't any long running open txns. As you are aware, of course, checkpoint can't truncate passed the truncation point (oldest open txn)....it might be interesting to see where things are with respect to that....you might want to look up loginfo() and during one of the batches do something like:
declare @n int
set @n=0
while @n<=1000
begin
select loginfo(<dbname>,first_page) as first_page,
loginfo(<dbname>,checkpoint_page) as checkpoint_page,
loginfo(<dbname>,oldest_active_transaction_page) as oldest_txn_page
waitfor delay '00:00:01'
select @n=@n+1
end
go
These are three rows I got before the issue came up, this is the ouput:
Hora first_page checkpoint_page oldest_active_transaction_page
------------------------------ -------------------- -------------------- ------------------------------
Jul 24 2015 12:05:42:853PM 80638734 80639039 80700044
Jul 24 2015 12:05:43:853PM 80638734 80639039 80702299
Jul 24 2015 12:05:44:870PM 80638734 80639039 80704142
When the issue came up, as you can see, the rows doesn't change and the transaction log began to grow.
Hora first_page checkpoint_page oldest_active_transaction_page
------------------------------ -------------------- -------------------- ------------------------------
Jul 24 2015 12:54:17:500PM 80638734 80639039 0
Jul 24 2015 1:02:38:650PM 80638734 80639039 0
During the issue, I executed the next commands but I got and error:
use master
go
1> use master
2> go
1> sp_dboption base_test, "trunc log on chkpt", false
2> go
Msg 2753, Level 20, State 1:
Server 'xxxxx', Procedure 'sp_dboption', Line 1264:
Received an error code of '-158' from 'intl_strblist()'.
Msg 2753, Level 20, State 1:
Server 'xxxxx', Procedure 'sp_dboption', Line 1294:
Received an error code of '-158' from 'intl_strblist()'.
(return status = 0)
go
1> use base_test
2> go
1> dbcc traceon (3604)
2> go
DBCC execution completed. If DBCC printed error messages, contact a user with System Administrator (SA) role.
1> dbcc dbtable ('base_test')
2> go
ACTIVE DBTABLES:
DB Table at: 0x1af865580
dbt_dbid=6 dbt_stat=0x2208 (0x2000 (DBT_DEFNULL), 0x0200 (DBT_DDLINTRAN), 0x0008 (DBT_AUTOTRUNC))
dbt_extstat=0x0 (0x00000000)
dbt_stat2=0x1 (0x0001 (DBT2_LOGFULLABT))
dbt_stat3=0x20000 (0x00020000 (DBT3_SYSPARTITIONS_EXISTS))
dbt_stat4=0x4000 (0x00004000 (DBT4_ATOMIC_OPS_ENABLED))
dbt_stat5=0x0 (0x00000000)
dbt_runstat=0x0(0x0000)
dbt_state=0x2(0x0002 (DBST_ACTIVE)) dbt_keep=17 dbt_hdeskeep=0 dbt_next=0x1af85d500 dbt_systask_keep=0 dbt_detachxact_keep 0
dbt_dcompver_default=1
The only way to empty the log is executing "dump tran with truncate_only"
Although the parameters are configured like you asked me, after I got the issue, sysmon doesn't work any more. Never finish.
exec sp_configure 'stack size', 194560
exec sp_configure 'stack guard size', 20480
When I execute a query on MDA's I got this output:
Insufficient memory is available to allocate structures needed to access a remote site.
Msg 11223, Level 16, State 9:
Insufficient memory is available to allocate structures needed to access a remote site.
I attached additional information I want to share with you.
Thanks in advance for your help.
Juan Carlos.
I have a simple test about insert speed between ASE1570 and ASE1254.
Any idea to improve the throughput about ASE1570.
Testing Env:
ASE1570 SP130
ASE1254 ESD11.2
running on the same box.
Here's the testing steps:
use tempdb
go
create table MM (
MM01 int not null ,
MM02 numeric(8,0) identity ,
MM03 char(1) not null ,
MM04 char(8) null ,
MM05 datetime not null ,
MM06 char(7) not null ,
MM07 char(5) not null ,
MM08 char(1) not null ,
MM09 char(7) not null ,
MM10 char(7) not null ,
MM11 char(1) not null ,
MM12 char(20) not null ,
MM13 char(10) null ,
MM14 char(10) null ,
MM15 char(1) not null ,
MM16 char(1) null ,
MM17 char(1) null ,
MM18 char(1) not null ,
MM19 char(4) not null ,
MM20 char(1) null ,
MM21 char(3) null ,
MM22 char(1) null ,
MM23 smallmoney not null ,
MM24 smallmoney null ,
MM25 smallmoney null ,
MM26 smallint not null ,
MM27 smallint null ,
MM28 smallint null ,
MM29 int null ,
MM30 int null ,
MM31 smallint null ,
MM32 char(1) null ,
MM33 char(1) null ,
MM34 char(1) null ,
MM35 char(1) null ,
MM36 char(1) null ,
MM37 char(7) not null ,
MM38 char(1) not null ,
MM39 char(6) not null ,
MM40 char(5) not null ,
MM41 char(1) not null ,
MM42 char(7) not null ,
MM43 char(1) not null ,
MM44 char(6) not null ,
MM45 char(5) not null ,
MM46 char(1) null ,
CONSTRAINT PK_MMTSF PRIMARY KEY NONCLUSTERED ( MM01,MM02 ))
go
edit a sql file for example 1.sql:
use tempdb
go
truncate table MM
go
select getdate() as DT into #TT
go
INSERT INTO MM (MM01,MM03,MM04,MM05,MM06,MM07,MM08,MM09,MM10,MM11,MM12,MM13,MM14,MM15,MM16,MM17,MM18,MM19,MM20,MM21,MM22,MM23,MM24,MM25,MM26,MM27,MM28,MM29,MM30,MM31,MM32,MM33,MM34,MM35,MM36,MM37,MM38,MM39,MM40,MM41,MM42,MM43,MM44,MM45,MM46) VALUES
(1,'I','549 ','2015-01-01 10:08:17.306','A100000','ZZ001','O','A100000','0000333','1','TTO0100000 ',' ',' ','S',' ',' ','0','F905','S','000','L',100.0000,0.0000,0.0000,10,0,0,10,0,0,'O','O',' ','I',' ','TXO ','C','201508','09000',' ',' ',' ',' ',' ','1')
go 5000
select datediff(ms,DT,getdate()) from #TT
go
and using isql to exec the 1.sql:
the result will look like:
ASE1570
/home/sybase/ase/phsu > isql -Usa -SSYBASE -Psybase -i1.sql
(1 row affected)
(1 row affected)
5000 xacts:
-----------
6176
ASE1254
/home/sybase/ase/phsu > isql -Usa -SASE1254 -P -i1.sql
(1 row affected)
(1 row affected)
5000 xacts:
-----------
3476
We can see ASE1254 is faster then ASE1570, any idea to improve ASE1570 throughput?
Hi Dawn Kim,
Your suggestion is about using setQueryTimeout of JDBC is correct, I have set same timed out for JDBC call, when the timed out hit. I checked the spid status via Processes screen of Sybase Central and saw the value of command column has changed from OPEN_CURSOR to AWAITING COMMAND, and CPU did not spike 100% any more.
In my thought, we set the timed out for JDBC call and then it is applied at transaction level, when the transaction is timed out, query will also go away too. We don't need to terminate the spid as my first thought, because it may cause some potential issue. Please correct me If I am wrong
I'm really appreciate for your help.
Thanks
Khoa Tran
Thanks Avinash,
You are correct about the dynamic SQL, basically our product support a WEB UI that allow user to define the search criteria and then use it for querying database. With more than 10 million records, the query seem to run over and over although timed out has already occurred on WEB UI. That is the reason why I need a way to stop the query when timed out is hit.
As Dawn's suggestion has solved my problem by using setQueryTimeOut function of JDBC without killing the spid. And I think that is safe way.
Thanks for so much for your advice about sp_who/master...sysprocessess . I will pay attention on them in the future.
Thanks
Khoa Tran
Curiously, when I click on the link in the posting above, I get a page indicating the KBA document has been deleted. However, if I enter the URL into the browser directly, it brings up the KBA. I have no idea why this is happening.
-bret
Hi Dave,
Do you have ASE exact version (select @@version). Also, post the output from sp_version, just from the Script value " OLEDB MDA Scripts".
I tried ASE 16.0 SP00PL03, using 16.0.0.2 Provider and my test worked okay.
I did create the table on mssql either as nullable or not null. In all cases I was successful in the bulk copy.
My thought is something is out of whack in the metadata - ASE Provider calls sp_oledb_columns to gather this information.
Also, provide the version info on MSSQL Server and the DDL for creating the table in case there is something there.
Cheers,
-Paul
CR 782596 exists in SP134, the fix for CR 782596 has been checked into 15.7 SP135 and higher and into 16.0 SP01 PL02 and higher.
I've updated and republished the KBA, hopefully that will correct the problems with it no longer showing up as valid.
-bret
HI Paul,
select @@version gives:
Adaptive Server Enterprise/16.0 GA PL01/EBF 22540 SMP/P/X64/Windows Server/ase160sp00pl01/3523/64-bit/FBO/Tue Apr 15 19:04:17 2014
exec MyDb..sp_oledb_columns @table_qualifier = "MyDb", @table_owner = "dbo", @table_name = "target" gives:
TABLE_CATALOG,TABLE_SCHEMA,TABLE_NAME,COLUMN_NAME,COLUMN_GUID,COLUMN_PROPID,ORDINAL_POSITION,COLUMN_HASDEFAULT,COLUMN_DEFAULT,COLUMN_FLAGS,IS_NULLABLE,DATA_TYPE,TYPE_GUID,CHARACTER_MAXIMUM_LENGTH,CHARACTER_OCTET_LENGTH,NUMERIC_PRECISION,NUMERIC_SCALE,DATETIME_PRECISION,CHARACTER_SET_CATALOG,CHARACTER_SET_SCHEMA,CHARACTER_SET_NAME,COLLATION_CATALOG,COLLATION_SCHEMA,COLLATION_NAME,DOMAIN_CATALOG,DOMAIN_SCHEMA,DOMAIN_NAME,DESCRIPTION
'MyDb','dbo','target','e_type_id',,,1,false,,16,false,131,,,,8,0,,,,,,,,,,,
On MsSql select @@version gives:
Microsoft SQL Server 2012 - 11.0.2100.60 (X64)
Feb 10 2012 19:39:15
Copyright (c) Microsoft Corporation
Standard Edition (64-bit) on Windows NT 6.1 <X64> (Build 7601: Service Pack 1)
DDL & insert:
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
SET ANSI_PADDING ON
GO
CREATE TABLE [dbo].[T_SEC_EXCHANGE](
[exchange_id] [int] NOT NULL,
[exchange_type_id] [numeric](8, 0) NOT NULL,
[code] [varchar](50) NOT NULL,
[name] [varchar](40) NOT NULL,
[open_time] [datetime] NULL,
[close_time] [datetime] NULL,
[last_action_by] [int] NOT NULL,
[last_action_datetime] [datetime] NOT NULL,
[SYSTEM_INSERTED] [datetime] NULL,
[SYSTEM_UPDATED] [datetime] NULL,
[SYSTEM_CHANGEDBY] [nvarchar](50) NULL,
[SYSTEM_PRIORITY] [int] NULL
) ON [PRIMARY]
GO
SET ANSI_PADDING OFF
GO
ALTER TABLE [dbo].[T_SEC_EXCHANGE] ADD CONSTRAINT [DF__T_SEC_EXC___4E0988E7] DEFAULT (getdate()) FOR [SYSTEM_INSERTED]
GO
ALTER TABLE [dbo].[T_SEC_EXCHANGE] ADD CONSTRAINT [DF__T_SEC_EXC___4EFDAD20] DEFAULT (getdate()) FOR [SYSTEM_UPDATED]
GO
ALTER TABLE [dbo].[T_SEC_EXCHANGE] ADD CONSTRAINT [DF__T_SEC_EXC___4FF1D159] DEFAULT ('UNKNOWN') FOR [SYSTEM_CHANGEDBY]
GO
ALTER TABLE [dbo].[T_SEC_EXCHANGE] ADD CONSTRAINT [DF__T_SEC_EXC___50E5F592] DEFAULT ((1)) FOR [SYSTEM_PRIORITY]
GO
INSERT INTO [dbo].[T_SEC_EXCHANGE]
([exchange_id]
,[exchange_type_id]
,[code]
,[name]
,[open_time]
,[close_time]
,[last_action_by]
,[last_action_datetime]
,[SYSTEM_INSERTED]
,[SYSTEM_UPDATED]
,[SYSTEM_CHANGEDBY]
,[SYSTEM_PRIORITY])
VALUES
(
0, 1, 'XML', 'OMLX THE LONDON SECURITIES AND DERIVATIV', NULL, NULL, 936,'2004-02-25 09:36:23.177',
'2015-07-24 11:27:00.527','2015-07-24 11:27:00.527','dave', 1)
Additionally, the method which gets the data from MsSql is:
public static void GetDataReaderSelect_t_sec_exchange(out SqlConnection conn, out IDataReader reader)
{
conn = new SqlConnection(@"Data Source=LOCALHOST;Initial Catalog=MySqlDb;Integrated Security=SSPI");
conn.Open();
string sql = "SELECT top 1 exchange_type_id from t_sec_exchange order by 1 desc"; //exchange_type_id is a NUMERIC (8,0) in SqlSvr.
Console.WriteLine("Reader: " + sql);
SqlCommand cmd = new SqlCommand(sql, conn);
cmd.CommandTimeout = 60;
reader = cmd.ExecuteReader();
}
Dave
Hi Dave,
Yes the version you have should have the fix for that old cr.
Paul is looking at this.
Can you give us the output of sp_version from the ASE?
Thanks,
Dawn Kim
Hi Dave,
So what is the version of the driver you are using?
I am thinking since Paul works, it might just mean you need to update the MDA scripts on the ASE.
Give the output of sp_version from the ASE.
ref: Installing the MDA scripts for the drivers - SAP Connectivity - SCN Wiki
See if the scripts in your current directory are older than your current driver version 16.0.2.
If they are update these scipts.
If you still have the issue I will find the version that Paul has and I will upload them.
Thanks,
Dawn Kim
Check your environment variables. If the drivers are installed correctly there should be no issues...
Find out what version of the Microsoft C++ redistributable package you have.
For ASE 16 base you will need the following the 2010 version and the 2005 version for the isql and dsedit to work correctly. dsedit and isql are C based. I think it later changed to a 2008.
DBISQL connects with java. All the java should be installed. I do think this is related to the environmental variables as stated above.
Did you install the new ASE over an older installation?
ASE 16 changed some of its path stuff. You might want to check and clean out the environmental variables for an old location and a new location.
Can you post the error you are seeing with DBISQL?
Thanks,
Dawn Kim
At high level you are right (with [4] pointing definitely to the new environment ASE-16_0 & and -r resource file pointing to the sqlupgrade.adaptive_server.rs found in $SYBASE_ASE/init/sample_resource_files/... - filled with values first...).
I do, though, highly recommend to stick to the protocol as outlined in the SAP ASE Installation Guide for your platform (in particular Chapter 9: SAP ASE Upgrades). There are potential pitfalls you may find yourself in if you do not:
1. auditing - must be turned off, otherwise the upgrade process may have a good chance hanging upgrading sybsecurity database.
2. master/sybsystemprocs database sizes must be bumped up (should be taken care by the preupgrade).
3. Reserved Words / Missing SP source code/dbcc upgrade_object - all should be attended to.
4. DB consistency checks before and after (at least the minimal checkcatalog) should not be missed.
I found myself relying on the "upgrade" utility rather than preupgrade + sqlupgrade[res] - the choice depends how well the former run on your platform (there is a section on using each utility in the Installation Guide).
In general the process is pretty straightforward and works well. Just follow the guidelines from the Installation Guide.
Good luck!
HI Dawn,
Here is the output:
Script,Version,Status
'ODBC MDA Scripts','16.0.00.00.1013/Mon Jan 13 UTC 03:33:25 2014','Complete'
'OLEDB MDA Scripts','16.0.00.00.1013/Tue Jan 14 UTC 21:27:15 2014','Complete'
'installcommit','16.0 GA PL01/EBF 22540 SMP/P/X64/Windows Server/ase160sp00pl01/3523/64-bit/OPT/Tue Apr 15 14:51:12 2014','Complete'
'installjdbc','jConnect (TM) for JDBC(TM)/16.0 GA (Build 27008)/P/EBF22326/JDK 1.6.0/jdbcmain/OPT/Mon Mar 24 09:41:12 PDT 2014','Complete'
'installmaster','16.0 GA PL01/EBF 22540 SMP/P/X64/Windows Server/ase160sp00pl01/3523/64-bit/OPT/Tue Apr 15 14:51:12 2014','Complete'
'installmodel','16.0 GA PL01/EBF 22540 SMP/P/X64/Windows Server/ase160sp00pl01/3523/64-bit/OPT/Tue Apr 15 14:51:12 2014','Complete'
'montables','16.0/22540/P/X64/Windows Server/ase160sp00pl01/3523/64-bit/OPT/Tue Apr 15 21:50:31 2014','Complete'
I can zip up the C# app and a backup of the MSSQL DB (it only has the one table with a single row in it) and email it or attach to this issue if that would help.
Dave
Dear Subhram
If this is a SAP system on ASE , please refer to these SAP notes:
1982469 - SYB: Updating SAP ASE with saphostctrl / sapdbctrl
2162735 - SYB: Performing a major ASE upgrade (Windows)
2162715 - SYB: Performing a major ASE upgrade (UNIX/Linux)
HTH
Tilman Model-Bosch
Hi Dave,
I have uploaded a more current version of the script. You need to rename this file to a .sql and run it to the ASE like so:
isql -U username -P password -Sservername_in_interfaces -i oledb_mda_1600000.sql
Thanks,
Dawn Kim
Dawn,
I'm not sure if running this is to update the result of sp_version or to make the bulk update work.
Either way, I have run it and the result of the bulk update is the same.
When I run sp_version I now get
--now
Script,Version,Status
'ODBC MDA Scripts','16.0.00.00.1013/Mon Jan 13 UTC 03:33:25 2014','Complete'
'OLEDB MDA Scripts','16.0.01.00.1014/Sun Jan 25 UTC 03:42:56 2015','Complete'
'installcommit','16.0 GA PL01/EBF 22540 SMP/P/X64/Windows Server/ase160sp00pl01/3523/64-bit/OPT/Tue Apr 15 14:51:12 2014','Complete'
'installjdbc','jConnect (TM) for JDBC(TM)/16.0 GA (Build 27008)/P/EBF22326/JDK 1.6.0/jdbcmain/OPT/Mon Mar 24 09:41:12 PDT 2014','Complete'
'installmaster','16.0 GA PL01/EBF 22540 SMP/P/X64/Windows Server/ase160sp00pl01/3523/64-bit/OPT/Tue Apr 15 14:51:12 2014','Complete'
'installmodel','16.0 GA PL01/EBF 22540 SMP/P/X64/Windows Server/ase160sp00pl01/3523/64-bit/OPT/Tue Apr 15 14:51:12 2014','Complete'
'montables','16.0/22540/P/X64/Windows Server/ase160sp00pl01/3523/64-bit/OPT/Tue Apr 15 21:50:31 2014','Complete'
..which differs only in the row
'OLEDB MDA Scripts','16.0.01.00.1014/Sun Jan 25 UTC 03:42:56 2015','Complete'
Dave