How to increase SSIS performance

Hello again,

I'll just throw my question: how could I increase SSIS-performance?

I have a really heavy job with thousands of records my base selection, then I perform some lookups (I replaced most of them by sql) and derived columns (again, I replaced as much as possible by sql). Finally, after a slowly changing dimension task, I do update/insert on a given table. Is there a trick to speed up lookups and inserts (something like manipulating the buffer sizes - just asking).
Fact is that I replaced a script task by pure sql-joins and gained 6 of the 12 hours this job took.

Any ideas?

Greets,
Tom

Tom De Cort wrote:

Hello again,

I'll just throw my question: how could I increase SSIS-performance?

I have a really heavy job with thousands of records my base selection, then I perform some lookups (I replaced most of them by sql) and derived columns (again, I replaced as much as possible by sql). Finally, after a slowly changing dimension task, I do update/insert on a given table. Is there a trick to speed up lookups and inserts (something like manipulating the buffer sizes - just asking).
Fact is that I replaced a script task by pure sql-joins and gained 6 of the 12 hours this job took.

Any ideas?

Greets,
Tom

Speeding up by replacing the script task doesnt surprise me. If you have a high-performant database engine at your disposal, why not use it!

Some links for you that may help:

Whitepaper on Performance Tuning Techniques
http://blogs.conchango.com/jamiethomson/archive/2006/04/09/3594.aspx

Donald Farmer's Technet webcast (highly recommended)
http://blogs.conchango.com/jamiethomson/archive/2006/06/14/4076.aspx

-Jamie

|||

Hi Jamie,

Great, I've read the whitepaper and discovered a lot!

Greets,
-Tom

How to increase SSIS performance

Hello again,

I'll just throw my question: how could I increase SSIS-performance?

I have a really heavy job with thousands of records my base selection, then I perform some lookups (I replaced most of them by sql) and derived columns (again, I replaced as much as possible by sql). Finally, after a slowly changing dimension task, I do update/insert on a given table. Is there a trick to speed up lookups and inserts (something like manipulating the buffer sizes - just asking).
Fact is that I replaced a script task by pure sql-joins and gained 6 of the 12 hours this job took.

Any ideas?

Greets,
Tom

Tom De Cort wrote:

Hello again,

I'll just throw my question: how could I increase SSIS-performance?

I have a really heavy job with thousands of records my base selection, then I perform some lookups (I replaced most of them by sql) and derived columns (again, I replaced as much as possible by sql). Finally, after a slowly changing dimension task, I do update/insert on a given table. Is there a trick to speed up lookups and inserts (something like manipulating the buffer sizes - just asking).
Fact is that I replaced a script task by pure sql-joins and gained 6 of the 12 hours this job took.

Any ideas?

Greets,
Tom

Speeding up by replacing the script task doesnt surprise me. If you have a high-performant database engine at your disposal, why not use it!

Some links for you that may help:

Whitepaper on Performance Tuning Techniques
http://blogs.conchango.com/jamiethomson/archive/2006/04/09/3594.aspx

Donald Farmer's Technet webcast (highly recommended)
http://blogs.conchango.com/jamiethomson/archive/2006/06/14/4076.aspx

-Jamie

|||

Hi Jamie,

Great, I've read the whitepaper and discovered a lot!

Greets,
-Tom

How to import several excel files using one dts package

It's very simple import, without any data modifications; from several
excel files to one table.
I tried the wisard and it gives me selection for only one file.
I am not used to create DTS packages and schedule jobs, so I would need
some help.

Thank you
InnaHi Inna,

Perhaps this article will be of help:
http://www.devx.com/getHelpOn/10MinuteSolution/18088

The article at the link above shows how to import Excel data into SQL
Server from an indeterminate number of Excel workbooks.

If the # of Excel files you have is small, you can do the following in
the DTS Designer:

1) Drag 1 Excel connection object per file into the DTS "canvas" (the
main area of the DTS Designer) and define the connections as
appropriate

2) Drag a connection object for your SQL Server and define it as
appropriate. One thing to note is that you should set the DB to connect
to the same as the DB you want to import the data into

3) Define a "Transform Data Task" (see your Tasks menu in the DTS
Designer) between each Excel connection and the SQL Server connection.

4) A line will appear between each Excel connection and the SQL Server
connection. Right click on each line and select "Properties" from the
menu that appears. Alter the properties of the task (e.g. column
mappings) as appropriate.

5) Execute the task

Hope that helps a bit.