Thursday, July 24, 2014
   
Text Size
Login

SSIS: How to loop through multiple flat files as data sources

What?
This is an article to demonstrate a quick step-by-step on having an SSIS package loop through a directory/folder of files in order to populate a database table. We could add each file as a separate connection manager but this is inefficient and not versatile enough to accommodate files that get added later.

Why?
I am creating an extract SSIS package intended to take a text file as its source and to populate a database table with this data. Note that this only works if all the text files to be used as source data have the same number of columns and where the column widths match.

How?
I've adapted my real working product with an example. As my work was for a Personnel/HR project, and data confidentiality is somewhat important, some of the images will be censored or data changed in the example below.

SSIS: Cannot open Data Flow Task: DTS.Pipeline.2 issue

What?
I started getting this error:

  1.  The task with the name "Data Flow Task" and the creation name "DTS.Pipeline.2" is not registered for use on this computer 


Why?
We use a combination of MS Visual Studio 2008, 2010 and 2012. VS2008 for our old SSIS packages and VS2010 for database solutions running against SQL Server 2008 R2. We can use VS2012 for both but this requires upgrading all the packages and then making them suitable for our new SQL Server 2012 instances.

How?
This applies to a workstation (hopefully you're not doing development directly on the server). I re-installed sql server setup, re-ran the repair to no avail. A clue came from the MS site for an older version of the pipeline in that you access the SQL Server Configuration Manager. This may not be the fix for you but it was for me:
  1. Open Start > All Programs > SQL Server 2012 (or your latest)
  2. Expand "Configuration Tools" and open "Sql Server Configuration Mnaager".
  3. Right-click on "SQL Server Integration Services 10.0" and select "STOP".
  4. Right-click on "SQL Server (SQLEXPRESS)" and select "STOP".


SSIS: Add Reference Microsoft.SqlServer.TxScript

What?
This article is so that I don't spend so long in googling what the TxScript referenced assembly is when creating an SSIS custom component.

How?
This is simply the "Script Component" under "Extensions". No MSDN website said this... obvious to Microsoft staff so why should their documentation include it?

SSIS Script: convert UPPERCASE to Mixed-Case using TitleCase

Applies to:
  • Microsoft Business Intelligence Development Studio (BIDS) Visual Studio 2008
  • Microsoft Windows 7
  • Microsoft .NET Framework 3.5
  • Microsoft Visual C# 2008

What?
So like lots of people on the net, I've been given a datasource with names and addresses all in UPPERCASE. Initially, no one seemed bothered but now the request came through asking if this could be restored to a normal case, a mixture of mostly lowercase and some UPPERCASE.

Why?
I'm writing this article because other solutions on the net were either incomplete, did not work for me or simply did not match the requirements (do they ever?).
  1. The script must be able to be copied into other SSIS packages.
  2. Need the option to merely tick input column names rather than hardcode these.
  3. "It " should be an acronym for IT, so "IT Department" and "Director of IT".
  4. "O'REILLY" should become "O'Reilly" and not "O'reilly".
  5. "MCDONALDS" should become "McDonalds" but do not apply for "MacDonalds" because of names like "MACK" and "MACHURA".
  6. " and ", "la", "le", " van ", " de ", " of " and similar should remain lowercase.

How?

Trim in T-SQL and SSIS

What?
Some methods of removing trailing spaces, tabs, carriage returns and line feeds (new lines).

How?
First in Transact-SQL:

Latest Posts

  • Joes Quicklist Weblinks (JQW)

    • Thu 17-Jul-14
      Hi, thanks a lot. But I have got a problem with download it. Error: "File could not be found Important!
      Kat  
    • Tue 15-Jul-14
      Hi Artur, I forgot to get back to you sooner. I was going to include a set of instructions to do this ...
      Webmaster  
    • Mon 14-Jul-14
      Hi Skarosg, Not sure if we took this to e-mail but I noticed your site is displaying the images now.
      Webmaster  
    • Fri 11-Jul-14
      Hi, How can I put user name and category of weblink in module, near the date? Thanks..
      Artur
  • Joes Word Cloud (JWC)

    • Tue 17-Jun-14
      I'm using JE 3.1.19 stable.
      alve89