SQL - Add Header Row to SSIS Generated CSV Files

After generating a flat file in SSIS I needed to update that file with the column headers as the first row.  In order to do this I used a SQL stored procedure.

STEP 1: Run the code below to create the stored procedure.

USE MIGRATION_DB

IF OBJECT_ID('GenerateBCPforSSIS') IS NOT NULL
DROP PROC GenerateBCPforSSIS
GO

CREATE PROCEDURE GenerateBCPforSSIS
(
@db_name varchar(1000),
@table_name varchar(1000),
@file_name varchar(1000)
) AS

Declare @Headers varchar(MAX),@HeadersRaw varchar(MAX),@sql varchar(MAX), @header_file varchar(MAX), @filename_short varchar(MAX)

--Generate column names as a recordset
Select @Headers = IsNull(@Headers + ',', '') + '""' + Column_Name + '""'
From INFORMATION_SCHEMA.COLUMNS
Where Table_Name = @table_name ORDER BY ORDINAL_POSITION ASC

--Create a dummy file to have header data
select @header_file=substring(@file_name,1,len(@file_name)-charindex('\',reverse(@file_name)))+'\data_file.csv'

set @sql = 'bcp "select ''' + @Headers + '''" queryout "'+@header_file+'" -c -C RAW -t, -S localhost -T'
print @sql

set @sql = 'type "'+@file_name+'" >> "'+@header_file+'"'
print @sql

set @sql = 'del "'+@file_name+'"'
print @sql

set @filename_short = reverse(substring(reverse(@file_name),1,charindex('\',reverse(@file_name)) -1 ) )

set @sql = 'rename "'+@header_file+'" '+'"'+@filename_short+'"'
print @sql

GO




STEP 2: Run the following command which will produce sql commands for all the views and tables in your database.  You can add where clauses to the statements to limit which views/tables commands are created for.  Note: Make sure to update the MIGRATION_DB string to the name of your database

USE MIGRATION_DB

SELECT 'exec GenerateBCPforSSIS ''MIGRATION_DB'',''' + name + ''',''E:\SSIS Output\' + name + '.csv''' AS CommandName
FROM sys.views 

UNION

SELECT 'exec GenerateBCPforSSIS ''MIGRATION_DB'',''' + name + ''',''E:\SSIS Output\' + name + '.csv''' AS CommandName
FROM sys.tables



STEP 3: Copy the output from the command in step 2 and paste it into a new SQL query window.


STEP 4: Copy the output from step 3 and past it into a batch file.


STEP 5: Run the batch file, it will open your SSIS csv file and insert a header row with the column names. 


Notes:



  • The CVS file output from SSIS must match the names of your Views/Tables

  • If you change, add, or remove any of the columns in your table/view you will have to run steps 2-5 again.

Comments

Post a Comment

Popular posts from this blog

Add User As Local Administrator On Domain Controller

I recently was settting up a new Microsoft SharePoint 2010 machine and had promoted the machine to a domain controller before creating my SharePoint admin accounts.  I needed to add several of my accounts to the local Administrators group.  Unfortunately after you promote a server to a domain controller you can no longer access the GUI for Local Users and Groups.  Instead I had to use the command line to add the users.

Open a command promt using the "Run as administrator" function and then run the following command.

net localgroup Administrators /add {domain}\{user}

Note: do not include the {} brackets.
Read more

An error occurred while applying security information to

While attempting to move files from a hard drive I recovered I kept getting the following error on a specific set of files.

An error occurred while applying security information to
{G:\Folder}
Failed to enumerate objects in the container. Access is denied.

In order to fix it did the following.

-Open a Command Prompt "As Administrator"

-Run the following commands:

takeown /f G:\folder/r /d y
icacls G:\folder /grant administrators:F /T
Read more

Display ADFS 2.0 Forms Authentication Login Page Instead of Windows Authentication Prompt

Image
After installing ADFS 2.0 for SharePoint a Windows login prompt was shown when the SharePoint site forwarded to the ADFS server instead of the ADFS Forms Authentication login screen. 



No matter what account I tried to use here I would eventually receive a 401 Not Auhorized error.



The reason for this is that the ADFS website tries to use Windows Authentication before trying to use the Forms authentication which displays the loging page below.




To fix this do the following on the ADFS server:
1. Open IIS and Explore under Default Website\adfs\ls

2. Open the web.config file with Notepad, look for the localAuthenticationTypes section.


3. Move the line for Forms above the line for Integrated and save the web.config file.  This will force the ADFS application to use the Login Page authentication before trying to use Windows Authentication.


Read more