Posts

Becoming a Data-Driven Credit Union

, ,

Data warehouses, multi-dimensional analysis cubes, and predictive analytics can appear to be imposing goals.

Where do we start?

How quickly can we accomplish our data goals?

Do I have the right talent?

These questions, and many like them, create doubt and concern amongst credit union executives wondering how they can start utilizing data better and begin to develop BI practices. Rich Jones and I have developed a business intelligence strategic engagement opportunity with credit unions seeking to answer these questions about business intelligence and data utilization. From the $300 million credit union or community bank without an MCIF to the $2.8 billion organization working to build their first data warehouse, we help each organization design a very specific roadmap that will enable them to accomplish their business intelligence goals and become a data-driven credit union.

So how does it work?

We recently completed a very successful business intelligence strategy engagement with a large east coast credit union. Their story was like most credit unions – there were many third-party applications and data sources spread throughout the organization. The information contained within these disparate applications made it incredibly challenging to get a 360-degree view of their membership and their internal business operations. Realizing that they could do more with their data, they sought our assistance to develop a long-term business intelligence and data strategy that could lead them to success.

Once we arrived onsite, we met with nearly 30 individuals over a four-day period to identify the various applications used throughout the credit union, the current reports and data uses, the needs of each department, and a variety of other relevant details. From the CEO to the Vice President of Consumer Lending to the lead Lending Analyst, each individual interviewed gave us the insight necessary to understand the organizational data needs and current data challenges. These interviews were absolutely critical to the success of the engagement and comprise the overwhelming majority of the time that we spent onsite.

With the interviews completed (and many pages of notes!), we consolidated our discovery notes and developed an 18 to 36 month data strategy that will allow the credit union to achieve their data goals. Our very detailed implementation plan included hiring milestones, third-party data integration milestones, business intelligence unit establishment deadlines, and several other key deadlines and milestones. These very specific deadlines allow the credit union to monitor their progress as they undertake each step of the process. Sequential, logical, and iterative, the roadmap breaks down the business intelligence strategy into very achievable and digestible tasks that reduce the stress that often accompany significant organizational projects.

Prior to this, the credit union was struggling with how to take the right first steps in achieving their data goals. The project loomed ominously as they continued to struggle with business intelligence and data utilization.

Now, the credit union understands exactly what they need to do, whom they need to hire or train, when they need to achieve certain milestones, and all of the details in between. Stress and concern over business intelligence has turned into excitement and opportunity. They now have the plan in place to become a data-driven credit union!

Why us?

Our implementation plans and business intelligence roadmaps are based on significant past experiences and expertise. Rich’s experience in leading the establishment of business intelligence units at multiple large credit unions, and my expertise in the tactical development of data warehouses and business intelligence operations creates a powerful combination of two distinct yet complementary skill sets. Together, we can help change the credit union world from a backwards-looking, report-reading industry into one that is forward-looking and data-driven.

You may contact Brewster Knowlton at brewster@knowlton-group.com or Rich Jones at rich@leading2leadership.com. Click to get additional information about our business intelligence strategic engagement!

/by

Import Data into SQL from Excel

, , , ,

In our last lesson, we showed you how to use SSIS to export data to Excel. In fact, all of our previous SSIS lessons have focused on exporting data from SQL into some other format. In this lesson, however, we will show you how to import data from Excel into SQL.

This is very helpful as you start to develop your business intelligence program in your bank or credit union. Think of how much data exists at your financial institution sitting in the form of Excel spreadsheets. By importing this data into SQL, it can be cross-referenced and/or integrated with existing SQL data. This is a key first step in any business intelligence program, and in this lesson we will help you get started!

Setting up our Data Source

The Excel spreadsheet that I will be importing into SQL is based on data compiled by the NCUA. The spreadsheet contains a list of all credit unions and their associated member totals and total assets (as 6/30/2014). The image below gives you a quick snapshot of what the data looks like:

Spreadsheet Header

With our source Excel spreadsheet identified and described, open up your SSIS solution named “Credit Union SSIS Solution” if you have been following along with previous examples. Otherwise, create a new SSIS solution and name it “Import to SQL.dtsx”. Just like we have in past examples, drag a “Data Flow Task” from the “SSIS Toolbox” onto the “Control Flow” designer. Right click on the task, click “Rename”, and then rename the task “Import NCUA File into SQL”.

Renamed Data Flow Task

Now, double-click on the newly renamed task to be brought to the “Data Flow” editor. From the “SSIS Toolbox”, drag an “Excel Source” item onto the data flow designer. Rename this item “NCUA Excel File”.

NCUA Excel File Task Item

Double-click on the renamed item; the “Excel Source Editor” window will appear on your screen. Click the “New” button to the right of the “Excel connection manager” dropdown. Click “Browse”, then locate the file you will be importing (in my case, the NCUA file), press “Open” and then press “OK” when you are brought back to the “Excel Connection Manager” window.

In the “Name of the Excel sheet” dropdown, choose the specific spreadsheet within the Excel workbook that you wish to choose as your data source.

Excel Source Editor

Press “OK” once this has been completed.

Defining the Destination to Import Data into SQL

With the Excel source defined, drag an “OLE DB Destination” item from the “SSIS Toolbox” onto the data flow designer. Rename this item “SQL Destination”. Click the “NCUA Excel File” item so that it is selected then drag the blue arrow from the source item to the “SQL Destination” item.

Connected Items

Next, double-click on the “SQL Destination” item. Here, we defined where the NCUA data will reside in SQL. Click the “New” button to the right of the “OLE DB connection manager” dropdown. Select the connection from the left panel on the “Configure OLE DB Connection Manager” window that appears or define a new one by clicking the “New” button at the bottom of the window. Press “OK”.

For this example, we will need to create a new table in the database for this Excel data to reside. To do this, click the “New” button to the right of the “Name of the table or the view” dropdown. The “Create Table” window will appear on screen containing the SQL code that will be used to create a new table. I want to make one subtle change and alter the name of this table. After “CREATE TABLE” replace “[SQL Destination]” with “NCUA_Data”.

Create Table Window

Press “OK” once you have done this. If you open SQL Server Management Studio, you will see that this new table has been created in the database. This is important; the table is created once you press “OK” from the “Create Table” window – NOT when you execute the package for the first time.

To finish setting up the destination, press the “Mappings” option from the left side of the “OLE DB Destination Editor” window. Ensure that the column mappings are correct, and then press “OK”.

Execute the package (by pressing F5 or the green play button on the toolbar) to see if everything works. Assuming you successfully execute the package in SSDT, open up SQL Server Management Studio. We want to view the contents of the table we just sent the Excel data to in order to validate the process properly imported all data.

SQL Validation

It’s that easy! The process required to import data into SQL from Excel is nearly identical to the previous export examples we have completed throughout the series. We simply define the source, define the destination, and then the mappings between the two. Learning this relatively simple task in SSIS can open up tons of new opportunities for you to be able to use your data more effectively. Eliminate silos of Excel data by bringing that data into SQL. Doing this will allow you to integrate and cross-reference it with your other SQL data sources ultimately bringing you closer to business intelligence success!

Download the Excel File from this lesson by click this link: NCUA_Data.xlsx

/1 Comment/by

Using SSIS to Export Data to Excel

, ,

In our previous post, we learned how to append the date to flat file exports. This represents an intermediate level skill in SSIS but can be incredibly helpful in production business environments. Unfortunately, everything we have looked at in SSIS so far has related to flat files. More often than not our business users would prefer to receive data extracts in Excel files. In this lesson, we explore how to use SSIS to export data to Excel.

Defining the Source Data

If you haven’t done so already, open up the SSIS solution in SQL Server Data Tools (SSDT) or Business Intelligence Development Studio (BIDS) named “Credit Union SSIS Solution” or create a new SSIS solution.

Right clicking on the folder named “SSIS Packages” in the “Solution Explorer”, create a new package and rename it “Export to Excel.dtsx”.

As we have done in previous SSIS examples, drag a “Data Flow Task” from the “SSIS Toolbox” onto the blank Control Flow editor of the SSIS package.

Data Flow Image

To proceed to the “Data Flow” editor, either select the “Data Flow” tab to the right of the “Control Flow” tab just above the main editor panel or double-click on the “Data Flow Task” item in the control flow designer.

Like with any “Data Flow Task”, we need to define the source of our data. Since our data source will be the result of a SQL query, drag an “OLE DB Source” item from the “SSIS Toolbox” to the designer.

OLE DB

Next, we need to define the data source. To do this, double-click on the “OLE DB Source” item. A blank “OLE DB Source Editor” window will appear on screen.

Blank OLE DB Source Editor

Set the “OLE DB connection manager” dropdown to the proper database in which your data resides. We walked through this in our post that talked about how to use SSIS to export data to flat files. Refer to that post if you need some assistance walking through that process.




Once you have defined the “OLE DB connection manager” dropdown, change the “Data access mode” dropdown to “SQL Command”. This allows us to return the results of a specific query as opposed to an entire table. I am going to complete a query that returns the names of all current employees and to what branch they are assigned. To this, I will use the query seen in the image below:

OLE DB Source 2

Once you have the query defined, press the “Columns” option in the window’s left panel. Verify that the columns that appear on the screen are the columns that you wish to export to Excel.

OLE DB Columns

Press “OK” once you have done this. We now have our data source defined!

Defining the Excel Destination

Exporting to an Excel destination has some differences compared to exporting to flat files. SSIS essentially treats the Excel destination like a SQL table in some ways which we will see upon completion of this walkthrough.

To define our Excel destination, drag the “Excel Destination” item from the “SSIS Toolbox” out onto the designer.

Dragged Excel Destination

Click on the “OLE DB Source” item and drag the blue arrow to the “Excel Destination” item and then release. If done correctly, the two items should be connected by the blue arrow:

Connected Items

Double-click on the “Excel Destination” item. The “Excel Destination Editor” window will appear. This is nearly the same exact window that appears when you define an OLE DB source. The only difference here is that we are defining the destination instead of the source of the data.

Excel Destination Editor Blank

Click the “New” button to the right of the “Excel connection manager” dropdown. When the “Excel Connection Manager” window appears, press “Browse”. A file explorer window will appear.

Create Excel Connection

Navigate to wherever you would like to create your Excel destination. I will be leaving mine on my Desktop. In the “File name:” text box, type “Active Employees” and then press “Open”. This creates the Excel destination. When you press “Open” you will be returned to the “Excel Connection Manager” window. Leave the checkbox next to “First row has column names” checked and then press “OK”.

Learn SQL Course

Looking to learn SQL? Click here!

You will be brought back to the “Excel Destination Editor” window. We have, at this point, defined what Excel file our data will be sent to. However, we now need to define the actual sheet that the data will reside in. Think of the Excel file as the database and the spreadsheet as the actual table.




To define this, click the “New” button to the right of the “Name of the Excel sheet” dropdown. You will notice that a script is automatically generated in the “Create Table” window that appears. It recognizes the columns that are contained in the OLE DB Source and properly defines the data types for each column. Press “OK”.

When you are returned to the “Excel Destination Editor” window, select “Excel_Destination” from the “Name of the Excel sheet” dropdown. Once you have done this, click the “Mappings” option from the left panel of the window.

Mappings

Press “OK” once this has been completed.

Export Data to Excel

Now that all of the steps have been completed, we can test to see how our package works. Press the green play button or F5 to start debug mode and execute the package. If you get an error that mentions something about running the package in 32-bit mode, head to the main menu, hover over “Project” and choose “Credit Union SSIS Solution Properties” – the last item in the menu. When there, click the “Debugging” option under “Configuration Properties”. Set the “Run64BitRuntime” property to “False”. Apply and press “OK”.

If the package successfully executed, your screen should like the image below:

Successful Execution

Navigate to the Excel file that you sent the data to and open it up. Notice the number of rows in your Excel spreadsheet (in my case, 755 including the header). Close the Excel file and then go back to SSIS and execute the package again. Once executed, re-open the Excel file the data was exported to. Notice the number of rows now.

SSIS will not overwrite the rows and will simply add another 754 data rows (in my case) to the spreadsheet. Since SSIS treats the Excel destination like a SQL table in many regards, the data will be appended instead of being overwritten. This is a subtle yet critical difference between working with Excel exports and flat file exports.

Despite some subtle differences, the majority of the steps to use SSIS to export data to Excel are identical to exporting data to flat files. As you continue to use SSIS, you will become more familiar with these processes and be able to quickly make simple packages with ease!

/by

Append the Date to Flat File Exports with SSIS

, , , ,

In one of our previous posts, we provided instructions on how to use SSIS to export data to flat files. Often times, our business users would like to have a running history of exported data. For example, if we had an export that output all orders for the prior day our users wouldn’t want to see a file named “Orders Placed Yesterday”. Data consumers would prefer to see a file named “Orders Placed 6-30-2015”. To append the date to flat file exports in SSIS is a relatively common and simple task.

Setting Up the File Name Variable

To start, open up the SSIS solution named “Credit Union SSIS Solution”. Right click on the “Export to Flat File” package in the “Solution Explorer” and select “Copy”. Then right-click on the “SSIS Packages” folder and click “Paste” to create a copy of the package. Rename this package “Export with Date Append”.

Make sure that you have a “Variables” panel on the left side of your screen. If you cannot find this panel, click on the “View” option from the main menu, hover over “Other Windows” and choose “Variables”.

Our plan is to use variables to store the fully qualified file name with the appropriate date value. Then, we will use this variable value as the connection string for our flat file connection.

To start, click the “Add Variable” button in the top left corner of the “Variables” panel.

New Variable Button

A new row will appear in this “Variables” panel. Name the variable “FilePath” and then set the “Data type” column value to “String”.

First Variable

With the variable’s row highlighted, navigate to the “Properties” panel which should be on the right side of your screen. Find the property named “EvaluateAsExpression”; set this value to “True”. Directly beneath this property is another property named “Expression”. If you click in the text box to the right of the “Expression” property, an ellipsis button will appear to the right; click this button.

Properties

The “Expression Builder” window will appear on your screen. Here is where we will define how this variable will be evaluated. I won’t dive too deep into how to create different SSIS expressions, but if you click here you will be redirected to SSIS expression documentation from the Microsoft Developer Network that provides a wealth of knowledge on the subject.




In the “Expression” text box, type in the following expression (including all quotation marks as they appear and of course modifying for your specific location and folder names):

“\\\\localhost\\c$\\Users\\Brewster\\Desktop\\AccountTypes as of ” + (DT_WSTR, 2)MONTH(GETDATE()) + “-” + (DT_WSTR, 2)DAY(GETDATE()) + “-“+ (DT_WSTR, 4)YEAR(GETDATE()) + “.csv”

Once you have typed or copied the above expression, click the “Evaluate Expression” button below the text box. The “Evaluated value” read-only text box will display how the expression evaluates. In this case, it contains the fully qualified file path containing the current date appended to the file name and the appropriate “.csv” extension.

FilePath Expression

Press “OK” once this has been completed.

Adjusting the Connecting String

Next, we need to adjust the connection setting for the flat file export. To do this, highlight the “AccountTypes.csv” connection from the “Connection Managers” panel towards the bottom of your screen:

Flat File Connection




Navigate to the “Properties” panel, and find the “Expressions” property. Click the ellipsis button to the right of the property. The “Property Expressions Editor” window will appear. In the first row, choose “ConnectionString” from the “Property” column dropdown. In the “Expression” column either type “@[User::FilePath]” or click the ellipsis and choose the variable name from the “Variables” folder in the resulting window that appears. This specifies that we want to send our data to the location specified by the “FilePath” variable.

Property Expressions

Press “OK” once this has been completed.

Guess what? That’s it! That is all it takes to append the date to the file name for a flat file export using SSIS. To test the package, click the green triangle (play button) on the toolbar underneath the main menu. Your package should like the screen below if everything executed successfully:

Debug Mode

With the package successfully completed, navigate to the location of the export and check to be sure that the file was exported properly. You can see in the image below that I have a file with the date appended to the location I expected to see the file:

Desktop Image

What might some like an otherwise challenging task can be completed rather simply using SSIS. Understanding what types of expressions can be created and how they can be employed throughout your SSIS packages is a valuable skill. This simple example hopefully provides you with some insight on other ways you can utilize SSIS in your organization!

/by

Using the SQL Agent to Automate SSIS Packages

, , , , ,

In our last post, we learned how to use SSIS to export data to a flat file. While that is certainly a step in the right direction towards removing all manual effort from our reporting processes, we still had to manually kick off that SSIS package. In this post, we are going to learn how we can use the SQL Agent to schedule this package to automatically execute.

What is the SQL Agent

The SQL Agent is a Windows service that executes scheduled tasks or jobs. Creating a SQL Agent job allows us to schedule when an SSIS package, for example, is executed automatically. SQL Agent jobs can be created to execute a number of tasks including: executing T-SQL code, SSIS packages, SSAS queries, PowerShell scripts, and a number of other types of tasks. For our purposes, we will be creating SQL Agent jobs to automatically execute our SSIS packages. ETL jobs are often automatically executed at a set time (usually nightly) via SQL Agent jobs, therefore understanding the basics of the SQL Agent are critical to becoming well-versed in the data warehouse/business intelligence toolkit contained within Microsoft SQL Server.

Creating the SQL Agent Job

Before we create the SQL Agent job, open up the “Credit Union SSIS Solution” in SSDT if it is not currently open. In the “Solution Explorer”, right click on the “Package.dtsx” package name and click “Rename”. Rename the package “Export to Flat File.dtsx”. Press the “Save All” icon on the toolbar and then exit SSDT (SQL Server Data Tools).

NOTE: the next steps require you have the proper authorization and permissions to create, schedule, and execute SQL Agent jobs. You may need to speak with your IT department for proper authentication depending on your SQL environment.

Open SQL Server Management Studio (SSMS) and login with your Windows credentials. At the bottom of the “Object Explorer” you should see an item named “SQL Server Agent”. Click the toggle button to the left of it to expand the folder. Once expanded, expand the “Jobs” folder.

Object Explorer

This where the SQL Agent jobs are stored. Notice that two jobs currently appear – these are generated by the system and can be ignored. To create a job, right click on the “Jobs” folder and click “New Job”.

New Job

Give this job the name “Exporting to Flat File”. Leave the other default options unmodified. In every SQL Agent Job you can have many steps. So, for example, we could have multiple SSIS packages executed in the same job. This is a nice feature if you have recurring processes that need to be executed on the same schedule. You could create a single job for all of your weekly recurring reports, so that you would only need to monitor and maintain a single job instead of one for each SSIS package.




Click on the “Steps” option on the left side of the “New Job” window.

Steps

Click the “New” button near the bottom of the window to create a new job step. Name this step “Export to Flat File SSIS Package”. In the “Type” dropdown, select “SQL Server Integration Services Package”. You will notice the bottom half of the window will change significantly once you make the selection.

In the “Package Source” dropdown, choose “File system”. This is telling the SQL Agent that we will be choosing a package located in a folder instead of being stored somewhere else (there are other package storage mechanisms within SQL Server that could be selected). With “File system” selected, click the ellipsis button to the right of the “Package:” text box near the bottom of the window. Locate the SSIS package “Export to Flat File.dtsx” that we created in the previous post and renamed earlier in this post. Press “Open” once you have found this item.

Job Step

After making the necessary changes, your window should look similar to the image above. Press “OK” once you have made the modifications. Once you press “OK”, you will be brought back to the “New Job” window which should now look like:

New Job Part 2

We have created the SQL Agent job with the appropriate step created to execute the SSIS package we created in the last post. Now we must schedule the job to execute at a specific time.

Click the “Schedules” option from the panel on the left. Once the window changes, press “New” towards the bottom of the screen.




When the “New Job Schedule” window appears, type “Sample Ad Hoc Execution” in the “Name” text box. Then, in the “Schedule type:” dropdown, choose “One time” from the menu.

Schedule the job to execute about ten minutes from when you are completing this step. So, I am completing this step at around 11:50 AM on 6/29/2015, so I will set the job to execute at 12:00 PM.

Job Schedule

Press “OK” once you have made the schedule updates. Press “OK” at the bottom of the “New Job” window to save the SQL Agent job we just created. If there were any errors or you do not have the necessary permissions to create the job, SQL will throw an error and a pop-up window will appear.

You will notice that the “Exporting to Flat File” job now appears under the “Jobs” folder in the “Object Explorer”.

Updated Jobs Folder

Once the scheduled execution time passes, right click on the job in the “Object Explorer” and click “View History”. You will see a successful execution logged with the appropriate details on the screen. If anything failed in the job, this window will tell you why the error occurred. Using this log viewer is incredibly helpful when debugging a failed SQL Agent job or step.

View History

You can navigate to wherever you saved the exported flat file and see that the file was modified and created right at the time we specified.




And just like that you created your own SQL Agent job and automated the execution of an SSIS package. This is a critical step towards leveraging the most valuable tools within Microsoft SQL Server and its business intelligence operations. If you need to automatically import data into a SQL table or automatically execute a set of SSIS packages, using a SQL Agent job provides you with the flexibility and automation you need to be efficient and successful!

/1 Comment/by

Creating Data-Driven SSRS Subscriptions

, ,

In our last lesson, we explored how to create a basic SSRS subscription. This lesson is going to add some dynamic capabilities to the standard SSRS subscription to create a “data-driven SSRS subscription”.

We are going to set up a data-driven SSRS subscription that, on a pre-defined schedule, outputs a report for share accounts opened for each branch in our credit union. So, we will have one report automatically sent to a location on our network with the appropriate details for each branch. Instead of creating and managing eleven separate standard SSRS subscriptions, we can accomplish this with a single data-driven subscription.

What is a data-driven SSRS subscription?

A data-driven SSRS subscription is defined by Microsoft as “a data-driven subscription provides a way to use dynamic subscription data that is retrieved from an external data source at run time”. Where a standard SSRS subscription has the parameters, rendering method, and file path statically pre-defined, a data-driven SSRS subscription allows you to define all of the parameters and options at run-time. These parameters and options are defined by data from some data source – like the results of a query or data contained in a table. We will walk through one example of how you can create and use a data-driven SSRS subscription.

Modify One of Our Existing Reports

Open up SSDT or BIDS to the project “Credit Union Reports” if you haven’t already. There should be a report named “Share Account Details by Branch”. This was the detail report that we used for our drill-down report example. Right click on the report’s name in the “Solution Explorer” and click “Copy”. Then select the “Reports” folder in the “Solution Explorer” – so it is highlighted – and then press “Control” and “V” on your computer to make a copy of the report.

A new report named “Copy of Share Account Details by Branch” should appear. Rename this report “Share Account Details by Branch with Dates.rdl” by right clicking on the report and selecting “Rename”. Double click the newly renamed report to open it up.

Once the report is opened, double click on the “ShareAccountDetails” dataset in the “Report Data” panel on the left side of the screen. The “Dataset Properties” window will appear on your screen. Currently, this report has only one parameter: the branch name. We want to edit the query so that we can also filter based on the date in which the account was opened. See the highlighted line in the image below to see how to modify the query:

Modify Dataset

Once you have added the additional criteria to the WHERE clause of the query, press “OK”. Edit the “StartDate” and “EndDate” parameters from the “Parameters” folder of the “Report Data” panel so that each parameter’s data type is set to “Date/Time”. Preview your report to ensure that the parameters are working as intended.

Save the report and the entire solution. Upload the new report to the report server like we have done in the past if you haven’t already.




Creating the Data-Driven SSRS Subscription

Navigate to your report server in your browser. Hover over the report that we just uploaded named “Share Account Details by Branch with Dates”, choose the drop down arrow that appears, and select “Manage”. Click the “Data Sources” option on the left navigation panel once the report properties page appears. Just like we did when creating a basic SSRS subscription, we need to store our credentials securely in the report itself. Review the previous post if you need to review how to complete this step.

Once you have entered and saved the credentials for the data source, choose “Subscriptions” from the left navigation panel on the screen. On the top menu, choose “New Data-driven Subscription”:

New Data-driven Subscription

Step 1

The page that appears will have the header “Step 1 – Create a data-driven subscription: Share Account Details by Branch with Dates”. In the “Description” text box, enter a name for this subscription that will make it clear what the subscription is used for. I’m simply going to enter “Share Details for each Branch”. In the “Specify how recipients are notified:” dropdown, choose “Windows File Share”. If the radio button next to “Specify for this subscription only” is not selected, do so now.

Step 1

Press “Next” once complete.

Step 2

Step 2 of the process is where we define in which data source our data-driven data is going to reside. We will be using a branch list that is contained in the “CreditUnion” SQL Server database on the local machine. In the “Connection string:” box, type “data source=localhost; initial catalog=CreditUnion”. You may replace “localhost” with your SQL Server instance name and “CreditUnion” with your specific database name.

Under “Connect using:” add your Windows credentials is the username and password fields. Check the box next to “Use as Windows credentials when connecting to the data source”. Note: if you have been accessing the database using SQL authentication instead of Windows authentication, do not check the box.

Step 2

Press “Next” once complete.

Step 3

In this step, we define the query that we will use to retrieve the data for the data-driven SSRS subscription. I am going to use the query that you see on the screen in the image below:

Step 3

I am returning two columns from a table named “Branch”: the “Branch_Name” column and a concatenated string expression that returns the output “2010 Accounts Opened for [Branch_Name]”. This expression, named FileName, will be used to name each branch’s file that gets output to the network location. We will see this used shortly.

For now, simply press the “Validate” button towards the bottom of the screen to ensure that your SQL syntax is correct. Once validated, press “Next”.




Step 4

Step 4 is where we get to the real details of the data-driven subscription. Here is where start to define all of the dynamic or static options we will be using for our subscription. Under “File name” select the radio button next to “Get the value from the database”. From the dropdown, choose the column name “FileName”. This is the concatenated string expression we defined in step 3. This tells SSRS how to name the report for each branch.

Under “Path” leave “Specify a static value” selected. I am going to send the files to my C drive, so I will use the path “\\localhost\c$”. Under “Render Format”, select “Excel” from the static value list dropdown. For “Write Mode”, choose the static value “Overwrite”. Similarly, for the “File Extension” section, choose “True” from the static dropdown list. Add your Windows credentials for the “User name” and “Password” sections just like we have done in the past. Your page for step 4 should look like the image below:

Step 4

Press “Next” once you are complete.

Step 5

In step 5, we are able to specify the values for our report parameters. Under “Branch Name” choose “Get value from the database” and select “Branch_Name” from the dropdown. For the “Start Date” and “End Date” parameters, you can statically enter a date. I am going to enter “1/1/2010” and “12/31/2010” based on the data for this fake credit union.

See the image below to ensure each parameter was defined correctly:

Step 5

Press “Next” once complete.

Step 6

Step 6 is where we define how the subscription will be executed and scheduled. In our scenario, we want the report to be schedule to run at a specific time. To do this, select the radio button next to “On a schedule created for this subscription”.

Step 6

Press “Next”.

Step 7

In step 7, we are going to create the schedule for the data-driven SSRS subscription to execute. We have done this before when scheduling a basic SSRS subscription. I am creating this post on a Monday at 11:20 AM. So, I am going to schedule the subscription to execute ten minutes from my current time, or 11:30 AM. To ensure that the subscription only executes once, I am going to choose the radio button next to “Repeat after this number of days” and make sure that a “1” is in the text box. Adjust the “Start time” to about five or ten minutes from whenever you are making these changes (i.e. so if it is 1PM for you, set the “Start time” value to 1:10PM.)

Step 7

Press “Finish” once you have completed the schedule step.

Monitoring the Execution

Once you pressed “Finish” in the last step, you are brought back to the “Subscriptions” page for the “Share Account Details with Dates” report. If you look at the image below, you will see the subscription clearly exists with a status of “New Subscription” meaning that it has not yet been executed.

Pre Execution Status

When the time comes for the report to execute, this status will change. In my case, since I told SSRS to execute the subscription at 11:30 AM, I will wait until 11:30 AM and then refresh this page. You will notice that the row is slightly different now:

Post Execution

Since I have eleven branches in this fake credit union, eleven files are created – one for each branch. If I navigate to my C drive, you can see that all of the files are created properly. Notice that the file name is exactly as we wanted it to appear in the SQL query that we created in step 3.

Files in C Drive

We have successfully created our first data-driven SSRS subscription! Before using a data-driven SSRS subscription, we would have needed eleven separate SSRS subscriptions. Each would have to be manually maintained and managed. Data-driven SSRS subscriptions allow us to create dynamic and efficient subscriptions with significantly less administrative overhead involved with managing them.

There are some very inventive and creative ways in which data-driven SSRS subscriptions have been used in the past. Spend a few minutes searching Google to learn more!

/by

Creating SSRS Subscriptions

, , ,

In our last post, we showed you how to create drill down SSRS reports so you can create incredibly dynamic reports. Drill down reports create dashboard-like functions right within SSRS. For those credit unions trying to maximize their BI investment, initially using SSRS over third-party BI visualization tools can minimize your upfront costs.

What is an SSRS Subscription?

This post will introduce you to creating SSRS subscriptions. But, what is an SSRS subscription? Think of any other subscription you have in your life to a blog, newsletter, or magazine. Your subscription allows you to receive something that you want at a specified time. SSRS subscriptions are very similar; you can subscribe to a particular report and have it delivered to you at a specified time and location.

SSRS allows you to receive reports in a few different ways. The most common delivery methods are to have the report sent to a windows file share or to your email. If you have Microsoft SharePoint in your organization, you can have SSRS reports delivered directly to SharePoint as well. In this particular post, we will show you to how to create a windows file share subscription for a single report. There are some very inventive ways to use SSRS subscriptions beyond what we will discuss in this post. Head over to Microsoft’s SSRS Subscription and Delivery page to learn more if you are interested!

Why use SSRS Subscriptions?

Suppose we have a key weekly lending report that is currently being created the old-fashioned way: manually through Excel. The person creating that report in Excel will complete the report and send it to three or four people. Those three or four people save the copy locally to their drive thus creating multiple copies of the same document. Along with having multiple copies of the same document stored on the shared drive, we are also using unnecessary space on the mail server. IT network and server staff deal with this dilemma regularly.

With SSRS, you could create that report and have the data dynamically retrieved from a SQL database like we have done in past examples. Then, we create an SSRS subscription that places the report output in a specified location at 7 AM every Monday morning. Now, those three or four individuals who used to receive the report in their inbox can simply head to the designated location to view the report. They never have to ask for the report and the person who used to develop the report never has to manually produce and send it again.

Are you that person stuck developing a report manually and then sending it out to various people who request it? Then this post is for you!

Setting up the Report Security

To create subscriptions, we need to first navigate to the report browser. This may be the first time you have done this. To do so, ask your IT/BI area for the SSRS URL. It typically has the form: http://[server name]/Reports. This is the URL that you use to view reports and manage subscriptions. Once you are there, you may need to upload the reports that we created if you haven’t deployed them already.

First, create a new folder on the home page (if you have permission to do so) named “Credit Union Reports”.

Create New Folder

Once the folder is created, click on it to be brought to the page for the “Credit Union Reports” folder. In the SSRS menu, there is an option title “Upload File”. Click that button, browse to where you saved your SSRS reports from previous posts, and select the .rdl files for each report. I will be uploading two reports: “Share Account by Branch” and “Share Accounts Opened”.

If you have done this correctly, the “Credit Union Reports” page of SSRS in your browser will look like the image below.

Folder with Reports

If you click on any of the reports, you should be able to view the output of those reports. This is similar to clicking the “Preview” button for a report in SSDT or BIDS.

Hover over the report named “Share Accounts by Branch”. You should see a dropdown arrow to the right of the report name. Click this arrow and choose “Manage”.

Manage Report

When the browser changes, select the “Data Sources” from the left side of the screen. If the radio button next to “A custom data source” is not selected, select it now. Under “Connect Using:” select the radio button next to “Credentials stored securely in the report server”. Subscriptions require that credentials be stored securely in the report properties. It is recommended that you have a dedicated username with the proper privileges – something like “Data_Warehouse_Reports”. That way, you don’t have to use your own personal credentials in the report data source along with having some other benefits.

Once you choose the radio button next to “Credentials stored securely in the report server”, enter either your personal credentials or the credentials for your dedicated report/data warehouse account (this is more of an IT thing so you may or may not have one). Click the check box next to “Use as Windows credentials when connected to the data source” (this assumes you are using Windows credentials through active directory – which is most common – instead of a SQL Authentication account created for the SQL instance).

Data Source Security setup

Press “Test Connection” towards the bottom of the screen to ensure that all properties and credentials have been entered correctly and that you have the necessary privileges. If everything was entered properly, green text under the “Test Connection” button will appear saying “Connection created successfully”.

Test Connection

Press “Apply” to save your changes.

Using the breadcrumb trail at the top of the browser page, head back to the “Credit Union Reports” page.

Creating the SSRS Subscription

Now that we have stored the credentials in the report, we can now create the subscription. From the “Credit Union Reports” page of the report server, hover over the “Share Accounts by Branch” report. When the dropdown arrow appears, click on it and then choose “Subscribe”. The subscription delivery options page should appear:

Blank Subscription

This screen allows you to specify how, where, and when you would like the report delivered. In the “Delivered by:” dropdown, choose “Windows File Share”. This method will allow us to send the report to a location on our network. When you choose “Windows File Share” from the menu, the screen should change significantly:

Blank Windows File Share

We can leave the “File Name” text box with the default value of the report’s name. The “Path” text box is where we specify the location that we would like the report to be placed. There is an important subtlety to note here: the path must be in UNC (Universal Naming Convention) form. If we wanted to place the report on our C drive, we couldn’t use the path “C:\”. Instead, we would have to type it with the form “\\[Computer or Server Name]\c$”

To keep things simple, let’s place this report on our C drive. In the “Path” text box, type “\\[Computer or Server Name]\c$” where the [Computer or Server Name] is replaced with the name of your computer or server you are working on. For example, if I wanted to place this file on a computer named “Brewster”, I would use the path “\\Brewster\c$”.

In the “Render Format:” dropdown, choose “Excel”. Before we enter credentials, click on “Select Schedule” towards the bottom of the screen.

Select Schedule Screen

This screen allows us to specify when we would like this report to run. We can be very specific with the scheduling process which gives us some nice, flexible options. We want to set this report to run in about five or ten minutes from whenever you are completing the subscription. So, if it is 2pm on a Wednesday when you are following along with this post, be sure to select the checkbox next to “Wed” and set the “Start Time” to about 2:05 pm.

Press “OK” once you have this set up. Again, we want this to run within the next few minutes (if you are following on), so be sure you have your times set up correctly.

Now, you should be brought back to the “Subscription” screen. Go ahead and enter your credentials in the “User Name:” and “Password:” text boxes. I am writing this post at approximately 1:20 pm, so I have scheduled the subscription to execute at 1:25 pm.

Completed Subscription Details

With everything completed as you need, press “OK”. As long as there were no errors immediately flagged by the report server, your SSRS subscription should be scheduled. You should be returned to the “Credit Union Reports” page in SSRS.

Now, wait the few minutes left until the subscription should be executed. In my case, this means waiting until 1:25 pm. While you wait, you can click the “My Subscriptions” hyperlink at the top of the page.

My Subscriptions

When you click that hyperlink, you will be brought to a screen that contains all of your subscription details. Since this is the first subscription we have set up, you should see just a single row with the “Status” field set to “New Subscription”.

My Subscriptions Details

Once the subscription executes, this screen will be updated with the results of the last subscription execution. I have waited a few minutes, so it is past 1:25 pm which is when I had the subscription scheduled to execute. I am going to refresh the “My Subscriptions” page.

Successful Subscription

If you look at the “Status” column now, you will see that the value has changed to reflect the status of the last execution. Since this subscription executed without issue, it tells me that “The file ‘Share Accounts by Branch.xlsx’ has been saved to the “\\XXXXX\c$” file share.”.

To be sure, let’s navigate to the C drive where we told SSRS to place the report.

Windows Explorer Proof of Subscription

We can see that the “Share Accounts by Branch” report has been placed to the C drive in an Excel format just like we expected!

SSRS subscriptions can be incredibly helpful when we need to automate the delivery of reports that we have created. One of the key components of business intelligence is automating and improving the efficiency of our reporting operations. By taking advantage of the inherent features of SSRS, we can drastically reduce the manual effort involved in completing and submitting reports throughout our credit unions.

/by

Drill Down SSRS Reports

, , ,

Our previous post showed you how to create an SSRS report with multiple parameter data types. By doing that, we created even more dynamic reporting opportunities.

An incredibly invaluable function of SSRS is the ability to “drill down” to other reports. The concept of “drilling down” is where you can click on a specific item in the report to get more detail. For example, suppose you have a table showing the count of all members grouped by the state in which they reside. We could create drill down capabilities so that if you clicked on the name of the state, a new report would appear showing the name and address for all members who reside in the state you selected. Effective dashboards frequently leverage the concept of drill downs; we are going to apply this concept to our SSRS reports.

Setting Up our Summary and Detail Reports

When creating drill down SSRS reports, you must have at least two reports: a summary report and a detail report. The summary report will be the report that user will interact with to drill down to additional details. The detail report will contain all of the details we want to display after a user clicks on a specific row or item in our summary report.

In our initial post on building your first SSRS report, we created a report that showed the number of share accounts opened at each branch and the total current balance of those shares. For this post, we will use the report created in our first lesson as our summary report.

Since we already have our summary report, we need to now build a detail report. In this detail report, we want to show details for each account opened for a particular branch. Our goal is to have users be able to click on a branch name from the summary report and then be brought to the detail report for that specific branch.

To start, open up our existing project in SSDT or BIDS if it is not already. In the “Solution Explorer” on the right side of your screen, right-click on the “Reports” folder, hover over “Add”, and select “New Item”. Make sure that “Report” is selected on the window that appears, and give this report the name “Share Account Details by Branch.rdl”. Press “OK” once complete.

Add Detail Report Item Screen

Create the “Data Source” to whatever database in which your data resides. Name this data source “CreditUnion” (see the building your first SSRS report post to review how to do this).

Once the data source is created, right-click on the “Datasets” folder and a new dataset. Using the “CreditUnion” data source we just created, we need to define the SQL query that will populate this report. We are going to include many columns including: member number, account open and close dates, current balance, etc. One of the most critical aspects of the query you place in the new dataset, which we will call “ShareAccountDetails”, is that it includes a WHERE clause filtering on the branch:

ShareAccountDetails Dataset

This parameter is critical to our ability to create the drill down function. Press “OK” to save your dataset if you have not done so already.

You should see the “Branch_Name” parameter in your “Parameters” folder in the “Report Data” panel on the left side of your screen. If you remember back to our last post on creating an SSRS report with multiple parameters, we dynamically defined the available options for the “Branch_Name” parameter using a dataset. That dataset included a distinct list of all branch names from our database. Using the previous post as a guide if you need to review, create a dataset that contains a list of distinct branches. Then, modify the “Branch_Name” parameter so that its available values is defined by the branch list dataset. Unlike the last post, set the “Branch_Name” parameter to only accept a single value instead of multiple values. The “Available Values” section of the “Report Parameter Properties” window should similar to the image below:

Branch_Name Parameter

Press “OK” to save the parameter options if you haven’t done so already.

Next, we need to create a table in our report for the data to reside in once rendered. Either drag a blank table from the toolbox on the left side of your screen, or right-click on the blank report, hover over “Insert”, and select “Table”. Move the table to the top left corner of the report window.

Now, we need to add columns from our “ShareAccountDetails” to our table. Remember to hover in a cell of the data row (bottom row) of the table and wait for the table icon to appear. When we completed this when creating our first SSRS report, we only had one dataset. Since we have multiple datasets in this report, you must choose both the data source and the dataset from which you want to retrieve data:

Add Dataset Columns to Table

That was a subtle difference, but one that is certainly worth mentioning. To keep things easy, add each column of the “ShareAccountDetails” dataset to the table in our report.

Just like we have done before, make the header row bold and resize each column’s width to fit. In our previous posts, we formatted the column’s data type by right-clicked on a cell and selecting “Text Box Properties”. Make sure to format the “Account Opened Date”, “Account Closed Date”, and “Current Balance” columns appropriately.

Before previewing your report, it should look similar to the image below:

Pre Preview Table

Go ahead and preview this report now.

Detail Report Preview

This report is pretty basic, but it accomplishes what we need it to for the time being. At this point, we can say we have completed our “detail report”.

Creating the Drill Down Function

At this point, we have created both a summary report and a detail report. Now, we need to define the drill down action to tell SSRS what we want to happen when users click on the branch name from the summary report.

First, open up the summary report named “Share Accounts by Branch”. You can do this by double-clicking on the report in the “Solution Explorer” on the right side of your screen. Below is a reminder of what this report looks like:

Share Accounts by Branch

To begin creating the drill down action, right click on the “Branch_Name” cell in the data row (bottom row) of the table. Select “Text Box Properties”.

Branch Name Right Click

Select the “Action” option from the left side of the “Text Box Properties” window. This section of the “Text Box Properties” window allows you to define actions that occur when you click on the values in this cell or column. Choose the radio button next to “Go to Report”.

At this point, we are telling SSRS that we want to go to some report when we click on the values in this column of our report. Next, we need to define for SSRS what report we want to view and pass any necessary parameters to the detail report.

In the dropdown menu beneath “Specify a report”, choose the “Share Account Details by Branch” report that we created earlier in this post. Below that, press the “Add” button. If you remember in our detail report, we had a parameter named “Branch_Name”. We want this parameter to be automatically defined based on whatever branch we click on in our summary report.

In the “Name” dropdown, select “Branch_Name”. This dropdown contains a list of parameters from the “Share Account Details by Branch” report. Next, in the “Value” dropdown, choose “[Branch_Name]”. The “Value” dropdown indicates what value from the current report do we want to pass to the detail report. In this case, we want to pass the name of the branch that we select. The “Text Box Properties” window should look like the image below:

Branch Name Text Box Properties

Press “OK” once you have made the necessary updates to the “Action” section of the “Text Box Properties” window.

Go ahead and preview your report now. Try hovering over one of the branch names in the table; you will notice the cursor changes. Choose one of the branch names and click on its name. If everything was set up correctly, you will be brought the detail report with the “Branch_Name” parameter automatically populated based on whatever branch you initially clicked.

That’s it! That is all that it takes to create a drill down SSRS report. It all comes down to a few simple questions:

  • What report do I start with (our summary report)?
  • What report do I want to drill down to (the detail report)?
  • What do I want to click on to be sent to the detail report?
  • What values do I want to pass from my summary report to parameters in the detail report?

Those basic questions sum up the entire process of creating a drill down report within SSRS.

/by

5 Steps to Start Developing your Business Intelligence Strategy

, ,

Big Data, Analytics, and Business Intelligence have become the most commonly used buzzwords in business.  Despite their varying uses and connotations, each boils down to a single proposition: how can we best use our data?  

The answer may vary dramatically from credit union to credit union.  Perhaps your goal is to improve marketing returns through increased analysis into member product propensity; for others it may be to improve reporting and operational efficiency through automation and data centralization.  No matter what the specific aim of your business intelligence strategy is, the ultimate goal is always to use data more effectively.

With the advancements in software and technology, business intelligence technology is now within any credit union’s reach.  However, simply having business intelligence software does not mean you have business intelligence.  Credit unions must invest time and resources to develop a successful Business Intelligence/Data Strategy (BIDS) that is both successful and sustainable.  Below are several points to consider to help you get started.

1. This is not purely a technology initiative; ensure the support of key business areas.

Many equate business intelligence with software and technology.  While this is certainly true to an extent, business intelligence is equal parts business and technology.  Successful business intelligence programs are designed to most effectively cater to the data, analysis and reporting needs of business functions of the credit union.  However, understanding the technology involved in developing your business intelligence environment is critical to understanding what is possible within the project’s scope of time and budget.  Sustainable Business Intelligence/Data Strategies leverage key insights held by both business users and technology professionals.

Before beginning your business intelligence investment, it is recommended that you receive buy-in from various departments throughout the credit union.  Lending and Marketing are two departments that typically benefit the most from BI; describe the benefits these areas could receive and ensure their support for the business intelligence initiative.

2. Develop a "Data Dictionary"

Do the terms account, member, individual, and household mean different things to different departments within your credit union?  Perhaps your MCIF does not include charge-off accounts, yet account-level reporting from your core typically does include these accounts.  Member-level reporting may involve different filtering criteria when different departments complete the same request.  Does your information systems department report slightly different account totals than your Accounting/GL application?

These subtleties are not uncommon, yet they represent an important aspect of developing your business intelligence strategy: not all terms mean the same thing to everyone.  A critical step to a successful enterprise business intelligence/data strategy is creating a "data dictionary".  Your data dictionary will include a list of terms used in the business and their specific definition as it will be implemented within your business intelligence program.  Key personnel in all areas of the credit union should agree upon these terms.  Term ambiguity can cause misinformation and lack of clarity for reports and dashboards – a major detriment to the success of your business intelligence/data strategy.

3. Assess your current business intelligence environment

Being realistic about your current business intelligence and data utilization environment is essential to developing a realistic strategy roadmap.  Are users consistently compiling reports?  Do relatively simple data requests take days to complete?  Are departments storing silos of information in stand-alone Access database or Excel spreadsheets?  Do you have staff competent with SQL?  All of these questions are essential to assessing your credit union’s current BI environment and BI readiness.

An invaluable exercise in assessing the current state of your business intelligence environment is to compile a "data inventory".  In this "data inventory", you will identify every data source within the credit union, which department or individual is responsible for that data source, and what type of data is stored in the source.  This is often a mix of Excel files, Access Databases, SQL Databases, and various credit union applications (core, residential LOS, residential/mortgage LOS, MCIF, etc.).  By compiling the "data inventory" you gain a tremendous amount of knowledge about your credit union’s current data landscape.  If you engage with an outside vendor to develop your business intelligence solution – as do most credit unions – the "data inventory" is invaluable in helping your vendor or consultant gain insight into the data environment of your organization.

4. What do you really want to measure?

Data warehouses are often the foundation of a business intelligence strategy.  In short, a data warehouse integrates data from various sources.  The data warehouse is designed for the purpose of reporting and analysis.  As such, it is critical to understand what your credit union seeks to measure before beginning to develop a data warehouse.  

A great way to assess what your credit union most frequently measures is to compile a "report inventory".  The "report inventory" is a list of all reports that are produced by the credit union.  It is best to focus on recurring reports, but it also valuable to understand what types of reports are being requested on an ad hoc basis.  In the "report inventory", capture which department compiles the report, who the report recipient is, how long the report takes to compile, what information is captured in the report, and how frequently is the report needed (i.e. daily, weekly, monthly, quarterly).  If you choose to engage with a business intelligence consultant, a "report inventory" allows the consultant to understand the type of data that is often requested and develop the data model accordingly.

5. Are you willing to invest in the long-run?

A successful business intelligence/data strategy is not a short-term solution.  It is not like Microsoft Excel where you buy it, install it, and begin using it in a matter of minutes.  Business intelligence solutions are best developed in smaller, targeted phases while keeping the larger picture in mind.  

For example, the first three months after beginning your business intelligence program might involve much of the analysis as to the current state of your organization’s data environment.  Simultaneously, you may be training a select number of staff on how to write SQL queries or develop reports and dashboards. 

In budgeting for your business intelligence initiatives, remember that this is not an immediate, one-time purchase.  A successful data program requires continuous investment as the data needs of your organization grow.  In nearly all business intelligence/data strategy engagements, you should be looking to institute a minimum of an 18-36 month roadmap. 

Now What?

Quite a bit is involved when developing a successful business intelligence/data strategy.  It involves a combination of business and technical expertise while simultaneously balancing the goals of your credit union, timeline, and costs.  To summarize, when beginning to develop your initial business intelligence/data strategy you should consider the following key points:

  • This is not purely a technology initiative; ensure the support of key business areas.
  • Develop a "data dictionary"
  • Assess your current business intelligence environment
  • What do you really want to measure?
  • Are you willing to invest in the long-run?

 

The Knowlton Group has partnered with Rich Jones, a 20+ year credit union executive and strategist, to create a turnkey business intelligence/data strategy consulting solution – providing both strategic solutions and technical expertise.  Together we will help you define a business intelligence/data strategy roadmap while also providing the necessary business intelligence technical expertise to develop a successful and sustainable data-driven culture.

Visit www.knowlton-group.com or email brewster@knowlton-group.com to find out more information!