ETL Tools and Build Tools

Posted by Ngu Soon Hui on Stack Overflow See other posts from Stack Overflow or by Ngu Soon Hui
Published on 2009-11-12T01:45:14Z Indexed on 2010/05/20 1:40 UTC
Read the original article Hit count: 710

Filed under:

etl

|

build-tools

|

database

I have familiarities with software automated build tools ( such as Automated Build Studio). Now I am looking at ETL tools.

The one thing crosses my mind is that, I can do anything I can do in ETL tools by using a software build tool. ETL tools are tailored for data loading and manipulation for which a lot of scripts are needed in order to do the job. Software build tool, on the other hand, is versatile enough to do any jobs, including writing scripts to extract, transform and load any data from any format into any format.

Am I right?

© Stack Overflow or respective owner

Related posts about etl

SQL SERVER – Introduction to Adaptive ETL Tool – How adaptive is your ETL?

as seen on SQL Authority - Search for 'SQL Authority'
I am often reminded by the fact that BI/data warehousing infrastructure is very brittle and not very adaptive to change. There are lots of basic use cases where data needs to be frequently loaded into SQL Server or another database. What I have found is that as long as the sources and targets stay… >>> More
SQL SERVER – 4 Tips for ETL Software IDE Developers

as seen on SQL Authority - Search for 'SQL Authority'
In a previous blog, I introduced the notion of Semantic Types. To an end-user, a seamlessly integrated semantic typing engine significantly increases the ease of use of an ETL IDE (integrated development environment, or developer studio). This led me to think about other ease-of-use issues I have… >>> More
PostgreSQL to Data-Warehouse: Best approach for near-real-time ETL / extraction of data

as seen on Stack Overflow - Search for 'Stack Overflow'
Background: I have a PostgreSQL (v8.3) database that is heavily optimized for OLTP. I need to extract data from it on a semi real-time basis (some-one is bound to ask what semi real-time means and the answer is as frequently as I reasonably can but I will be pragmatic, as a benchmark lets say we… >>> More
ETL Operation - Return Primary Key

as seen on Stack Overflow - Search for 'Stack Overflow'
I am using Talend to populate a data warehouse. My job is writing customer data to a dimension table and transaction data to the fact table. The surrogate key (p_key) on the fact table is auto-incrementing. When I insert a new customer, I need my fact table to reflect the id of the related customer… >>> More
Kimball University: Three ETL Compromises to Avoid

as seen on SQL Server Central - Search for 'SQL Server Central'
Why neglecting slowly changing dimensions, failing to capture metadata and overlooking scope creep can be the undoing of a dimensional data warehousing initiative. >>> More

Related posts about build-tools

ETL Tools and Build Tools

as seen on Stack Overflow - Search for 'Stack Overflow'
I have familiarities with software automated build tools ( such as Automated Build Studio). Now I am looking at ETL tools. The one thing crosses my mind is that, I can do anything I can do in ETL tools by using a software build tool. ETL tools are tailored for data loading and manipulation for which… >>> More
Build tools for php, html, css, js web app development

as seen on Programmers - Search for 'Programmers'
What are some recommendations for a build tool that would allow me to upload changes to a web server or a repository and minify the js and css automatically, and possibly even run Closure compiler on the JavaScript? Im not worried about doing anything with the php code other than update with most… >>> More
make-like build tools for data?

as seen on Programmers - Search for 'Programmers'
Make is a standard tools for building software. But make decides whether a target needs to be regenerated by comparing file modification times. Are there any proven, preferably small tools that handle builds not for software but for data? Something that regenerates targets not only on mod times… >>> More
MSBuild appears to only use old output files for custom build tools

as seen on Stack Overflow - Search for 'Stack Overflow'
I have an ANTLR grammar file as part of a C# project file and followed the steps outlined in the User Manual. <Project ...> <PropertyGroup> <Antlr3ToolPath>$(ProjectDir)tools\antlr-3.1.3\lib</Antlr3ToolPath> <AntlrCleanupPath>$(ProjectDir)AntlrCleanup\$(OutputPath)</AntlrCleanupPath> … >>> More
Replacing build.xml with Build.java - using Java and the Ant libraries as a build system

as seen on Stack Overflow - Search for 'Stack Overflow'
I've grown disillusioned with Groovy based alternatives to Ant. AntBuilder doesn't work from within Eclipse, the Groovy plugin for Eclipse is disappointing, and Gradle just isn't ready yet. The Ant documentation has a section titled "Using Ant Tasks Outside of Ant" which gives a teaser for how to… >>> More