Data is the currency of digital transformation. Having available data that is understood, organized, and believable strengthens all major corporate initiatives. However, maintaining this basic resource is a growing challenge for most organizations because sources and volumes of interesting data are expanding rapidly. The cloud and the proliferation of SaaS companies has contributed to the data explosion. While the possibilities of the cloud and its many applications can quickly grow the capabilities of an organization, the data spread it creates can lead to problems such as decentralized data leading to inaccurate findings, or wasted time spent rebuilding pipelines instead of driving results.
In this report, we compare the three major data pipeline platforms: Matillion, Stitch, and Fivetran; and run them through a series of selected tests that highlight their degree of automation, ease of setup, and documentation. We evaluated aspects that include the time and effort required to set up a sourcedestination connection, the degree of automation throughout the process, and the quality of documentation to support the effort. These areas address the three major “humps of work” we have encountered in our field work with data pipelines.