Data integration is the process of combining data from different sources into a single, unified view. Integration begins with the ingestion process and includes steps such as cleansing, ETL mapping, and transformation. Data integration ultimately enables analytics tools to produce effective, actionable business intelligence.
When it comes to your marketing technology stack, that source data can be complex. It can live as social ads data in a Facebook data center, or in Excel files on a colleague’s laptop. We’ve found that, on average, marketing departments and agencies use more than 40 different technologies on a daily basis that map to various stages of the customer journey. Before integration, data exist in siloed data centers around the globe, in different formats, and with different protocols for access. Data integration means getting all this data under the same roof– and all speaking the same language.
Importance of data integration for marketers
In his 2016 Cool Vendor report, Gartner analyst Marty Kihn described data integration as the “critical behind-the-scenes part of the marketing stack.” Data integration is like the unifying glue between all of marketing’s performances, outcomes, investments, and stakeholders. By integrating all of your marketing data from your digital campaigns, offline campaigns, web analytics, CRM and post-sale systems (to name a few), marketers can create a seamless and joined a view of the customer journey that aids in smarter decisions making. This is often referred to as the single source of truth.
With this single source of truth, marketers can manage and optimize marketing holistically through KPIs, metrics, and dimensions that don’t exist in your individual source systems. These include aggregate metrics such as your total impressions, engagements, and conversions for all of marketing’s programs. They also include efficiency metrics like media spend pace, cost per engagement, customer acquisition cost and marketing ROI. These business-level metrics, commonly viewed through marketing dashboards, all depend on the foundational data integration layer.
Cutting edge data integration for marketers
The relationship between marketers and data integration is unique. This is largely in part because marketing is responsible for attracting, engaging, converting and retaining audiences and customers that move across a myriad of touchpoints. In order to scale marketing’s performance and impact, a connected view is needed to evaluate which programs, channels, campaigns, audiences, etc. are best driving their strategic goals.
Getting on a new platform or switching an old one out for a new requires fast data integration to keep the single source of truth moving ahead without interruption. Similarly, new analytics are continually added to different execution systems meaning that current connections need to be flexible to update quickly. And finally, marketers often classify their key metrics and dimensions in like terms across systems– for example, in UTM parameters on landing pages.
Marketers need a simple way to line all of those classifications up so that data can be compared from a CMO-level view. When these classifications change or diverge, agile realignment is essential.
Data integration challenges
Data integration is often discussed around the subject of business intelligence or BI. That’s because traditional BI tools were some of the first to take on the task of integrating data from different sources to help business aggregate data from and across different departments. This type of data integration is often characterized as an exhaustive and time-consuming process that requires advanced technical and coding skillsets. In fact, within organizations that possess these scarce talents, data integration can take up to 80% of data scientists’ time as Forbes recently covered. For organizations without these skills, data integration is often handled on a consulting or outsourced basis with specialists.
Cutting edge data integration approach for marketers
As the number of marketing data sources available (and even required) has exploded, the traditional model of using specialists to build connections is no longer sustainable for business. That’s why most modern analytics tools offer some number of built-in connections to common databases (like Hadoop and BigQuery), and the most-used APIs (like Facebook and Google AdWords). These steps have been a positive direction to making data integration more marketer-friendly, but they’re still not enough to keep pace.
The next generation of data integration for marketers takes a hybrid approach that blends API connectors with AI machine learning to automate connections to any data source regardless of an API connection. This approach enables out-of-the-box connectivity for every system and reporting output. Datorama’s data integration is the first to spearhead the hybrid approach for marketers. TotalConnect, for example, is a first-of-its-kind data integration technology that allows you the create API-like connections for any data source using this method The business result for marketers is a single source of truth that can be updated and changed quickly and scaled across markets, teams and other solutions without advanced technical skills. This puts marketing in total control.
Relation of “ETL” and data integration
One important part of data integration is ETL. This is a term that is often used interchangeably with data integration– you may have heard it used by technical practitioners in conversations about business intelligence. Extract, transform and load is three database operations responsible for actually moving your data into a common database. Extraction reads the data in the original database, transformation changes the format so it’s ready for querying and analysis, while loading writes the data to your destination database.
ETL is perhaps the most issue-prone part of data integration because an error in one step causes inaccurate or missing data throughout. And each system comes with its own set of unique types of problems it can have. That’s why the right technology is so important in the ETL process, so you can focus less on micromanaging the movement of data and have more time to spend on analysis and decision making.
