Your numbers live in five or ten different systems, none of them agree with each other, and someone has asked you to go pick a tool that fixes it. Search “data integration software” and you get ranked vendor lists scored on connector counts and ease of setup, nearly all of them written on the assumption that buying a packaged platform is the right move. Often that’s the right call. But nobody in those posts ever checks it, and it sits underneath the part of this decision where the most money is at stake.

We build integration pipelines as production software, so we spend most of our time in the part of the decision those guides skip: what a tool takes off your plate, what it leaves sitting there for you, and how to work out in advance which of those two lists is longer at your company. This is the framework we walk clients through before anyone opens a pricing page.

The phrase is broad enough to be almost unhelpful on its own, which is why the category feels muddled the moment you start shopping. Four rough families sit underneath it, and a lot of the confusion comes from comparing tools that belong to different ones.

• ETL and ELT tools pull data out of source systems and land it in a warehouse, transforming it either on the way in or after it arrives. Fivetran, Airbyte, Matillion, and Stitch live here, and this is usually what people mean when they say “data pipeline.”
• iPaaS and workflow platforms connect applications directly to each other instead of routing everything through a warehouse. Boomi, MuleSoft, SnapLogic, and Workato sit in this family, with Zapier at the lighter end. The unit of work is a triggered flow between two systems.
• EDI and B2B integration handles the document formats trading partners still run on, things like purchase orders, invoices, and advance ship notices, over whatever protocol each partner requires. If you sell into large retailers, you already know this world exists.
• Transformation and analytics tools such as dbt sit downstream of all of the above, turning raw loaded tables into modeled ones your reporting can trust.

Vendor comparisons tend to grade every one of these on the same scorecard: how many prebuilt connectors ship in the box, how deep the warehouse integration goes, whether you can drop into code when the visual builder runs out of room, how pricing scales, how long implementation takes, and how steep the learning curve is. Those are reasonable questions and you’ll want answers to them eventually, but none of them is the first thing to settle.

A good connector-based tool solves a real problem, and a tedious one: moving bytes from A to B, on a schedule, without you writing and maintaining the plumbing. It handles authentication and token refresh, pagination, rate limits, retries, schema drift in the source, and incremental syncs so you’re not reloading all of history every night. That’s weeks of unglamorous engineering per source, and it’s roughly the same weeks at every company, which is exactly why you should buy it instead of building it.

So if your situation is a handful of well-known SaaS systems, a warehouse, and reporting on top, a packaged tool is probably the right answer, and you can stop reading comparison posts and pick one on connector coverage and support quality. Salesforce to Snowflake, Shopify to BigQuery, HubSpot to Postgres are all paved paths, and paying someone else to keep them paved is a good trade.

What a tool can’t buy you is judgment about your data. It’ll pull a field called revenue out of every system you point it at, faithfully and on schedule, but whether those fields all mean the same thing is still your problem.

Three situations turn a solved problem back into an unsolved one, and they’re worth checking for before you shortlist anything.

The first is definitional disagreement between sources. Two systems both report revenue and the numbers differ, because one counts at order placement and the other at fulfillment, or one is net of returns and the other is gross, or one is in the customer’s currency and the other has already been converted. A connector will deliver both numbers faithfully and has no opinion about the gap between them. Reconciling them into a single number the business can act on is modeling work, and nobody ships a connector for your definitions.

The second is business logic no vendor expresses: allocating shared costs across channels, applying your contract terms to raw transaction lines, or deriving a margin that accounts for the fees your specific agreements carry. We wrote about the e-commerce version of this in analytics engineering for e-commerce teams, where channel fees and returns pull the reported top-line number away from the one worth managing. The general shape holds anywhere: if the useful metric depends on a rule that lives in someone’s head or in a spreadsheet they maintain by hand, no integration tool can compute it for you.

The third is data that revises itself after the fact. Plenty of sources restate history: an order gets refunded, a trading partner corrects last month’s file, an ad platform trues up attribution a week later. A pipeline built on the assumption that yesterday’s data is final goes wrong quietly, because the rows still load on schedule and the dashboard still renders. The numbers on it just aren’t the ones you reported last week anymore. Handling revision properly takes incremental refresh logic that knows what to reload and when, and that logic is specific to how each source behaves.

License cost is the visible line, so it’s what gets compared across vendors and what lands in the budget request. It also tends to be the smaller and far more predictable half of the total.

The half that grows is engineering. Someone maps each source’s fields to your model, writes and tests the transformation logic, builds the monitoring that tells you when a sync failed or a row count fell off a cliff, and carries the pager when a source changes its schema on a Tuesday with no notice. That cost scales with the number of sources and the strangeness of your business logic, not with which tool you picked. Buying a platform lowers it, sometimes by a lot, but it doesn’t take it to zero, and teams that budget as though it did end up paying for a tool and staffing a person to keep it running anyway.

The other cost worth naming up front is ownership, because a pipeline nobody owns decays at exactly the speed its sources change. When we build one, we treat it as software instead of a configuration exercise: it goes in version control, it has tests, and something watches it and complains when a load fails. Our client projects are maintained at 80 to 90 percent test coverage, and we hold data pipelines to the same bar as application code, because a silent data error costs more than a loud application error and usually takes weeks longer to notice.

How many sources do you have, and how often do they change? Below roughly five stable, well-known systems, connector coverage decides it and a packaged tool wins comfortably on cost of effort. As the count climbs and the sources get stranger (a partner’s SFTP drop, a legacy internal database, a vendor portal with no real API), the chance that any one tool has a maintained connector for all of them drops fast. Each gap becomes custom work you’re doing anyway, inside a platform that also charges you for the privilege.

Do the numbers already agree? You can test this one in an afternoon: ask two systems for the same metric over the same period and compare them. If they match, or differ in a way someone can explain in a sentence, you have a movement problem, and tools solve movement problems well. If they differ and nobody can say why, you have a modeling problem wearing a movement problem’s clothes, and buying a faster way to move disagreeing numbers into one place just gets you a dashboard nobody trusts.

How idiosyncratic is your business logic? Standard metrics coming out of standard systems are a solved problem, and you should treat them as one. Metrics that depend on your contracts, your fee structure, your channel mix, or a definition your finance lead has carried in their head for six years are code that someone has to write. The only real question there is whether that code lives in a transformation layer you control and can test, or in a spreadsheet that breaks the week its author goes on vacation.

For most businesses the answer turns out to be both, in a fairly specific division of labor. Use a packaged tool for extraction wherever a maintained connector already exists, because reimplementing Salesforce authentication is a poor use of your budget. Then build the modeling layer on top of it, where your definitions live, because that layer encodes how your business works and nobody sells a version of yours.

Buy outright when the sources are few and standard, the definitions already line up, and your reporting needs look like everyone else’s. Lean toward building when the sources are numerous or unusual, when the numbers disagree in ways only your team can settle, or when the metric that runs the business exists in no vendor’s schema.

When the build side wins, the work has a predictable shape. On our Databricks engagements it runs from workspace setup through connecting the data sources, then data modeling, then incremental refresh pipelines that keep everything current as sources revise themselves, and finally the reporting layer people actually open. Databricks consulting covers how we scope that work. Adding AI features on top of the result is a separate build with its own decisions, and we’ve written about that one in artificial intelligence integration.

Whichever way you go, settle the modeling question before the vendor question. You can swap the tool later without much drama, but the definitions your reporting is built on get much harder to change once the company has spent a year making decisions with them.

If you want a second opinion on which side of this you land on, the fastest way to get one is to be concrete about it. List the systems your data currently lives in, then name one number that two of them disagree about. Those two things are usually enough for us to tell you whether a packaged tool covers your situation or whether the disagreement itself is the real project. Send them over and we’ll give you a straight answer.