SharePoint list to a database: every route compared

The list began as a simple tracker. Now it holds tens of thousands of rows, the Power BI refresh times out twice a week, finance wants to join it to the ledger, and the flow someone built to copy it into SQL stopped picking up deletions months ago. You search for a way to get a SharePoint list into a database and get a dozen answers, most of them from a company selling one.

This page puts those answers side by side. It covers the Microsoft routes, the routes you build yourself, the vendor connectors and SPSync, and asks each one the same eight questions. Every row restates a page on this site that covers that route in full, with its sources and the date we checked them, and the row links to it. It applies equally to a list you open in Microsoft Lists, since that is a SharePoint list underneath.

How do I get a SharePoint list into a database?

There are two broad kinds of route. One kind reads the list where it sits, every time someone asks: an Excel or Access link, a Power Query connection in Power BI, or a Dataverse virtual table. Nothing is copied, so nothing can fall behind, but every reader pays the cost of a live SharePoint query, and SharePoint is built to keep those queries small. Past the list view threshold of 5,000 items, a query that filters or sorts on a column with no index is refused.

The other kind keeps a copy in a database and refreshes it. A Power Automate flow, a PnP PowerShell or Python script, an SSIS package, a Data Factory pipeline, Fabric mirroring, a vendor connector and SPSync all belong here. Reports and apps then read the copy at database speed. What separates the copying routes is how much of the job you end up writing yourself: turning lookup columns into something a query can join, noticing that an item was deleted, reading a large list without tripping the threshold, and running on a schedule someone owns.

The routes side by side

Each cell describes the route as it ships, before anyone adds code around it. A script or a pipeline can close any of these gaps if you are willing to own the code that does it.

Route Lists or files Direction Lookups resolved Deletes Past 5,000 items Scheduling Where it runs Cost model
Excel and Access links List items, over a live link A link from the file to the list, read on refresh Looked up row by row on each refresh No record kept; a refresh shows the list as it stands now Refresh slows as rows and lookups grow; a big list can fail to open Whenever someone refreshes the file The PC of whoever has the file open Part of the Excel or Access licence you have
Power Query in Power BI List items, through the SharePoint Online list connector Into the report's own model Worked out when the query runs; never kept as keys Untracked; a deleted item drops out at the next refresh Filters on unindexed columns meet the threshold; v2.0 allows 12 joins per query The whole list is read again at every refresh Excel, Power BI Desktop or the Power BI service Comes with Excel or Power BI licences
Power Automate and Logic Apps Items and files, an action at a time Whichever way each flow was built Each field mapped by hand inside the flow A second flow on the deletion trigger, open to site collection admins only Get items needs pagination switched on, and the cap follows the performance profile On each item event, or on a recurrence Microsoft's cloud, plus a gateway to reach an on-premises database Per-user or per-flow plans; the SQL Server connector is Premium
PnP PowerShell and Python scripts Whatever the script asks Graph or the REST API for Whatever the script writes Code you write and then maintain Detection is yours to write, often by diffing full reads Paging works; filtering on an unindexed column still trips the threshold Task Scheduler, cron or a CI job Any machine that runs the script The tools are free; the engineering time is the bill
SSIS packages List items, from an OData source or a third-party component Into whatever the package writes to Ids or text, joined inside the package Missed unless the package compares full loads Depends on how the source component pages through the list A SQL Server Agent job A Windows server running Integration Services Ships with SQL Server; a third-party component carries its own licence
Azure Data Factory, Fabric Dataflow Gen2 and Copy job Lists, through the SharePoint Online List connector Out of SharePoint into a pipeline sink or a Fabric item Data Factory's mapping turns lookups and people into Int32 ids Undocumented for the Data Factory connector; a Copy job reloads in full A filtered query on an unindexed column can hit the threshold Pipeline triggers; Dataflow Gen2 refreshes up to 48 times a day Azure, or a Fabric capacity Activity and runtime charges, or Fabric capacity units
Fabric mirroring for SharePoint lists Lists, into Delta tables in OneLake, in preview One way, into OneLake only Learn does not say how lookup columns arrive Not described on Learn's SharePoint list pages Learn publishes no limitations page for the SharePoint list source Continuous, near real time by Microsoft's account A running Fabric capacity Replication compute is free; queries draw on your capacity
Dataverse virtual tables Lists, surfaced as a virtual table No copy at all; every read passes through to SharePoint Kept inside Dataverse's own relationships, with no SQL export Live, because nothing is copied Each read is a live call to the list No schedule needed Dataverse, inside Power Platform Dataverse licensing through Power Apps or Dynamics 365
Vendor connectors Lists; Fivetran handles files with a separate connector Mostly one way; Skyvia and Layer2 also run two-way Fivetran writes an id column; others map generically or as display text Set by the product and the sync mode you choose Paging happens inside each connector; each comparison covers it The vendor's own scheduler, on an interval or plan you pick A vendor cloud, a self-hosted install, or a Windows Service for Layer2 Monthly Active Rows, record allowances, connection bands or per-connector licences
SPSync Lists and document library metadata; file contents stay in SharePoint One way, from SharePoint to your destination Key columns you can join on, with multi-value columns as child tables In database tables the row stays, flagged with sp_deleted_at, by default Graph delta reads with paging; 10,303 items across four lists read on a live tenant Every five minutes by default, plus a reconcile, weekly by default The desktop app on Windows or macOS, the command line on your server, or your own Azure Licence per installation and destination family, quoted after a call

Microsoft sources, checked on 23 October 2026: the list view threshold, Power Automate limits, the SharePoint connector reference, the Power Query SharePoint Online list connector, the Data Factory SharePoint Online List connector, the Fabric connector overview, Dataflow Gen2 refresh and mirroring a SharePoint list in Fabric. Vendor details come from each comparison page, where every figure carries its source.

What each route does well, and where it stops

Excel and Access links

A workbook connected to a list, or an Access database with linked tables, is often the first copy anyone makes, and for a few hundred rows it works well. The trouble grows with the list. Each refresh fetches the lookups row by row, so Refresh All gets slower with every item, and a large list can fail to open at all. The link also lives in one person's file. The fix that keeps the workbook is to point it at a database table that something else keeps current, which Excel and Access refreshing from SQL walks through.

Power Query in Power BI

Power Query reads the list live on every refresh. That is quick to set up and fine for a small list behind one report. A larger list means many pages of requests on each refresh, a filter on an unindexed column runs into the threshold, and the v2.0 connector is limited to 12 join operations per query, which a list with many lookup and person columns can exceed. Lookups are resolved inside the report, so two reports on the same list each do the work again. SPSync vs Power Query covers when to stay direct, and fixing slow and failing refreshes covers the symptoms.

Power Automate and Logic Apps

Flows are good at reacting to one item: an approval, a Teams message, an update to another system. Keeping a whole table in step with a whole list is a different job. Every item is its own action against the connector's call limits, field mappings are built by hand, and new columns need someone to edit the flow. Deletes need a second flow on the deletion trigger, which Microsoft's SharePoint connector reference limits to site collection admins. Get items must have pagination switched on, and Microsoft's Power Automate limits cap paginated items according to the plan's performance profile. Writing to SQL Server also needs the Premium connector. SPSync vs Power Automate and Logic Apps sorts the flows worth keeping from the ones worth retiring.

PnP PowerShell and Python scripts

A script gives you total control and costs nothing to download. PnP PowerShell and the Python clients for Graph can read any list, and a nightly export is a morning's work. The cost arrives later, when the script becomes something people rely on. Lookups, multi-value columns, delete detection, throttling and retries, a scheduler and the alerting when it fails all become code that someone owns, usually the person who wrote it until they change jobs. SPSync vs PnP PowerShell and custom scripts lists what you take on, and what it costs to keep an export script running prices it.

SSIS packages

Teams with an Integration Services estate often add the list as another package. The list comes in through an OData source or a third-party SharePoint component, and the package runs from a SQL Server Agent job like everything else on that server. Lookups arrive as ids or text for the package to join, a deleted item is only noticed if the package compares one full load against the last, and the package metadata needs a refresh whenever someone changes a column. How a large list behaves depends on how the component pages through it. The SSIS column in the SharePoint to SQL Server route table sets these out against PowerShell, flows and Data Factory, and SPSync vs SSIS covers the built-in OData source on SharePoint Online and the third-party components.

Azure Data Factory, Fabric Dataflow Gen2 and Copy job

Data Factory and the Fabric data tools read lists through the SharePoint Online List connector, and they make sense when the list is one source among many in a data platform you already run. In Data Factory's documented type mapping, lookup and person columns arrive as Int32 ids with no display value, and the connector says nothing about deletes. Microsoft's Fabric connector overview lists the Copy job as full load only, so each run reads the whole list again. Dataflow Gen2 can refresh up to 48 times a day. The Data Factory, Fabric, Power Automate or a replica guide takes these one question at a time, and the comparisons with Azure Data Factory and Fabric dataflows go further.

Fabric mirroring for SharePoint lists

Microsoft can now mirror a SharePoint list into OneLake continuously, as Delta tables with a read-only SQL analytics endpoint. The SharePoint List source is listed as preview in Microsoft's mirroring overview, it needs a running Fabric capacity, and replication compute carries no charge. Learn's SharePoint list pages do not say how lookup and person columns arrive or what happens to deleted items, so check both on one mirrored list before you build on it. If every report already lives in Fabric, mirroring is a strong default. If the list needs to reach a database outside OneLake, it does not get you there. The Fabric comparison and SharePoint to Microsoft Fabric cover both cases.

Dataverse virtual tables

A virtual table shows a list inside Dataverse without copying it, so a Power Apps app sees current data and deletes show up straight away. Each read goes back to SharePoint, the relationships live inside Dataverse's own model, and nothing outside Power Platform gets a SQL table to query. It fits a single app that needs live list data in Dataverse. SPSync vs Dataverse and virtual tables sets out where it fits, and SharePoint to Dataverse covers copying lists into real Dataverse tables for apps that need delegation.

Vendor connectors

General data tools with a SharePoint connector handle lists alongside dozens of other sources. Fivetran syncs lists through its Microsoft Lists connector and charges on Monthly Active Rows, writing each lookup as an id column you can join. Airbyte has a free self-hosted core and a Marketplace Microsoft Lists connector that runs full refreshes, and CData Sync reaches lists through a generic driver on an annual licence. Skyvia and Layer2 Cloud Connector both sync in two directions, which SPSync does not. None of them documents multi-value columns split into child tables, so expect to model those yourself. Each has its own page: Fivetran, Airbyte, CData Sync, Skyvia and Layer2 Cloud Connector.

SPSync

SPSync does one job. It replicates the lists and library metadata you pick, one way, into a database, warehouse or index you run, and SharePoint stays the system of record. After the first full load it reads only what changed through Graph delta queries, every five minutes by default, with a reconcile that re-checks every item, weekly by default, as quality assurance and for recovery after an outage. Lookups arrive as key columns you can join, multi-value columns as child tables, and in a database a deleted item keeps its row with sp_deleted_at set. In testing against a live tenant it read 10,303 items across four lists, one of them past the threshold. It runs as a desktop app on Windows or macOS, as a command-line tool on your own server, or in your own Azure subscription, and the product overview covers each of those. File contents are not copied, and nothing is written back to SharePoint.

Which SharePoint list ETL tool?

Start with who reads the copy and how fresh it needs to be. One report on a small list can stay on Power Query. When the list is one source in an existing Data Factory or Fabric estate, it belongs in that estate. A flow that reacts to one item changing should stay a flow. The question gets harder when a copy has become something people depend on every day.

For that copy, four checks separate the tools quickly. First, whether a lookup arrives as a key you can join or as a number someone has to resolve. Second, whether the copy keeps a record when an item is deleted or the row simply vanishes. Third, whether the read survives a list past 5,000 items without someone indexing columns to suit it. Last, who owns the schedule, the credentials and the alert when a run fails. Whichever tool passes all four without new code is the ETL tool for that list. The decision guide works through the Microsoft options with these same checks.

Lookups, deletes and large lists in the copy

Most copies that go wrong do so in one of these three places, and usually long after the copy was built. A lookup column that arrives as an id is harmless until a report needs the name behind it. Missing deletes stay invisible until the monthly totals stop matching SharePoint. Once a list crosses the threshold it can work for months, then a filtered read is refused on the day someone adds a view.

In an SPSync copy, each lookup is written as two columns, {col}_id and {col}_value, and each person as a key into one shared sp_users table, as the schema reference sets out. Generated reporting views leave deleted rows out, so a report sees current items, while an auditor can still find what was removed and when. What breaks past 5,000 items explains why reading changes by delta avoids the threshold.

Is there a SharePoint list sync tool?

Yes, several, and the word sync means different things on each vendor's page. Some sync one way, from the list into a database. Skyvia and Layer2 Cloud Connector also sync the other way, writing database changes back into the list, and the comparison pages for each describe how a conflict is settled. Fabric mirroring syncs one way, into OneLake only.

SPSync is a one-way sync tool: the list is where people edit, and the database is where reports, apps and AI tools read. If your plan needs edits made in the database to flow back into SharePoint, a two-way tool is the better fit, and two-way sync between SharePoint and Postgres sets out what works today and how write-back is designed. If the copy only needs to be current, the next step is to pick where it goes.

Every destination SPSync replicates into, by family

The same lists can go to more than one destination, with one replication for each. Pick the store whoever reads the data already uses. The destinations overview compares them in one table.

Relational databases

Warehouses and lakes

Search and vector stores

Every comparison

Each comparison takes one route, is fair about where it fits and shows how to move the job if you decide to. Two of them cover Microsoft tools that people try for this and that do not copy list items into a database, Microsoft Graph Data Connect and SharePoint search. The Stacksync page explains what its own documentation says about SharePoint. The comparison hub puts all of them in one table, and SharePoint to SQL Server tools compared narrows it to the tools that write to SQL Server or Azure SQL.