SharePoint list to Azure Synapse Analytics

Three routes from a SharePoint list into Azure Synapse: the SharePoint connector in a Synapse pipeline, SPSync Parquet on ADLS Gen2 for serverless SQL, or an Azure SQL copy.

Book a call

Talk through your use case, see further walkthroughs or demos, and set up a trial in your own environment.

Last updated

Status: Coming soon · product: SPSync for Warehouses & Lakes . We prioritise destinations by demand. Book a call to tell us you need it.

SPSync copies SharePoint lists one way into files and databases that Azure Synapse can read, and SharePoint stays the system of record; SPSync has no Synapse destination yet. Synapse pipelines can also copy a list themselves with Microsoft’s SharePoint Online List connector. This page compares the three ways in and what each does to your columns.

How do I copy a SharePoint list into Azure Synapse?

Synapse pipeline with the SharePoint connectorSPSync Parquet on ADLS Gen2, read by serverless SQLSPSync to Azure SQL
Who runs itYou build and schedule the pipelineSPSync, on its scheduleSPSync, on its schedule
Lookup and person columnsItem ID only (Int32){col}_id and {col}_value; people as {col}_user_id joined to sp_usersThe same as Parquet
Multi-choice columnsOne choice data type per source, per LearnA child table, one row per valueA child table, one row per value
Deleted itemsNot described on LearnKept with sp_deleted_at setKept with sp_deleted_at set
Where Synapse reads itWherever the pipeline’s sink writesOPENROWSET over the files, no loadA pipeline copy, or Power BI directly

What the SharePoint connector does with each column type

Microsoft’s SharePoint Online List connector page covers Azure Data Factory and Synapse pipelines together. Its type map turns a Lookup column into Int32, and a Person or Group column into Int32 as well, so the copy carries the SharePoint item or user ID and no name. Managed Metadata, Attachment, Task Outcome and External Data columns are listed as unsupported, and the page notes that you cannot select more than one choice data type for a SharePoint Online List source. The page does not describe incremental reads, so a pipeline reads whatever rows its OData query selects on each run.

SPSync differs on lookups, people and multi-choice columns. A lookup arrives as {col}_id and {col}_value, a person as {col}_user_id joined to an sp_users table with the name and email, and a multi-choice column as a child table. It shares two of the connector’s gaps: managed metadata columns are skipped and reported on each run, and attachments are not copied. The schema reference lists every column type. If you already run a Synapse pipeline and only need text, number and date columns, the connector may be all you need; the comparison with Azure Data Factory goes further.

Reading SPSync’s Parquet from a serverless SQL pool

Replicate the lists to the shipped Azure Data Lake Storage Gen2 destination. Each table becomes a folder in your container with a current.parquet snapshot replaced on every run, plus one history file per run. A serverless SQL pool reads the snapshot in place with OPENROWSET:

SELECT *
FROM OPENROWSET(
BULK 'https://<account>.dfs.core.windows.net/<container>/<prefix>/it_tickets/current.parquet',
FORMAT = 'PARQUET'
) AS tickets;

Wrap that in a view and Power BI or any SQL client sees a table. The same pattern works for each child table and for sp_users, so a second view can join tickets to the people assigned to them. The run history files let a query go back to an earlier run.

Trade-offs of the serverless route

A SQL replica in Azure SQL

SPSync’s Azure SQL destination keeps each list as a table in Azure SQL Database, updated from SharePoint’s delta query on every run, with upserts and soft deletes. Power BI and applications read it directly, and a Synapse pipeline can copy from it into a dedicated pool if your warehouse needs the data there. The same Synapse documentation now carries a tip that anyone new to data warehousing should start with Fabric Data Warehouse, so if you are choosing a platform today, the Fabric routes are worth reading too.

What a native Synapse destination would add

Writing straight into a dedicated SQL pool would give you tables with SPSync’s usual upserts and soft deletes, without a serverless view layer on top. Before it can be built, the destination has to settle whether it targets a dedicated pool, a serverless pool or both, how tables are distributed and indexed, and how loads are staged and signed in. Delta sync, the schema mapping and soft deletes come from SPSync’s engine, so a Synapse destination would inherit them.

Tell us which pool type you run, roughly how many lists and rows you would sync, and how current the data needs to be. That decides the order of the roadmap, and we will contact you when Synapse ships. Like every destination, it would arrive in the desktop app, SPSync in your Azure and the command-line tool at once.

Frequently asked questions

Does SPSync have a Synapse destination?

No. A Synapse destination is on the roadmap. Today SPSync writes Parquet to ADLS Gen2, which a serverless SQL pool reads in place, and to Azure SQL Database, which a Synapse pipeline or Power BI can read.

How does the Synapse SharePoint connector handle lookup columns?

Microsoft's connector page maps a Lookup column, and a Person or Group column, to Int32, so the pipeline copies the item ID and leaves you to resolve the name. SPSync writes the ID and the display value side by side.

Would a native destination use a dedicated or a serverless pool?

Not decided. A dedicated SQL pool is a table store SPSync could write to; a serverless pool reads files. Which one, or both, depends on what customers run.

Should a new SharePoint reporting project start on Synapse or Fabric?

Microsoft's own Synapse documentation now suggests Fabric Data Warehouse for anyone new to data warehousing. If you are choosing today, read the Fabric routes as well.

Talk to us about Azure Synapse Analytics

SPSync is licensed per installation and per target family, with Standard Support included. Tell us which targets you need and how many machines, and we will size it with you. See how licensing works.

Script it

Everything you can click, you can script. The CLI, YAML config and automation below reproduce this scenario exactly. See the CLI reference.

# This destination has no --destination-kind in the command line today.
# In the meantime, run this from a shipped destination (Postgres or SQL Server)
# and repoint --connection-string-env at your database.
# YAML for this destination's kind ships with its --destination-kind support.

Cron (this destination nightly):

# every night at 02:00
0 2 * * * cd /opt/spsync && SPSYNC_DB_CONNECTION="$(cat /etc/spsync/db.env)" uv run spsync run replication.yaml >> /var/log/spsync/run.log 2>&1

GitHub Actions:

# .github/workflows/spsync-run.yml
name: SPSync
on:
  schedule:
    - cron: "0 2 * * *"
  workflow_dispatch: {}
jobs:
  run:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v3
      - run: uv sync
      - run: uv run spsync run replication.yaml
        env:
          SPSYNC_DB_CONNECTION: ${{ secrets.SPSYNC_DB_CONNECTION }}
          SPSYNC_LICENCE_KEY: ${{ secrets.SPSYNC_LICENCE_KEY }}

Azure DevOps:

# azure-pipelines.yml
schedules:
  - cron: "0 2 * * *"
    branches: { include: [main] }
    always: true
steps:
  - script: uv sync
  - script: uv run spsync run replication.yaml
    env:
      SPSYNC_DB_CONNECTION: $(SPSYNC_DB_CONNECTION)
      SPSYNC_LICENCE_KEY: $(SPSYNC_LICENCE_KEY)

Windows (PowerShell + Task Scheduler):

# Windows Task Scheduler action (see /docs/cli/automation/)
$env:SPSYNC_DB_CONNECTION = (Get-Content C:\spsync\db.env)
uv run spsync run C:\spsync\replication.yaml
schtasks /Create /SC DAILY /ST 02:00 /TN "SPSync" /TR "powershell -File C:\spsync\run.ps1"

macOS (launchd):

# ~/Library/LaunchAgents/io.spsync.run.plist, then: launchctl load ~/Library/LaunchAgents/io.spsync.run.plist

Secrets (licence key, connection strings) come from the environment, so never write them into the YAML. See scheduling and secrets.

Book a call

Tell us about your lists and where they need to go: we will talk through your use case, show further walkthroughs or demos, and set up a trial in your own environment.