Script a SharePoint list migration into a database

8 September 2026

Tags: MigrationDesktopCommand line

Moving list data off SharePoint and into a database is rarely a single event. It happens once in a test environment, again in staging, then in production, and sometimes again for the next business unit or the next tenant. A migration you can only repeat by clicking through the same screens is one that drifts between environments, and nobody can say afterwards exactly what was chosen.

This post covers the scripted route with SPSync: a CLI, one YAML file per replication, and batch definitions when there are many sites. SPSync copies list items into database tables one way and never writes to SharePoint, so the live list stays untouched until you retire it. Moving whole sites between tenants is a different job, done by a tenant migration tool. SPSync’s part there is the before and after copies described in keeping a copy through tenant migrations.

Why does a migration wizard fall over on the second run?

A wizard is fine for one move. You pick a site, pick some lists, click through a few screens and the tool does its work once. The trouble starts when the move has to happen again. Each environment means repeating the same choices by hand, often months later and often by someone other than the person who did it first. The choices are not in version control, so a reviewer cannot see them before they run, and a difference between staging and production shows up only when someone spots a missing column.

SPSync puts the choices in a file instead, with the CLI as its primary interface.

Export the list migration config with the CLI

Sign in once, then let export-config discover the site’s lists and write the replication YAML:

spsync auth login --client-id <client-id>

spsync export-config \
  --output replication.yaml \
  --pipeline-name migration \
  --prefix migration \
  --destination-kind postgres \
  --connection-string-env SPSYNC_DB_CONNECTION \
  --dataset-name migration \
  --client-id <client-id> \
  --site-url https://contoso.sharepoint.com/sites/example \
  --all-lists

spsync doctor replication.yaml
spsync run replication.yaml

Swap --all-lists for one --list flag per list when only some are in scope. Swap --destination-kind postgres for mssql and the same commands target SQL Server.

What the replication YAML holds

The YAML file is the entire migration in one place. It names the site, the lists, the table prefix and the destination. The connection string and the licence key appear only as environment variable names, resolved when the run starts, so no secret is written into the file and it is safe to commit.

What the file does not need is a column map you maintain by hand. SPSync reads each list’s schema and creates the tables itself. Internal column names become destination column names, so a column displayed as “Due Date” with the internal name Due_x0020_Date arrives as due_date. A lookup becomes {col}_id and {col}_value, and a multi-value column gets its own child table with one row per value. If a list has date-only columns, set the site time zone with --site-time-zone Europe/London (any IANA name), or those dates can arrive a day out.

Check permissions and connections before the run

spsync doctor replication.yaml checks the sign-in, Graph access to each list and the destination connection before anything moves. Put it ahead of spsync run in a pipeline and a broken permission or connection string fails the build with a clear message at the start, long before a migration could stall halfway.

Review a SharePoint migration in a pull request

Commit the YAML to a repository and changes go through the same review as any other config. A teammate opening the pull request sees which sites, lists and destination the change touches before it runs anywhere. When the staging run has been checked, the production change is the same file with a different connection string variable behind it, so what was reviewed is what runs.

Many sites from one batch file

Repeat --site-url on export-config and one config covers several sites. For a larger estate, spsync batch create batch.yaml writes one replication config per entry in the batch file’s replications list. The desktop app’s bulk selection can export that batch file for you. Going from five sites to fifty is then an edit to one file, where a wizard would be repeated once per site.

Rerun it in staging, production and the next tenant

spsync run does a full load the first time and a delta on every run after, so a retried or rescheduled job never double-writes. That is what lets the same commands run in each environment in turn. Leave the schedule running while the new application is built and the database keeps up with people who are still editing the list.

On cut-over day, spsync reconcile replication.yaml re-checks every item against SharePoint. It is the QA step that lets you sign off the copy before the application switches over. Cutting over a SharePoint list to a database walks through that day step by step.

What a scripted list migration does not move

Be clear about the edges before the plan is signed off. Managed metadata columns are skipped today and not replicated, so export the terms another way. The files in a document library are not copied; only list item data is. SharePoint item permissions are not carried into the database, so access to the tables is whatever your database grants. Calculated columns arrive as their computed values, never as the formula.

The SharePoint list to database migration planner turns all of this into a checklist, from the list inventory to the reconcile. See scriptable migrations for the full pattern, Run a SharePoint list to database sync from CI to schedule it, or CLI recipes for more commands.