Automation and the command line
Running SPSync unattended from the CLI: scheduled syncs, CI pipelines and server licensing.
Most replications start in the desktop app on someone's laptop. Sooner or later the team wants them on a server instead, run by a scheduler, with the config in version control and an alert when a run needs attention.
The SPSync command line is built for that. spsync export-config writes the replication as a plain YAML file naming the sites, lists and destination, and spsync run replication.yaml does one sync round, a full load the first time and a delta after that. It never loops or sleeps. Cron, launchd, Windows Task Scheduler or a CI pipeline decides when it runs, and a retried job is safe because a second run only reads what changed. For many sites, spsync batch create takes one batch file and writes a replication config per site, so going from five sites to fifty is an edit to one file.
Secrets stay out of the file. The YAML names an environment variable for each connection string and for the licence key, and the scheduler's own secret store supplies the value, so the config is safe to commit and diff. The exit code tells the scheduler how the run went. A run that completed with warnings, such as a column it skipped, exits with 2, and one refused because another run already holds the destination exits with 4. Add --json and each list's read, inserted, updated and soft-deleted counts come back in a shape a script can parse.
A few practical points before you plan the server. The command line is a separate install from the desktop app, set up with you on a call, on a machine you provide, such as a self-hosted build agent. Only one run per replication and destination goes ahead at a time on a machine, and two machines pointed at the same destination are not coordinated yet, so give each replication one home. Licences are per machine and per destination family, and the licensing page explains how unattended use is counted.
Start here
- What it costs to keep a SharePoint export script running
Counting what a home-made export script or flow costs once it runs on a schedule is the place to begin.
- Run a SharePoint list to database sync from CI
Scheduling spsync run in CI, cron, Task Scheduler or launchd is the practical step after that, with secrets kept out of the YAML.
- Script a SharePoint list migration into a database
A whole migration as config and commands you can run again in the next environment is the last step up.
All posts on this topic
- Export a SharePoint list to SQL Server with PnP PowerShell
A PnP PowerShell script that copies a SharePoint Online list into a SQL Server table, with app-only sign-in, and the places it breaks once it runs every night.
- Power Automate Get items past 5,000, 100,000 and beyond
Why Get items stops at 100, 5,000 or 100,000 items, what each cap belongs to, and what a flow copying a list to SQL Server misses on deletes.
- Python: a SharePoint list into Postgres or SQL Server
A tested Python script that copies a SharePoint list into Postgres or SQL Server with Microsoft Graph delta: paging, deltaLink storage, Retry-After and deletes.
- What it costs to keep a SharePoint export script running
Schema drift, deletes, lookups, throttling and monitoring: the five jobs a SharePoint-to-SQL Server script or flow takes on once it runs on a schedule.
- Run a SharePoint list to database sync from CI
Schedule spsync run to copy SharePoint lists into a database from GitHub Actions, Azure DevOps, cron, Windows Task Scheduler or launchd, with secrets kept out of the YAML.
Other topics on the blog
Frequently asked questions
Can the migration config be reviewed before it runs?
Yes. spsync export-config writes a plain YAML file naming the sites, lists and destination; commit it and open a pull request like any other config change. Connection strings and the licence key are environment-variable references, so the file holds no secrets and is safe to diff.
Does a re-run duplicate data?
It does not. The first spsync run against a config is a full load, every later run is a delta keyed on Graph's change tokens, and a reconcile, weekly by default, checks the two sides match. A retried CI job is safe to run again.
Where do secrets live for a scheduled spsync run?
In the scheduler's own secret store (GitHub Actions secrets, an Azure DevOps secret variable group, or a machine environment variable on Windows or macOS). The replication YAML only names the environment variable (connection_string_env), so the value itself is never written into the file.