Fix SPSync permission, throttling and connection errors

Start with spsync doctor replication.yaml. It runs five checks, in order: the ODBC driver (SQL Server destinations only), Graph token validity, site permission grants, the current adaptive sync interval, and destination connectivity. The sections below are ordered so the earlier ones are more likely to be the actual cause.

Permission errors

“401 Unauthorized” or auth login loops back to the sign-in prompt. Your token expired or was revoked. Delegated tokens stop refreshing after about 90 days of inactivity, or immediately if the person removed the consent or left the tenant. Run spsync auth login again. If this keeps happening for an unattended replication, switch to application permissions, because delegated auth is not meant to run unattended long-term.

“403 Forbidden” reading a site’s lists. doctor’s graph-permissions check reads a replication’s whole site at once, so a 403 here fails that one check, and doctor exits 1; it is not reported per list. It usually means item-level permissions or a tenant admin policy blocks the account or app being used for this site, not a bug. Ask a site owner to check the site’s permissions, or ask a tenant admin to check Conditional Access / app-enforced restrictions.

“Need admin approval” when you try to consent yourself. Your tenant requires admin consent for new apps. Use the admin-consent URL from SPSync’s web UI and send it to a tenant admin. See Admin permissions.

Application-permission grant fails with “insufficient privileges”. The signed-in admin needs Sites.FullControl.All, or Sites.Selected with Full Control/Owner on the specific site being granted. If they do not have that, SPSync prints the exact PnP PowerShell command for someone who does to run instead.

Throttling (429 / 503)

SharePoint Online enforces a request budget per app per tenant and does not return the usual RateLimit-* headers, so SPSync meters its own use and backs off before hitting Microsoft’s limit rather than after. If you still see it:

• SPSync honours Retry-After automatically, so a 429 or 503 in the logs is expected occasionally and not itself a problem as long as the run finishes.
• Repeated throttling across a whole tenant, on every replication, usually means more than one SPSync-connected app registration is active against the same tenant (each one draws from the same shared budget). Check you are not running the CLI, desktop app, and the Azure managed application against the same site concurrently with different credentials.
• Run full loads and reconciles at quiet hours for the tenant’s region, by scheduling the command-line tool from your own scheduler, if the tenant is large and near its budget already.

Entra sign-in and consent errors

When SPSync cannot get a token, the error carries Microsoft’s own code. These are the codes SPSync recognises, what each means and where to fix it. Anything else is shown as Microsoft sent it; the code is searchable on Microsoft Learn.

Code	What it means	Fix
invalid_grant, interaction_required	The saved sign-in was revoked, expired or needs a fresh prompt. SPSync treats both as “sign in again”, not as a transient failure	Run spsync auth login again; for an unattended replication use application permissions
AADSTS65001	Consent is missing: the user or an admin has not approved the app’s permissions	A tenant admin approves the app, see Admin permissions
AADSTS90094	Admin consent is required for this application in this tenant	Send the admin-consent URL to a tenant admin, see Admin permissions
AADSTS650053, AADSTS70011, AADSTS65005, invalid_scope	The requested permission is not available to a user on their own, or the app’s registration does not list it	An admin approves the permission, or fix the app registration’s API permissions, see Admin permissions
AADSTS650056, AADSTS7000112, AADSTS7000114	The application is misconfigured, disabled or not consented for this tenant (SPSync reads these as “admin consent not granted”)	Re-run the admin consent step in Admin permissions
AADSTS7000215	The client secret is wrong. Usually the secret’s ID was pasted instead of its Value, which is only shown once	Create a new secret and enter its Value
AADSTS7000222	The client secret has expired	Create a new secret under Certificates & secrets and enter its Value
AADSTS700016	No application with this client id exists in the tenant	Check the client id, and that the tenant id is the tenant the app was registered in
AADSTS90002	The tenant was not found	Check the tenant id or domain on the connection
AADSTS700027	The certificate does not match the one on the app registration, or was not uploaded	Upload the certificate’s public part under Certificates & secrets
AADSTS700024	The signed assertion is outside its valid time range (usually clock skew)	Check the clock on the SPSync machine
AADSTS50034	The account does not exist in the tenant	Sign in with an account from the tenant

Sign-in needed

A run marked Sign-in needed stopped because Microsoft Entra asked the connection’s account to sign in again. This is usually a Conditional Access policy (a new location, an unmanaged device, a required multifactor prompt), a revoked session after a password change, or a Continuous Access Evaluation event, where Microsoft ends a token early. The run shows Entra’s own message so you can see which policy applied.

SPSync already retried once before stopping: it fetched a fresh token and, when Microsoft sent a claims challenge, answered it. The run stops only when that is not enough.

1. Choose Sign in again on the run, or run spsync auth login, and complete the prompt Entra shows (often MFA or a compliant-device check).
2. Run the replication again. It resumes from where it stopped; nothing is re-copied.
3. If it keeps happening for an unattended replication, a person’s sign-in is the wrong identity for it. Use application permissions, which a user-based policy does not apply to, or ask your Entra admin to exclude the service account from the policy.

What Test connection and doctor check

spsync doctor (its destination-connectivity check) and the Test connection button in the app run the same kind of probe, with one difference: the app has no doctor run of its own, so it builds the probe from the form’s fields. Both stop at the first failure. A field left blank is named before any connection is attempted (elasticsearch_url is required, databricks_catalog is required, bucket_url is required, connection string is empty). Neither check writes a table or a row.

Kind	Test connection and doctor open	Does not prove
sqlite	The database file and run SELECT 1	Free disk space
duckdb	The local file or the MotherDuck database and run SELECT 1; with a Parquet export bucket, doctor also lists that bucket	Permission to write to the export bucket
postgres	A connection (and, in the app, SELECT 1)	CREATE privilege on the schema
mssql	A connection through the ODBC driver; doctor also runs an earlier odbc-driver check that finds the driver	CREATE TABLE rights
mysql	A connection (and, in the app, SELECT 1)	CREATE rights; TLS verification unless ssl_ca is set
oracle	A connection and SELECT 1 FROM dual	The second schema’s privileges, see Oracle
clickhouse	An HTTP connection and SELECT 1	The native port, which loading uses
bigquery	The service-account key, a SELECT 1 query and, in doctor, the dataset in the configured location	The BigQuery Data Editor role
databricks	A session on the SQL warehouse, SELECT 1 and SHOW SCHEMAS IN <catalog>	CREATE VOLUME and table privileges
dataverse	A token for the environment and a WhoAmI call	The publisher prefix, table-creation rights
azure-ai-search	A list-indexes call with the admin key	Index-creation quota
adls-gen2	A listing of the container and prefix	Write permission on the path
deltalake	A listing of the bucket or container	Write permission on the path
elasticsearch	A cluster ping	Index-creation rights
opensearch	A cluster ping	Index-creation rights
iceberg	A call to the REST catalog	Write access to the warehouse
lancedb	The dataset location (local path or object store)	Write access
qdrant	A call to the cluster	Collection-creation rights
weaviate	A readiness call to the cluster	Collection-creation rights
pgvector	A Postgres connection, the server version (PostgreSQL 15 or newer) and that the vector extension exists or can be created	The embedding provider

Where the app and doctor differ: for the search and vector kinds the app tests only what the form can collect (an API key or basic auth; Iceberg without a token, LanceDB without storage options), so a destination that needs more can pass in doctor and fail in the app, or the other way round.

Destination connection errors

If the destination check fails, first test the connection string’s host, port and credentials with a tool outside SPSync (psql, sqlcmd, mysql, curl). Most failures are network, firewall or credential problems. Driver and server messages below are quoted as the driver words them, which can change between versions; SPSync’s own messages are exact. The app removes passwords and keys from the messages it shows. The setup page for each destination is linked from its heading.

SQL Server and Azure SQL

Setup: SQL Server, Azure SQL.

• no 'ODBC Driver 17/18 for SQL Server' found (see ADR-007) (doctor’s odbc-driver check), or in the app the ODBC driver for SQL Server is not installed on this host. Install Microsoft’s ODBC Driver 18 for SQL Server on the machine that runs SPSync, including a machine running the desktop app.
• A handshake or certificate error mentioning Encrypt, SSL or certificate verify failed. ODBC Driver 18 encrypts by default and verifies the server certificate. Azure SQL has a public certificate and needs Encrypt=yes and nothing else. A server with a self-signed certificate needs either its CA installed on the machine or, for a test system only, TrustServerCertificate=yes in the connection string.
• Login failed. The login needs rights to create tables and indexes in the target database, because the first run creates the schema. On Azure SQL, also allow the machine’s IP address in the server firewall.

PostgreSQL, pgvector

Setup: PostgreSQL, pgvector.

• A certificate or sslmode error such as SSL error: certificate verify failed or server does not support SSL. Match sslmode to the server. require encrypts without checking the certificate; verify-ca and verify-full need the server’s CA certificate and fail without it. Managed services (Supabase, Neon, Azure, RDS) document which they need.
• pgvector destination needs PostgreSQL 15 or newer ... SPSync’s own message from the pgvector check: the server is older than 15. Plain postgres has no such limit.
• A permission error creating the vector extension. Ask a database admin to run CREATE EXTENSION vector once; the check then passes.

MySQL and MariaDB

Setup: MySQL, MariaDB.

• TypeError: Connection.__init__() got an unexpected keyword argument 'ssl_mode'. The driver (pymysql) has no ssl_mode option. To require and verify TLS, add ssl_ca=/path/to/ca.pem to the connection string.
• TLS verification looks switched off after adding ssl_verify_identity=true. Do not add it next to ssl_ca: it replaces the CA setting and turns verification off. ssl_ca on its own verifies the certificate and the host name.
• Access denied for a user@servername login. Older single-server Azure instances need that form; Flexible Server accepts the plain user name.

Oracle

Setup: Oracle.

• A connection error naming the service or listener. The connection string selects the database with ?service_name=, not a path segment. Use the pluggable database’s service name (FREEPDB1 on current free images).
• ORA-00942: table or view does not exist on the first run. On Oracle a schema is a user account. SPSync writes to the user in the connection string; a second schema (for reporting views) must exist and the user needs rights on it. The connection test cannot see this because it only runs SELECT 1 FROM dual.

ClickHouse

Setup: ClickHouse.

• A TLS or timeout error against a self-hosted server. secure=1 turns TLS on, so a server without TLS does not answer; use secure=0 for a plain test server. The check connects over HTTP, so add http_port (for example &http_port=8123) to the connection string if the server does not use ClickHouse Cloud’s default. Loading uses the native port in the host part of the string, so both ports must be reachable.

Elasticsearch and OpenSearch

Setup: Elasticsearch, OpenSearch.

• could not ping Elasticsearch cluster at <url>. SPSync’s own message when the cluster did not answer a ping. Check the URL and port, and that an API key or user and password are set.
• A certificate verification failure (CERTIFICATE_VERIFY_FAILED). The cluster’s certificate is not signed by a CA this machine trusts, typical of a self-managed cluster with its own CA. Install that CA on the machine rather than disabling verification.

BigQuery

Setup: BigQuery.

• A key error such as invalid_grant or a JSON parse error. The connection value is the whole service-account JSON key, not a file path. A key that was deleted or rotated fails here.
• Not found: Dataset ... or a location error. doctor looks up the dataset in the configured bigquery_location. A dataset in EU or europe-west2 with the location left unset is searched for in US and reported missing. Set the location to the dataset’s.
• Access Denied on the query. The service account needs the BigQuery Job User role as well as data access to the dataset.

Databricks

Setup: Databricks.

• A connection error that names the host or HTTP path. The connection string is databricks://token:<PAT>@<host>/<http-path>; the HTTP path is the SQL warehouse’s, from its Connection details tab.
• databricks_catalog is required. SPSync’s own message: the check tests the catalog too, so it must be set.
• A failure on SHOW SCHEMAS IN <catalog>. The catalog name is wrong or the principal lacks USE CATALOG. A stopped warehouse is started by the first statement, so the first check after a long idle can be slow.

Dataverse

Setup: Dataverse.

• A failure acquiring a token. See Entra errors: AADSTS7000215 (wrong secret), AADSTS7000222 (expired secret), AADSTS700016 (wrong client id) and AADSTS90002 (wrong tenant) are the usual ones. The app also needs an application user in the Dataverse environment.
• destination.dataverse_publisher_prefix must be 2-8 lowercase alphanumeric .... SPSync’s own validation: the prefix is 2 to 8 lowercase letters or digits, sps by default.
• DataverseThrottledError. Dataverse answered 429 or 503 repeatedly and SPSync gave up after its retries, honouring Retry-After each time. The next round continues; run large loads at quiet hours.

Other kinds

• ADLS Gen2, Delta Lake. The check lists the container or bucket, so a wrong account name, key, SAS token or container fails there. The credential needs list permission.
• Azure AI Search: could not list indexes at <endpoint>. SPSync’s own message. The check needs the service’s admin key and a reachable endpoint.
• Weaviate: Weaviate cluster at <url> is not ready. SPSync’s own message: the cluster answered but reports it is not ready, or could not be reached. Weaviate also needs its gRPC port reachable (50051 unless you set another).
• Iceberg, LanceDB, Qdrant, SQLite, DuckDB. The message names the URL, path or file that failed; check it from the same machine.

“Another run holds this destination”

SPSync runs one round per replication and destination at a time. This message (CLI exit code 4; on the Runs screen, a skipped run with a destination hint) means another process, such as a terminal spsync run or the background service, was already syncing the same replication to the same destination, so this round was skipped and nothing was written. It clears when the other run finishes, and a scheduled replication runs again on its next interval. If it persists, check for a stuck spsync run or a second copy of the service and stop it; the lock is released automatically when that process exits. This protects one machine only: do not point two machines at the same destination.

Repointing a replication at a new destination

You can change an existing replication’s destination without creating a new one: the next round after the change re-runs each list’s initial full load against a new destination (one that has never held this replication’s data before) rather than resuming a delta round. Delta state (delta links, known etags, last-reconciled timestamps) is scoped to the replication and destination together, not the replication alone, precisely so a new, empty destination is never skipped without trace. If you see an early round on a newly repointed destination come back empty, check the run’s own log for a pipeline.local_state_destination_mismatch event. It names the replication and both the old and new destination, and confirms SPSync itself discarded stale local state and ran a full load, which explains the destination’s rows.

That “full load” outcome is specific to a destination SPSync’s local working directory has never seen: dlt itself restores pipeline state from the destination before touching data (Pipeline._sync_destination), so repointing back to a destination that still carries its own prior _dlt_pipeline_state (nothing there was dropped or reset) correctly resumes as a delta round, with no re-read of rows already present, because SPSync’s local state directory is a cache of that destination-side state, not the only copy of it.

Schema evolution surprises

doctor does not have a schema-drift check. Schema changes show up as a difference in the destination tables themselves the next time run or reconcile re-discovers a list’s current columns from Graph. Two things you may see there are expected behaviour, not errors:

• A retyped column (e.g. Number changed to Currency in SharePoint) creates a new {col}_v2 column with the new type on SQL Server, PostgreSQL, MySQL (and MariaDB), ClickHouse, Oracle, SQLite and DuckDB; the original {col} is left as-is with its old values. Other destinations do not fork a retyped column today, so this does not apply to them. This is intentional (see the schema reference), so update anything reading {col} to read {col}_v2 instead when you are ready.
• A removed SharePoint column stays in the destination table; it is never dropped. SPSync does not delete data your reports might depend on just because the source column went away.

If a column’s values look wrong rather than its shape (e.g. a lookup showing the wrong list), check the schema reference for how that column type is supposed to map. Most “wrong value” reports turn out to be a mismatch between the display name shown in SharePoint’s UI and the internal name SPSync’s naming rules use.

Where the logs are

The desktop app and the spsync CLI write the same log files: one file per day, spsync-YYYY-MM-DD.log, kept for 14 days (a file that grows past 20 MB in one day rolls over to spsync-YYYY-MM-DD.1.log, and so on). Every line is one JSON object, so the files are easy to grep, filter with jq, or send to us.

Platform	Folder
macOS	~/Library/Logs/SPSync/
Windows	%LOCALAPPDATA%\SPSync\logs\
Linux	~/.local/state/spsync/logs/ ($XDG_STATE_HOME/spsync/logs/ if that is set)

SPSYNC_LOG_DIR overrides the folder on every platform. In the desktop app, Settings -> Logs shows the folder and how much disk it uses, and Open logs folder opens it in Finder or Explorer. From a terminal:

spsync logs path            # print the folder
spsync logs path --json     # folder, current file, level, disk usage, every file kept
spsync logs tail            # last 50 lines of today's file, one event per line
spsync logs tail -n 200 -f  # keep following as new lines arrive
spsync logs tail --raw      # the JSON lines as-is, for jq

Every record carries a timestamp, level, logger, a stable event name, and whatever context applies: run_id and replication_id (or replication for a CLI run), the list_id/list_name and site being read, and Microsoft’s request_id/ client_request_id on every Graph call, which match Microsoft’s own service logs if you ever open a support case with them. The events you will see most:

Event	Meaning
run.started, run.succeeded, run.failed	One sync round; run.failed carries the error, its category and a hint
list.full_load.started, list.round.started, list.round.collected	Per list: what mode the round ran in (full_load, delta, reconcile) and how many items and removals it read
list.delta.page	Each page of a delta round (debug level)
list.delta.resync	Graph asked for a fresh delta token (a 410); the list reloads in full
list.delta.deleted	The list no longer exists in SharePoint (a 404 on its delta call); this round is abandoned. The list’s rows are then marked soft-deleted in the destination and its selection is disabled, and it is not synced again until you re-select it
list.column.skipped	One or more columns of a list were not replicated because SPSync cannot map them (an unsupported or unrecognised column type). Fires once per list, naming up to 10 columns (columns, count), and again only when a column newly becomes unmappable or its reason changes. facet_keys is the names (never values) of the column properties Graph sent that SPSync does not model: send this to support. The same columns appear on the Runs screen and in spsync run output
list.column.skipped_by_design	info, with a count only: the system and hidden columns every SharePoint list leaves out on purpose (ID, ContentType, GUID, …). No action needed
graph.throttled	Graph answered 429/503; SPSync waits graph_retry_after_seconds and retries
graph.request	Every Graph call (debug level): method, URL, status, resource-unit cost, support ids. Metadata only, never list contents
destination.write	One dlt load into the destination finished

Log level. info (the default) records each run, each list’s round, and each write. debug adds graph.request and list.delta.page, which is what we will ask for when chasing a specific Graph problem. warning and error keep only throttling and failures. Change it in Settings -> Logs (saved for next time), with spsync --log-level debug run ..., or with SPSYNC_LOG_LEVEL=debug in the environment.

Secrets never appear in the logs. Bearer tokens, refresh tokens, client secrets, connection-string passwords, SAS signatures and OAuth codes are replaced with [REDACTED] before a line is written, so a log file is safe to attach to a support request as-is.

Still stuck

Run spsync doctor replication.yaml --json for machine-readable output, then look at today’s log file (spsync logs tail, or the folder above): the request_id and client_request_id fields on a graph.throttled/graph_non_2xx line match Microsoft’s own service logs if you need to open a support case with them for a specific request.

If none of the above gets you there, export a diagnostics bundle (Settings -> Send diagnostics then Export in the app, Report this run on a failed run’s row, or spsync diagnostics export) and attach it to your email to us; see Getting support for what it contains.