Connect SPSync to OpenSearch

SPSync writes one OpenSearch index per table: a list’s main table, each multi-value child table and the shared users table. Rows are indexed by key, and a row deleted in SharePoint is removed from the index rather than flagged. This guide covers a cluster you run yourself and the managed Amazon OpenSearch Service.

Before you start

An OpenSearch cluster you can reach over HTTP(S), and its URL, for example https://my-cluster.example.com:9200.
An administrator account on the cluster’s security plugin (or the domain’s master user on Amazon OpenSearch Service) to create a role and a user.
Nothing to install on the machine that runs SPSync. SPSync keeps a small local state file (a DuckDB file) that records what it last sent, so it sends only the rows that changed. Keep that file on a disk that survives restarts.

Create the database and a least-privilege user

With the security plugin on, give SPSync a role limited to the indexes it creates. This is from the OpenSearch security documentation and has not been run here against a test cluster. In Dashboards, Security, Roles, create a role, or call the API:

PUT _plugins/_security/api/roles/spsync_loader
{
  "cluster_permissions": ["cluster_composite_ops"],
  "index_permissions": [
    {
      "index_patterns": ["dev_*", "sp_users"],
      "allowed_actions": ["create_index", "crud", "indices:admin/mapping/put", "indices_monitor"]
    }
  ]
}

dev_* is the --prefix you give SPSync; use your own. crud covers indexing and deleting documents. indices:admin/mapping/put lets SPSync add a field when a SharePoint column is added. When a column changes type, OpenSearch cannot change the field in place, so SPSync starts a new index with _v2 on the end, which the same pattern covers.

Then create an internal user and map the role to it:

PUT _plugins/_security/api/internalusers/spsync_loader
{ "password": "use-a-long-random-password" }

PUT _plugins/_security/api/rolesmapping/spsync_loader
{ "users": ["spsync_loader"] }

On Amazon OpenSearch Service with fine-grained access control, do this in Dashboards as the master user. Mapping the role to an AWS IAM role instead (under Backend roles) is how a SigV4 caller is authorised; see below.

Allow SPSync to reach it

OpenSearch listens on TCP 9200 unless you changed it, and managed domains use 443. The connection starts from the machine that runs SPSync.

Self-hosted: allow the SPSync address in the host firewall and any load balancer.
Amazon OpenSearch Service, public access: in the domain’s access policy, allow the address, and keep fine-grained access control on.
Amazon OpenSearch Service, VPC access: a domain in a VPC has no public address. Run SPSync inside the VPC, or connect to it from where SPSync runs with a VPN or Direct Connect, and allow the SPSync address on the domain’s security group for port 443.
Aiven and other managed services: add the address to the service’s IP allow-list.

Encrypt the connection

Use an https:// URL. SPSync always verifies the server’s certificate, and there is no setting today to point it at a private certificate authority or to switch verification off. Amazon OpenSearch Service presents a certificate from a public authority, so it works. A self-hosted cluster that uses the demo certificates or a private CA fails until you install a certificate from a public authority, or add your CA to the operating system’s trust store on the machine that runs SPSync. SPSync does not rewrite the URL, so an http:// address sends the credential in clear text.

Authentication SPSync supports today

Three ways exist in the engine. Use exactly one.

API key: sent as an Authorization: ApiKey header.
Basic authentication: the user name and password from the previous section.
AWS SigV4 request signing: the usual choice for a managed domain. Set the AWS region (and the signing name, es by default or aoss for OpenSearch Serverless). It is available from the desktop app and server Destination screen, the command line and an AI assistant.

With SigV4, SPSync signs each request with the AWS credential on the machine: environment variables, a shared credentials file or an instance role. Give SPSync an explicit access key pair (and a session token for temporary keys) only if the machine has no AWS credential of its own, or you must override it. Entra ID and other single sign-on are not supported.

Connect from SPSync

Desktop app: open the Destination screen, choose OpenSearch, enter the OpenSearch URL and one sign-in: an API key, a username and password, or an AWS region for SigV4 (with the access key id and secret access key only if the machine has no AWS credential of its own), then choose Test connection. SPSync manages the local state file itself.

CLI: the config names environment variables, never the credential itself. SPSYNC_OS_STATE_PATH holds the path of the local state file, not a network address.

export SPSYNC_OS_STATE_PATH=/var/lib/spsync/opensearch-state.duckdb
export SPSYNC_OS_PASSWORD='use-a-long-random-password'
spsync export-config --output replication.yaml \
  --site-url https://contoso.sharepoint.com/sites/x --list Projects \
  --pipeline-name spsync_dev --prefix dev \
  --destination-kind opensearch --connection-string-env SPSYNC_OS_STATE_PATH \
  --dataset-name main \
  --opensearch-url https://my-cluster.example.com:9200 \
  --opensearch-username spsync_loader --opensearch-password-env SPSYNC_OS_PASSWORD \
  --client-id <entra-app-client-id>

For an API key use --opensearch-api-key-env SPSYNC_OS_API_KEY instead of the user flags. For SigV4 use --opensearch-aws-region us-east-1 (and --opensearch-aws-service aoss for Serverless) and no other credential flag.

AI assistant: with the admin MCP server connected, ask for the destination requirements, then let the assistant create the destination with the URL. For SigV4 it also sets the non-secret region and signing name. You enter the API key, password or AWS keys on the secure entry page, never in the chat; leave the keys out to sign with the machine’s own AWS credential.

What Test connection and doctor check

Both ask the cluster to respond to a ping. A pass means the URL is reachable, the certificate is trusted and the cluster answered. It does not prove the credential can create or write to an index; the first sync is the real test of the role. spsync doctor reports the probe as destination-connectivity.

Common errors

What you see	What it means and the fix
`Connection refused` or a timeout	A firewall, security group or access policy is blocking the port, or the domain is in a VPC SPSync cannot reach.
`SSL: CERTIFICATE_VERIFY_FAILED`	The certificate is a demo or private-CA certificate. Use one from a public authority, or trust your CA on the machine that runs SPSync.
`401 Unauthorized`	The user name, password or API key is wrong.
`403 ... no permissions for [indices:...]`	The role does not cover the index or the action. Widen `index_patterns` to match your prefix and check the role mapping.
`403 ... User: arn:aws:... is not authorized` (SigV4)	The AWS identity SPSync signed with is not allowed by the domain’s access policy, or is not mapped to a role in fine-grained access control.
`could not ping OpenSearch cluster`	The cluster answered but is not healthy, or a proxy is in the way. Check from the same machine with `curl`.