SharePoint to OpenSearch

Last updated 26 October 2026

One way: SharePoint stays the system of record, and SPSync keeps a read-only copy of your lists current in OpenSearch.

Status: Shipped · product: SPSync for Search & Vector Stores

SPSync writes SharePoint list items one way into OpenSearch indexes for keyword and filtered search; the documents carry no knn_vector field. For k-NN or hybrid queries, run your own step that embeds the text and stores the vectors in an index of yours. Vectors on sync for OpenSearch are only under consideration, with no plan or date; the destinations that get them from SPSync today are Postgres with pgvector and Azure AI Search.

On AWS, search often means Amazon OpenSearch Service, with IAM deciding who can reach the domain. SPSync signs its requests the way the domain expects, so the indexes follow the same access rules as the rest of your AWS estate. A self-managed OpenSearch cluster works too.

Three sign-in options

Use one of these, never more than one:

1. AWS SigV4 request signing. Set the region, and the service name: es for a managed domain, aoss for OpenSearch Serverless. Credentials come from an explicit access key pair in environment variables, optionally with a session token, or from the AWS credential chain on the host: environment variables, a shared credentials file, or an instance or task role.
2. HTTP basic auth, a username and password, the usual choice with the Security plugin on a self-managed cluster.
3. An API key, sent as an Authorization: ApiKey header.

You can set any of the three in the desktop app or server Destination screen, in the command-line tool’s YAML config, or through an AI assistant using the admin MCP server. The assistant sets the region and signing name; you type the keys, password or API key on a secure page, never in the chat. Leave the AWS keys empty and SPSync signs with the credential chain on the machine it runs on.

The URL is the domain or cluster endpoint, such as https://search-my-domain.us-east-1.es.amazonaws.com. Domains inside a VPC also need a network path from the sync host. SPSync keeps a local state file between runs, so give it a stable path.

Indexes and documents

Each SPSync table becomes an index: each list, each multi-value column and the shared sp_users. A policy register, gov_policies, with a multi-choice Applies to column gives gov_policies and gov_policies__applies_to. Documents carry lookup IDs and display values on the row, and person columns as user IDs that join to sp_users.

IDs are mapped as exact-match keywords, text and display values as analysed text, and numbers, dates and Yes/No fields with their own types for filters and aggregations. A SharePoint column that changes type sends future writes to a fresh _v2 index, since OpenSearch cannot change a mapped field’s type.

Deleted SharePoint items are deleted from the index on the next run. Each run sends only changed documents; a reconcile, weekly by default, re-sends all of them.

Uses

A search experience over SharePoint lists on AWS, a keyword tool for AI agents running beside the rest of the stack, and keyword retrieval for chat over SharePoint data. For vector or hybrid retrieval, see RAG on SharePoint lists.

Steps

1. Sign in to SharePoint with delegated access, or have an admin grant Sites.Selected.
2. Pick a sign-in option and grant the identity rights to create indexes and write documents; on a domain with fine-grained access control, map it to a suitable role.
3. Set the endpoint and credentials, choose the lists and run.
4. Point your search UI or agent at the indexes.

Index and shard sizing drive domain cost as lists grow. SharePoint permissions are not carried across, so the domain’s own access policy is the boundary.

spsync auth login --client-id 00000000-0000-0000-0000-000000000000
spsync export-config --output replication.yaml --pipeline-name opensearch --prefix opensearch --destination-kind opensearch --connection-string-env SPSYNC_DB_CONNECTION --dataset-name opensearch --client-id 00000000-0000-0000-0000-000000000000 --site-url https://contoso.sharepoint.com/sites/example --all-lists
spsync doctor replication.yaml
spsync run replication.yaml

pipeline_name: opensearch
prefix: opensearch
auth:
  mode: delegated
  client_id: 00000000-0000-0000-0000-000000000000
  tenant_id: common
destination:
  kind: opensearch
  connection_string_env: SPSYNC_DB_CONNECTION
  dataset_name: opensearch
scopes:
  - site_url: https://contoso.sharepoint.com/sites/example
    mode: entire_site
reconcile_interval_days: 7
create_reporting_views: false

# every night at 02:00
0 2 * * * cd /opt/spsync && SPSYNC_DB_CONNECTION="$(cat /etc/spsync/db.env)" uv run spsync run replication.yaml >> /var/log/spsync/run.log 2>&1

# .github/workflows/spsync-run.yml
name: SPSync
on:
  schedule:
    - cron: "0 2 * * *"
  workflow_dispatch: {}
jobs:
  run:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v3
      - run: uv sync
      - run: uv run spsync run replication.yaml
        env:
          SPSYNC_DB_CONNECTION: ${{ secrets.SPSYNC_DB_CONNECTION }}
          SPSYNC_LICENCE_KEY: ${{ secrets.SPSYNC_LICENCE_KEY }}

# azure-pipelines.yml
schedules:
  - cron: "0 2 * * *"
    branches: { include: [main] }
    always: true
steps:
  - script: uv sync
  - script: uv run spsync run replication.yaml
    env:
      SPSYNC_DB_CONNECTION: $(SPSYNC_DB_CONNECTION)
      SPSYNC_LICENCE_KEY: $(SPSYNC_LICENCE_KEY)

# Windows Task Scheduler action (see /docs/cli/automation/)
$env:SPSYNC_DB_CONNECTION = (Get-Content C:\spsync\db.env)
uv run spsync run C:\spsync\replication.yaml
schtasks /Create /SC DAILY /ST 02:00 /TN "SPSync" /TR "powershell -File C:\spsync\run.ps1"

# ~/Library/LaunchAgents/io.spsync.run.plist, then: launchctl load ~/Library/LaunchAgents/io.spsync.run.plist