SharePoint to Amazon Redshift

Redshift is a later item on the SPSync roadmap. Parquet on S3 with COPY or Spectrum, or federated queries to RDS, cover it for now.

Book a call

Talk through your use case, see further walkthroughs or demos, and set up a trial in your own environment.

Last updated

When the Amazon Redshift destination ships it will copy one way, and SharePoint will stay the system of record.

Status: On the roadmap · product: SPSync for Warehouses & Lakes . We prioritise destinations by demand. Book a call to tell us you need it.

Redshift sits in the “later” column of SPSync’s roadmap: tracked, not scheduled, and not something SPSync writes to today. If your analytics run on Redshift, two routes through shipped AWS destinations already put SharePoint list data within reach of your queries.

Route one: Parquet on S3

A DuckDB destination can upload every table as Parquet to an S3 bucket after each run. Each table gets a current.parquet snapshot replaced every run, and a history file for each run.

Deleted items are in each snapshot with sp_deleted_at set, so filter on it in views.

Route two: a federated query to PostgreSQL

Replicate the lists to Amazon RDS or Aurora PostgreSQL, where SPSync keeps them current with delta sync, upserts and soft deletes. Redshift’s federated query feature can then read those tables live and join them to warehouse data. This route works well for lists that change often, because there is no file to reload.

Choosing between the two

Pick Parquet on S3 when the lists are large, change slowly and feed heavy analytical queries, since Redshift then scans its own storage or Spectrum’s. Pick the federated route when freshness matters more than scan speed, or when an app already reads the PostgreSQL copy. Both keep SPSync’s delta sync on the SharePoint side, so the Graph load is the same either way.

What a native destination would involve

The shapes would match every shipped destination: {col}_id and {col}_value for lookups, {col}_user_id joined to a shared sp_users table, a child table per multi-value column, as the schema reference sets out. The Redshift-specific work is what keeps it on the later list until demand justifies it:

Tell us it matters

If Redshift is what you need, get in touch with the number of lists, a rough row count and how current the data has to be. That is what moves a later item onto the schedule, and we will contact you if it does.

Frequently asked questions

Is Redshift available today?

No. Redshift is tracked but not yet scheduled, further out than the destinations marked as coming soon, and SPSync cannot write to it today.

How can I query SharePoint data from Redshift now?

Export Parquet to S3 from a DuckDB destination and load it with COPY or query it in place with Redshift Spectrum. Or replicate to RDS or Aurora PostgreSQL and use a Redshift federated query.

What would move Redshift onto the roadmap?

Demand. Tell us what you would sync and how large the lists are. Distribution and sort keys, and how loads are staged, would be decided when the work starts.

How do I get notified if it is scheduled?

Leave your email on this page or get in touch. We will contact you if Redshift moves onto the schedule.

Talk to us about Amazon Redshift

SPSync is licensed per installation and per target family, with Standard Support included. Tell us which targets you need and how many machines, and we will size it with you. See how licensing works.

Script it

Everything you can click, you can script. The CLI, YAML config and automation below reproduce this scenario exactly. See the CLI reference.

# This destination has no --destination-kind in the command line today.
# In the meantime, run this from a shipped destination (Postgres or SQL Server)
# and repoint --connection-string-env at your database.
# YAML for this destination's kind ships with its --destination-kind support.

Cron (this destination nightly):

# every night at 02:00
0 2 * * * cd /opt/spsync && SPSYNC_DB_CONNECTION="$(cat /etc/spsync/db.env)" uv run spsync run replication.yaml >> /var/log/spsync/run.log 2>&1

GitHub Actions:

# .github/workflows/spsync-run.yml
name: SPSync
on:
  schedule:
    - cron: "0 2 * * *"
  workflow_dispatch: {}
jobs:
  run:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v3
      - run: uv sync
      - run: uv run spsync run replication.yaml
        env:
          SPSYNC_DB_CONNECTION: ${{ secrets.SPSYNC_DB_CONNECTION }}
          SPSYNC_LICENCE_KEY: ${{ secrets.SPSYNC_LICENCE_KEY }}

Azure DevOps:

# azure-pipelines.yml
schedules:
  - cron: "0 2 * * *"
    branches: { include: [main] }
    always: true
steps:
  - script: uv sync
  - script: uv run spsync run replication.yaml
    env:
      SPSYNC_DB_CONNECTION: $(SPSYNC_DB_CONNECTION)
      SPSYNC_LICENCE_KEY: $(SPSYNC_LICENCE_KEY)

Windows (PowerShell + Task Scheduler):

# Windows Task Scheduler action (see /docs/cli/automation/)
$env:SPSYNC_DB_CONNECTION = (Get-Content C:\spsync\db.env)
uv run spsync run C:\spsync\replication.yaml
schtasks /Create /SC DAILY /ST 02:00 /TN "SPSync" /TR "powershell -File C:\spsync\run.ps1"

macOS (launchd):

# ~/Library/LaunchAgents/io.spsync.run.plist, then: launchctl load ~/Library/LaunchAgents/io.spsync.run.plist

Secrets (licence key, connection strings) come from the environment, so never write them into the YAML. See scheduling and secrets.

Book a call

Tell us about your lists and where they need to go: we will talk through your use case, show further walkthroughs or demos, and set up a trial in your own environment.