Last updated
When the Amazon Redshift destination ships it will copy one way, and SharePoint will stay the system of record.
Status: On the roadmap · product: SPSync for Warehouses & Lakes . We prioritise destinations by demand. Book a call to tell us you need it.
Redshift sits in the “later” column of SPSync’s roadmap: tracked, not scheduled, and not something SPSync writes to today. If your analytics run on Redshift, two routes through shipped AWS destinations already put SharePoint list data within reach of your queries.
Route one: Parquet on S3
A DuckDB destination can upload every table as
Parquet to an S3 bucket after each run. Each table gets a
current.parquet snapshot replaced every run, and a history file for each run.
- COPY. Load
current.parquetinto a Redshift table on your own schedule, replacing its contents each time, since every file is a full snapshot. - Redshift Spectrum. Define an external table over the folder and query the snapshot in place, with no load step.
Deleted items are in each snapshot with sp_deleted_at set, so filter on it in views.
Route two: a federated query to PostgreSQL
Replicate the lists to Amazon RDS or Aurora PostgreSQL, where SPSync keeps them current with delta sync, upserts and soft deletes. Redshift’s federated query feature can then read those tables live and join them to warehouse data. This route works well for lists that change often, because there is no file to reload.
Choosing between the two
Pick Parquet on S3 when the lists are large, change slowly and feed heavy analytical queries, since Redshift then scans its own storage or Spectrum’s. Pick the federated route when freshness matters more than scan speed, or when an app already reads the PostgreSQL copy. Both keep SPSync’s delta sync on the SharePoint side, so the Graph load is the same either way.
What a native destination would involve
The shapes would match every shipped destination: {col}_id and {col}_value for
lookups, {col}_user_id joined to a shared sp_users table, a child table per
multi-value column, as the schema reference sets out. The
Redshift-specific work is what keeps it on the later list until demand justifies it:
- distribution and sort keys for replicated lists and their child tables;
- staging loads through S3 and
COPY, and merging them; - authentication for an unattended sync.
Tell us it matters
If Redshift is what you need, get in touch with the number of lists, a rough row count and how current the data has to be. That is what moves a later item onto the schedule, and we will contact you if it does.
Frequently asked questions
Is Redshift available today?
No. Redshift is tracked but not yet scheduled, further out than the destinations marked as coming soon, and SPSync cannot write to it today.
How can I query SharePoint data from Redshift now?
Export Parquet to S3 from a DuckDB destination and load it with COPY or query it in place with Redshift Spectrum. Or replicate to RDS or Aurora PostgreSQL and use a Redshift federated query.
What would move Redshift onto the roadmap?
Demand. Tell us what you would sync and how large the lists are. Distribution and sort keys, and how loads are staged, would be decided when the work starts.
How do I get notified if it is scheduled?
Leave your email on this page or get in touch. We will contact you if Redshift moves onto the schedule.
Talk to us about Amazon Redshift
SPSync is licensed per installation and per target family, with Standard Support included. Tell us which targets you need and how many machines, and we will size it with you. See how licensing works.
Script it
Everything you can click, you can script. The CLI, YAML config and automation below reproduce this scenario exactly. See the CLI reference.
# This destination has no --destination-kind in the command line today.
# In the meantime, run this from a shipped destination (Postgres or SQL Server)
# and repoint --connection-string-env at your database. # YAML for this destination's kind ships with its --destination-kind support.
Cron (this destination nightly):
# every night at 02:00
0 2 * * * cd /opt/spsync && SPSYNC_DB_CONNECTION="$(cat /etc/spsync/db.env)" uv run spsync run replication.yaml >> /var/log/spsync/run.log 2>&1 GitHub Actions:
# .github/workflows/spsync-run.yml
name: SPSync
on:
schedule:
- cron: "0 2 * * *"
workflow_dispatch: {}
jobs:
run:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v3
- run: uv sync
- run: uv run spsync run replication.yaml
env:
SPSYNC_DB_CONNECTION: ${{ secrets.SPSYNC_DB_CONNECTION }}
SPSYNC_LICENCE_KEY: ${{ secrets.SPSYNC_LICENCE_KEY }} Azure DevOps:
# azure-pipelines.yml
schedules:
- cron: "0 2 * * *"
branches: { include: [main] }
always: true
steps:
- script: uv sync
- script: uv run spsync run replication.yaml
env:
SPSYNC_DB_CONNECTION: $(SPSYNC_DB_CONNECTION)
SPSYNC_LICENCE_KEY: $(SPSYNC_LICENCE_KEY) Windows (PowerShell + Task Scheduler):
# Windows Task Scheduler action (see /docs/cli/automation/)
$env:SPSYNC_DB_CONNECTION = (Get-Content C:\spsync\db.env)
uv run spsync run C:\spsync\replication.yaml
schtasks /Create /SC DAILY /ST 02:00 /TN "SPSync" /TR "powershell -File C:\spsync\run.ps1" macOS (launchd):
# ~/Library/LaunchAgents/io.spsync.run.plist, then: launchctl load ~/Library/LaunchAgents/io.spsync.run.plist Secrets (licence key, connection strings) come from the environment, so never write them into the YAML. See scheduling and secrets.