Blog
Posts are grouped by topic below, after the five most recent. Each topic also has a guide page of its own.
Filter by tag
Latest
- Archive old SharePoint list items without losing the history
Moving closed items to an archive list just builds a second big list. Keep every row in SQL, then let a scheduled flow trim the live list safely.
- Azure AI Foundry agents and SharePoint list rows
The Foundry SharePoint tool (preview) retrieves text from documents, not list rows. What each Foundry tool reads today, and how to query a list replica.
Tags: Relational databasesSearch and vector storesAIDesktopCommand line
- SharePoint lists in Foundry IQ after On Your Data retired
Azure OpenAI On Your Data retired on 14 October 2026. How to ground answers on a SharePoint list in Foundry IQ, through an Azure AI Search index kept current.
- Copilot Studio says it cannot find the item at row 3,000
Your agent answers about some list items and says others do not exist. The 2,048-row read and search ranking explain it, with a test to show what it sees.
- SharePoint document library metadata in SQL Server, with folders and content types
You need a report of every document in a library: its content type, folder, owner and missing fields. How library metadata arrives in SQL, row by row, and what to check.
Getting started with SPSync
First syncs on the desktop app and SPSync in your Azure: install, sign in, permissions and the first replication, end to end.
- Mirrored, copied, synced or replicated: what each word means for a SharePoint list
Mirror, sync, copy and replicate mean four different things for a SharePoint list in 2026. What Microsoft means by each, and which one gets your list into SQL.
- Desktop app, command line or Azure: which to start with
The three SPSync editions run one engine and write the same tables. Who runs each, where its state lives, how it is scheduled and how a licence counts it.
- Your first SharePoint replication, start to finish
One list, from sign-in to a table you can query: Sites.Selected, picking the list, SQLite or Postgres, the first full load, the second run and checking row counts.
Reporting and Power BI on SharePoint data
Getting SharePoint list data into a database so Power BI, Excel and Access report against a table and leave the list connector behind.
- Export a SharePoint list to Excel automatically each morning
Export to Excel, a Power Automate flow and Power Query each get a list into a workbook. Where each one stops, and how a SQL copy keeps the workbook current.
- Why incremental refresh does nothing on a SharePoint list in Power BI
You set RangeStart, RangeEnd and a policy, and the refresh takes as long as before. Why the date filter never reaches the list, and a source where it does.
- SharePoint Online list 2.0 in Power BI: limits and errors
Switch the connector to 2.0 and a working report starts failing on lookups, missing columns or rows. What changed from 1.0, the exact errors, and your options.
- Tableau on SharePoint lists after the connector deprecation
Tableau retired its built-in SharePoint Lists connector. What the JDBC replacement does, why its extracts reread every row, and how a SQL copy helps.
- Claims data from SharePoint in Azure SQL or Managed Instance
An insurer's policies, claims and referrals replicated into Azure SQL Database or Managed Instance, with the firewall and VNet steps each one needs.
- Why Power BI refreshes on SharePoint lists slow down
The SharePoint List connector pages item by item, so refresh time climbs with every row and lookup. A replicated table puts Power BI back on database speed.
- Seven steps from a SharePoint list to a SQL Server table
Connect, review, pick, connect SQL Server, run, schedule and monitor: each step of a SharePoint list replication in the desktop app, Azure or the CLI.
- SharePoint list to SQL Server for a faster Power BI report
Projects, Tasks and Departments replicated from SharePoint into SQL Server on a schedule, then read by Power BI with DirectQuery. Video included.
Databases and apps on SharePoint data
Building an application read replica or a document catalogue in Postgres, Supabase or another database, with a normalised schema an app can query directly.
- SharePoint document library metadata in SQL Server, with folders and content types
You need a report of every document in a library: its content type, folder, owner and missing fields. How library metadata arrives in SQL, row by row, and what to check.
- Microsoft Lists to SQL Server: a Lists app list is a SharePoint list underneath
You built a tracker in Microsoft Lists and now it has to reach SQL Server. Where the list lives decides the route: team site lists replicate like any SharePoint list.
- Power Apps on SharePoint past the row limit: when to use SQL
Why a Power Apps gallery on a large SharePoint list misses rows, what SharePoint can and cannot delegate, and when the app's read screens belong on SQL.
- Querying a SharePoint list with SQL: four routes compared
An analyst wants one SELECT against a SharePoint list and a join to the finance database. Drivers, linked servers, the content database and a copy, weighed.
- SharePoint as a database: where a list stops being one
What a SharePoint list does well as a database, the limits it runs into, a decision table, and the options from Dataverse to a SQL copy you report from.
- SharePoint list to Oracle Database for an APEX report and a PL/SQL job
Your APEX report and a nightly PL/SQL job need a SharePoint list as Oracle tables. How to set up the two Oracle users, what the tables look like, and how the copy stays current.
- SharePoint to MySQL or MariaDB for a Grafana defects board
Ashcroft Build Group's defects, projects and subcontractor lists copied from SharePoint into MySQL or MariaDB, then charted in Grafana by site.
- SharePoint lists in PostgreSQL, ready for Metabase and apps
A retail walkthrough: three SharePoint lists replicated into PostgreSQL, with joinable lookups, child tables and a Metabase dashboard on top.
- An offline SQLite copy of SharePoint lists for caseworkers
Riverside Borough Council's FOI, enquiry and councillor lists kept in one SQLite file that a caseworker can query on a laptop without a connection.
- A client portal on Supabase, fed from SharePoint lists
A law firm's matters, clients and conflicts lists replicated into Supabase Postgres, ready for the table editor, the generated REST API and Realtime.
AI and search over SharePoint data
Agents, Copilot Studio, text-to-SQL chat and retrieval over SharePoint lists and library metadata, all reading one replicated copy, with what each tool can see set by the database role and row-level security you grant.
- Azure AI Foundry agents and SharePoint list rows
The Foundry SharePoint tool (preview) retrieves text from documents, not list rows. What each Foundry tool reads today, and how to query a list replica.
Tags: Relational databasesSearch and vector storesAIDesktopCommand line
- SharePoint lists in Foundry IQ after On Your Data retired
Azure OpenAI On Your Data retired on 14 October 2026. How to ground answers on a SharePoint list in Foundry IQ, through an Azure AI Search index kept current.
- Copilot Studio says it cannot find the item at row 3,000
Your agent answers about some list items and says others do not exist. The 2,048-row read and search ranking explain it, with a test to show what it sees.
- Keeping SharePoint data in region for AI: where prompts, embeddings and indexes go
Your data protection officer asks whether AI on a SharePoint list sends it abroad. Four hops decide it: the model, the embedding call, the index and the logs.
Tags: Warehouses and lakesSearch and vector storesAIDesktopCommand line
- Agents on SharePoint data without a Graph app registration
Your request for a Graph app registration for an AI loader was refused. One read-only Sites.Selected grant to a replica, then a database role per tool, is easier to approve.
- Copilot Studio and SharePoint lists: four routes compared
A Copilot Studio maker with a 9,000-item list has four ways to answer from it. What list knowledge, a flow, Dataverse and a SQL replica each cost.
- Supabase and SharePoint lists for an app and an AI agent
A field-ops app on Supabase and an AI agent reading the same replicated SharePoint lists, with row-level security per team and a read-only role for the agent.
- Human-in-the-loop AI agents over SharePoint evidence
An agent checks contractor certificates for each job against a SQL replica of SharePoint, writes the gaps to a review table, and a person approves the packet.
- One SQL copy of SharePoint for every agent you run
What a cross-list question costs an AI agent through Graph, worked for three real-sized lists, and the same question as one SQL join on a replica.
- Why your agent should not query SharePoint live
Agent tools that call Microsoft Graph share SharePoint's throttling budgets with every other integration. When a live read is right, and when to use a replica.
- Four ways to embed SharePoint list text, compared
A facilities incident list embedded four ways, from a Power Automate flow to embedding on sync into pgvector, with the upkeep each one brings.
Tags: Relational databasesSearch and vector storesAIDesktopCommand line
- Running a local AI model over SharePoint data in DuckDB
A law firm that cannot send client data to a public AI service replicates its Matters list into DuckDB on a laptop and queries it with Ollama.
Tags: Relational databasesWarehouses and lakesAIDesktopCommand line
- Library metadata is the catalogue an AI assistant needs
A policy assistant quoted a withdrawn document because it only read file text. A replicated metadata table lets it pick documents by status first.
Tags: Relational databasesSearch and vector storesAIDesktopCommand line
- Read SharePoint once and let the AI pipeline query the copy
A finance assistant hit 429s every month-end. The resource-unit arithmetic behind Graph throttling, and how one delta reader and a replica fix it.
Tags: Relational databasesSearch and vector storesAIDesktopCommand line
- Keeping a SharePoint vector index current as items change
An HR assistant kept quoting a carry-over rule that changed in March. The four kinds of embedding drift, and re-embedding only rows that changed.
- Training and evaluation sets from SharePoint records
A service desk scores a ticket-type assistant against three years of SharePoint tickets, with SQL over a replica and snapshots that can be rebuilt.
Tags: Relational databasesWarehouses and lakesAIDesktopCommand line
- An MCP server for SharePoint data that queries SQL
Claude, ChatGPT or Cursor reading a read-only SQL replica of SharePoint lists through a database MCP server, with no Graph permission of its own.
- Why Copilot cannot count the rows in your SharePoint list
Copilot Studio reads the first 2,048 rows of a list and counts what search returns. Here are the documented limits and what a SQL replica changes.
- Text-to-SQL on SharePoint data depends on the schema
A project office asked a chat box which clients had projects at risk and got ids and claims strings back. The schema fixes that make the SQL right.
Scale, large lists and multi-site consolidation
Working past SharePoint's list-view threshold and bringing the same list from many sites together into one reportable table.
- Graph list item delta: paging, tokens and missed deletes
You built a sync on /items/delta, deleted an item, and the next call never said so. How listItem delta pages, what its tokens carry, and the reconcile that catches the rest.
Tags: Relational databasesScaleAutomationDesktopCommand line
- The 12 lookup column threshold in views, flows, Power Apps and Power BI
Add a thirteenth lookup or person column and views, flows, apps and Power BI refreshes start failing with one error. What counts, where it shows, and a fix.
- Combine SharePoint lists from multiple sites into one table
Twelve project sites, one RAID log template, and a portfolio report built by hand every Monday. How a per-site prefix or one merged table puts every site's list in SQL.
- What breaks past 5,000 items and how delta reading avoids it
A service desk list passes 5,000 items and its views, flows and exports stop. What the list view threshold blocks, where indexes stop helping, and how a delta read copies the whole list.
Migration and cut-over
Moving SharePoint list data into a database for good and switching the applications and reports that depend on it over safely.
- Archive old SharePoint list items without losing the history
Moving closed items to an archive list just builds a second big list. Keep every row in SQL, then let a scheduled flow trim the live list safely.
- SharePoint list cutover to a database without losing rows
Replicate the list, check counts and a sample by item id, freeze edits, run a final delta and a reconcile, then switch the app and keep SharePoint read-only.
- Merging two SharePoint tenants' lists into one database
After an acquisition, replicate both tenants' SharePoint lists into one database with a connection and a table prefix per tenant, then report from one view.
- Script a SharePoint list migration into a database
Move SharePoint list data into a database with a CLI, a YAML config and batch definitions, so every run ends with the commands that reproduce it in the next environment.
Warehouses and lakes
Replicating SharePoint list data into Parquet, Delta Lake, Iceberg, BigQuery and Databricks, for analytics engines that read files and tables, not APIs.
- Fabric SharePoint list mirroring, or a SQL replica?
Fabric mirrors a SharePoint list for free, but only on a running Fabric capacity. What that costs a SQL Server or Postgres team, and when a SQL replica fits.
Tags: Relational databasesWarehouses and lakesDesktopCommand line
- Openflow reads SharePoint files, Lakeflow adds lists in Beta
Snowflake Openflow's SharePoint connector ingests files and permissions. Databricks Lakeflow Connect adds list ingestion in Beta. What each does with a list.
- dbt models on SharePoint data: keys, deletes and views
Staging SharePoint lists in dbt from a replica: which key to use, how child tables and soft deletes behave, when the generated views help, and tests that catch broken lookups.
Tags: Relational databasesWarehouses and lakesDesktopCommand line
- SharePoint lists in Parquet, Delta Lake and Iceberg
SharePoint list data as Parquet files, Delta tables and Iceberg tables: folder layout, soft deletes, what each run writes, a Fabric route and sample queries.
Automation and the command line
Running SPSync unattended from the CLI: scheduled syncs, CI pipelines and server licensing.
- Export a SharePoint list to SQL Server with PnP PowerShell
A PnP PowerShell script that copies a SharePoint Online list into a SQL Server table, with app-only sign-in, and the places it breaks once it runs every night.
- Power Automate Get items past 5,000, 100,000 and beyond
Why Get items stops at 100, 5,000 or 100,000 items, what each cap belongs to, and what a flow copying a list to SQL Server misses on deletes.
- Python: a SharePoint list into Postgres or SQL Server
A tested Python script that copies a SharePoint list into Postgres or SQL Server with Microsoft Graph delta: paging, deltaLink storage, Retry-After and deletes.
- What it costs to keep a SharePoint export script running
Schema drift, deletes, lookups, throttling and monitoring: the five jobs a SharePoint-to-SQL Server script or flow takes on once it runs on a schedule.
- Run a SharePoint list to database sync from CI
Schedule spsync run to copy SharePoint lists into a database from GitHub Actions, Azure DevOps, cron, Windows Task Scheduler or launchd, with secrets kept out of the YAML.