Openflow reads SharePoint files, Lakeflow adds lists in Beta

23 October 2026

Tags: Warehouses and lakesDesktopCommand line

You have Snowflake, or Databricks, and somebody has asked for the contracts register in the warehouse. The register is a SharePoint list with a Supplier lookup, an Owner person column and a multi-choice Region. You search for “snowflake openflow sharepoint”, find a connector at general availability, with a Marketplace listing, spend an afternoon on the app registration, and end up with a stage full of PDFs and no table of contracts. On the Databricks side the search turns up Lakeflow Connect, whose overview page leads with files and mentions lists in one line, with the Beta details on a second page.

Neither vendor hides what its connector does, but both write about files first. This post reads both sets of documentation as they stood on 23 October 2026, says what each connector does with a SharePoint list, and then sets out the routes that put list rows into each platform today.

Does Snowflake Openflow read SharePoint lists?

No. The Openflow Connector for SharePoint has reached general availability, and Snowflake describes it as connecting a SharePoint site to Snowflake “to ingest files and user permissions”. It comes in four variants. Two index files into a Cortex Search service, two ingest them into a Snowflake stage, and each pair offers a choice of carrying the document permissions with them or leaving them behind. None of the four reads the rows of a list.

The set-up page confirms it from the other direction. You give the connector a source folder, and the folder path is relative to a Shared Documents library. You give it a set of file extensions to ingest. There is no parameter for a list name or a list URL, and the page never mentions list items.

What the Openflow connector does with a document library

For the job it was built for, the connector is a sensible piece of work. It reads a library through a Microsoft Graph app registration granted Sites.Selected, so it only sees the sites you allow. The variants that carry permissions also need GroupMember.Read.All and User.ReadBasic.All to resolve SharePoint groups and user emails, and Snowflake asks for the fullcontrol site role so it can follow folder permission changes, with read as a narrower fallback.

The Cortex Search variants chunk the documents and index them, so an assistant can answer questions over policies or contracts with a filter on the user’s email. The Simple Ingest variants put the files in a stage for your own processing. Snowflake lists two known limitations: a folder that is moved or renamed is not picked up during incremental ingestion, and files of unsupported types are ignored.

If the question is “can an assistant in Snowflake read our contract PDFs”, Openflow is a good answer. If the question is “can I join the contracts register to the finance tables”, it has nothing to offer.

Does the Databricks Lakeflow Connect SharePoint connector ingest lists?

Yes, in Beta. The SharePoint connector in Lakeflow Connect is mainly a file connector. It ingests documents as binary content, parses CSV, JSON, XML or Excel files into Delta rows, or records file metadata without downloading the contents. The same page adds that you can also ingest SharePoint list data as structured Delta tables.

List ingestion is described on the pipeline page and is marked as a Beta feature. You set entity_type to LIST and point the URL at one list, which becomes one Delta table, or at a site, which ingests every list in it into a single schema with one table per list named <listId>_<listName>. The examples on that page run the pipeline on the preview channel.

The limitations page sets out what the Beta leaves out:

• ingesting every list across the tenant;
• lists that live in OneDrive;
• list attachments;
• reading lists through the SQL and Spark file APIs, since only the managed connector does it;
• creating the pipeline in the UI, so you write it as an API call or a bundle.

What a Lakeflow list table looks like

The connector reference maps each SharePoint column type to a Spark type. Text and choice columns become STRING, a multi-choice column becomes ARRAY<STRING>, numbers and currency become decimals, and dates become TIMESTAMP.

The lookup and person columns are where you will spend your time. A single-value lookup or person column arrives as one STRING. A multi-value lookup arrives as an array of structs holding the lookup id and the display value, and a multi-value person column adds the email to each struct. So the Owner on our contracts register is a string in the row, and a multi-person Approvers column is an array you explode in Spark before you can join anything to it.

That shape is fine for a notebook. It is harder work for a dimensional model, where you want a supplier key you can join to a supplier table and one row per contract and region. Treat it as Beta in the plain sense as well: the documentation does not say how an item deleted in SharePoint appears in the list table, so delete a test item and look before you build a report on it.

How do you get a SharePoint list into Snowflake today?

There are three routes that work now, and SPSync is part of two of them. SPSync copies SharePoint lists one way into a store you query, and SharePoint stays the system of record. Snowflake is on SPSync’s roadmap as a destination but is not one today, so in none of these routes does SPSync write to Snowflake directly.

The first route stages the lists as Parquet. SPSync’s DuckDB destination keeps a merged copy of each list and, at the end of every run, exports each table as Parquet to Amazon S3, Azure Blob Storage or ADLS Gen2. The export is set in the command-line tool’s YAML config today. Snowflake then reads the files from an external stage. The Snowflake page sets out the same route.

The second route stages the lists in a database you already load from. If an ELT tool already moves PostgreSQL or SQL Server tables into Snowflake for you, point SPSync at PostgreSQL or SQL Server and let that tool carry the tables the last mile.

The third route skips SPSync. General ELT platforms that load Snowflake from many sources have SharePoint list connectors of their own: Fivetran documents a Microsoft Lists connector, and Airbyte lists a SharePoint Lists connector among its Enterprise connectors. Our comparisons of SPSync and Fivetran and SPSync and Airbyte cover where each fits and how their schemas differ from SPSync’s.

Loading the Parquet files into Snowflake, step by step

This is the first route in detail, for a team that wants list rows in Snowflake on a schedule.

1. Set up a DuckDB destination in SPSync’s YAML config and add the bucket URL for the export, with the bucket’s credentials in environment variables.
2. Choose the lists and run once. The first run reads every item, including lists past the 5,000-item list view threshold, and writes a current.parquet file per table plus a history file for the run.
3. In Snowflake, create an external stage over the bucket or container.
4. Load each table’s current.parquet with COPY INTO, using MATCH_BY_COLUMN_NAME so a new SharePoint column in the file does not break the load.
5. Each file is a full snapshot, so replace the target table’s contents on each load, or merge on sp_item_id.
6. Schedule the SPSync run and a Snowflake task to follow it.

Later runs read only what changed in SharePoint, then export the full current state, so the file always matches the list. Items deleted in SharePoint stay in the snapshot with sp_deleted_at set, which lets a model filter them out or report on them. If you would rather have Snowpipe pick up every run as it arrives, point it at the per-run history files and reconcile the table to the latest snapshot in a task.

Every SPSync destination shares these table shapes. Our contracts register becomes a table with supplier_id and supplier_value, an owner_user_id that joins to sp_users, and a child table with one row per contract and region. For each SharePoint column type and what it turns into, see the schema reference.

Choosing between the Lakeflow list Beta and SPSync’s Databricks destination

Databricks is a shipped SPSync destination. SPSync connects through a SQL warehouse, writes Delta tables into the Unity Catalog schema you name, and updates each one with a MERGE INTO keyed on the SharePoint item id. The Databricks page covers the connection string, the privileges and the steps.

The Lakeflow Beta suits a team that wants everything in Databricks under one managed pipeline, is content to pull lookups out of strings and arrays in Spark, and can wait for the Beta to settle. SPSync fits a team that wants the lists as joinable tables from the first run: lookups as key columns, multi-value columns as child tables, people resolved against one shared users table, and deletes kept as a flagged row. SPSync also runs outside Databricks, on a machine or a scheduled job you control, which helps when the person who owns the lists is not the person who owns the workspace.

Whichever you pick, test it on your largest list with your most awkward columns before you commit, because both connectors are changing and SharePoint lists vary a lot.

Files in one pipeline, lists in another

The cleanest arrangement on either platform often uses both kinds of tool. The vendor’s connector reads the documents, where Openflow’s Cortex Search variants or Lakeflow’s file ingestion do work SPSync does not attempt. SPSync replicates the lists, and the metadata of the libraries, into tables. SPSync copies metadata and never the text of a file, so the two do not overlap.

On Databricks both can write into the same catalog, in separate schemas, and a query can join a document’s metadata row to the contract it belongs to. On Snowflake the documents sit in a stage or a Cortex Search service and the list rows sit in tables loaded from Parquet, under the same database.

One caution for any AI work over the result. Openflow can carry document permissions into Cortex Search. SPSync does not carry SharePoint item permissions into the replica, so the list tables are governed by the grants you set in the warehouse, and you decide who sees which rows there.

What to check before you commit

Vendor documentation on both platforms moves every few weeks, so read the pages again on the day you decide:

• Snowflake’s about page for any mention of list items, and the set-up page for a list parameter;
• the Databricks limitations page for whether list ingestion has left Beta and which exclusions remain;
• the Databricks reference for any change to how lookup and person columns arrive.

If you want to try the SPSync side, the warehouse use case shows how lists fit beside the rest of your sources, and the post on SharePoint lists in Parquet, Delta Lake and Iceberg shows the files each run writes. If Snowflake is the warehouse you need as a native destination, tell us how many lists and rows you have and how fresh the data needs to be.