Tags: Relational databasesScaleAutomationDesktopCommand line
You wrote the sync over a weekend. It calls /items/delta, follows the next links, saves
the delta link and runs every five minutes, and for a month the SQL table matched the
list. Then someone in finance asks why a cancelled order still shows in the report. It was
deleted from the list on Tuesday. You call the delta link by hand and get back nothing
that says so, and no line in your code knows the order is gone.
That is the gap this post covers. We build SPSync, which reads SharePoint lists into databases through Microsoft Graph delta, so we have written this code and been caught by the same things. The Microsoft behaviour below is cited to Microsoft Learn, read on 23 October 2026. Where we describe something we saw on our own tenant, or only in our test fake, we say so, because the two have disagreed before.
How does Graph delta work for SharePoint list items?
The
listItem: delta
page describes the round trip. The first call to
GET /sites/{siteId}/lists/{listId}/items/delta carries no token. SharePoint starts
enumerating the list and returns a page of items with an @odata.nextLink. You keep
calling the next link until a response carries an @odata.deltaLink instead. At that
point you hold every item, and the delta link is your bookmark. Calling it later returns
only items created, changed or deleted since.
Two lines on that page shape everything you build on top. The feed “shows the latest state for each item, not each change”, so an item renamed twice appears once with its latest name. And the same item “might appear more than once in a delta feed”, in which case you keep the last occurrence. Your write to the database has to be an upsert keyed on the item id, and running it twice has to be harmless.
The call supports $select, $expand and $top. It does not list $filter, so there is
no way to ask delta for only the open orders. You take the whole feed and filter in your
own store. Because delta never goes through a list view, the
5,000-item list view threshold
does not stop it, which is why our post on
what breaks past 5,000 items
recommends it for large lists.
Paging a delta query: next links, delta links and empty pages
Microsoft’s
delta query overview
adds the paging rules that apply to every delta resource, list items included. A page
never carries both a next link and a delta link. A next link can arrive with an empty
value array, and that still means more pages follow, so a loop that stops on the first
empty page stops too early. Page size can vary whatever $top you send, and Learn says
not to assume any order: an item can turn up on any page.
SPSync asks for 200 items a page and loops until a page has no next link. If the final
page also has no delta link, which Graph should never do, it logs a
delta_final_page_missing_delta_link warning rather than starting over with a full load
next time and saying nothing.
Cost is the other paging question. In Microsoft’s throttling guidance, a delta request with a token costs 1 resource unit and a multi-item query costs 2 resource units, with delta-with-a-token named as the exception. So the first, token-less pass over a large list is the expensive part, and a quiet list on a five-minute schedule costs very little after that.
Delta token expiry and the 410 Gone resync
Learn does not publish a lifetime for listItem delta tokens. The overview gives token
lifetimes for directory objects, education objects and Outlook items, and says nothing
about SharePoint. What the listItem page does document is the signal. When the service
cannot give you changes for a token, for example “if a client tries to reuse an old token
after being disconnected for a long time”, it answers 410 Gone with a Location header
holding a fresh next link that starts a new enumeration. The overview adds that a 410 can
also follow internal maintenance or a tenant migration.
Two things follow. You cannot schedule around expiry, because there is no published age to stay under, so the 410 path has to work on any run. And a fresh enumeration only tells you what exists now. Learn’s overview says that resources deleted before the initial delta query are not returned, so anything deleted while your token was dead will never appear as a delete. The only way to find it is to compare what the new enumeration returned with what your table holds.
SPSync handles a 410 in the same run. It follows the Location URL, keeping Graph’s own
parameters, reloads the list in full, and marks deleted every row it held whose id the
reload did not return. If the resync itself keeps answering 410, it gives up after five
retries and reports an error rather than looping. We have tested this path against a
simulated 410 in our Graph fake. The run log shows it as a list.delta.resync event,
listed with the other events in troubleshooting.
Storing the delta link and the $select inside it
The overview says state tokens “encode and include query parameters (such as $select)”
from the first request, and that later requests must not repeat or change them. Treat the
delta link as an opaque string: store it as it came and send it back the same way. If you
append $expand to a stored link, or hand it to an HTTP library that replaces the query
string when you add parameters, the token goes and the next call starts from scratch. We
hit exactly that bug early on, and SPSync’s Graph client now sends a stored link back with
its own query string intact.
Because the link remembers the $select, choose it carefully on the first call. On our
tenant in October 2026, a bare $expand=fields returned a lookup column’s id without its
display value; naming the column in fields($select=...) returned both. A delta link made
with the bare form keeps that shape until you start a new enumeration. SPSync always names
its columns, and stores the link in the destination database next to the data, keyed to
that replication and that destination, so pointing a replication at a new database starts
a full load there.
There is one ordering choice in SPSync’s first sync worth copying. It pages the token-less
delta call to the end first, to get the delta link, and only then reads every item. A
change made while the big read is running then replays on the next round. Read first and
take the link afterwards, and that change falls between the two and is lost until
something else catches it. The read itself goes through /items with an explicit
$select, and the delta link is saved only once that read has finished.
Why does listItem delta not return deleted items?
The documented answer is that it does. Microsoft’s
listItem delta page
shows a deleted item in its second example as an id, a parent reference, a content type
and a "deleted": {"state": "deleted"} facet. The sentence above that example reads
“Items with this property should be removed from your local state”, and on the page we
read it does not name the property. The general delta overview describes removed objects
differently again, with an @removed annotation, so code written from the overview looks
for the wrong key.
On our own dev tenant, on 6 September 2026, a deleted list item arrived with the
deleted facet, an empty fields object and no eTag. That matched the example, and SPSync
accepts both deleted and @removed.
Other developers have not been so lucky. A
Microsoft Q&A thread from April 2025
shows a deleted document coming back as an id, a parent reference and an empty fields
object, with no deleted marker at all. An
older thread from November 2023
reports no record of deleted items, and its most helpful answer moves document libraries
to the driveItem delta instead. Neither thread has a Microsoft fix. We have not reproduced
a missing delete on our tenant, but we cannot prove it never happens on yours, and a token
that expires hides deletes by design.
So handle both documented shapes when they arrive, and check the whole list against your table on a schedule, which also covers anything deleted while a token was dead. For a document library, SPSync also re-reads by id any delta row that comes back without the file details it asked for, and if that re-read returns 404 the row is marked deleted. The reconcile below covers the rest.
What a reconcile pass checks
A reconcile asks SharePoint a different question. Delta answers “what changed since my
bookmark?”, and a reconcile answers “what exists right now, and does it match my table?”
SPSync’s reconcile pages through /items asking only for id and eTag, 200 items a
page, and compares each eTag with the one it stored when it last wrote that row.
- An id SharePoint returns with a different eTag is read again in full and upserted.
- An id the table holds that SharePoint no longer returns gets
sp_deleted_atset. - An id that matches costs nothing beyond its share of the listing page.
One detail will bite you if you write this yourself. Graph quotes the eTag on
/items/{id} and leaves it unquoted on /items/delta, as our recorded responses from a
real tenant show, so the same version compares as different unless you strip the quotes
first. SPSync normalises both forms.
The cost is one paged listing per list, at Microsoft’s 2 resource units a page, plus a
full read of each item that drifted. That is why it runs weekly by default rather than on
every round. The interval is reconcile_interval_days in the replication config, and the
setup guide shows how to set it and how to run one by hand:
spsync reconcile servicedesk.yamlRun one before you hand a table to the people who own the list, as evidence the two match, and again after an outage on either side. The CLI automation page covers scheduling it alongside the normal runs.
Delta plus reconcile in SPSync, round by round
Put together, a list in SPSync goes through three kinds of round, and the run log names which one each was.
- The first round is a full load: take the delta link, read every item, write the table, save the link.
- Every scheduled round after that is a delta round from the saved link. Changed items are
upserted, deleted ones get
sp_deleted_at, and the new link is saved at the end. - When the reconcile interval has passed, that round is a reconcile instead, and the delta link stays where it was, so the next delta round picks up any change made in the meantime and the upsert absorbs the overlap.
Deletes stay as rows with a timestamp by default, so last month’s report still adds up, and the command-line tool has a per-replication setting to remove them outright. If a whole list is deleted, the delta call returns 404, SPSync flags the list’s rows as deleted and stops syncing it until you select it again.
None of this is exotic. A PnP PowerShell script or a scheduled function can do the same, and if you already run one, the checklist is the headings above: loop on next links through empty pages, store the link verbatim, handle the 410 by reloading and comparing, accept both delete shapes, and reconcile on a timer.
Is delta query change data capture for SharePoint?
Nearly, and the difference decides whether it suits you. SQL Server’s change data capture records each insert, update and delete made to a tracked table. Graph delta gives you the current state of each item that changed, collapsed, with no order promised. If a status went from Open to Hold to Closed between two calls, you see Closed once. For most reporting that is what you want: a table that matches the list a few minutes behind. For an audit of each edit, use SharePoint’s version history, which SPSync does not copy, or the Microsoft 365 audit log.
If your aim is a current copy of a large list in SQL Server or Postgres, kept honest about deletes, the 5,000-item threshold use case walks through picking the lists and the destination.