PnP Powershell SharePoint: How to Use Get-PnPSearchCrawlLog

Introduction

In SharePoint on premises, administrators could log directly onto servers to inspect the crawl log and monitor search indexing. In SharePoint Online, this is no longer possible, making it tricky to track crawl activity and diagnose indexing issues.

Fortunately, PnP PowerShell provides the Get-PnPSearchCrawlLog cmdlet, which allows you to inspect crawl activity programmatically.

Certain activities can significantly increase the load on SharePoint’s crawl and indexing processes, such as:

  • Migrating large volumes of content into the tenant
  • Enabling OneDrive for Business
  • Enabling or disabling Restricted Content Discoverability (RCD) to control content visibility for Copilot (see Microsoft docs)
  • Forcing reindexing of libraries or sites, for example after changes to the search schema or to resolve search issues

To avoid overloading your tenant and causing delays (such as intranet news not appearing promptly), it’s essential to plan these activities carefully—ideally performing them in phases or batches.

Common Issue: Unauthorized Operation Error

When running Get-PnPSearchCrawlLog for the first time, you might encounter the following error if you have been granted access:

Crawl log error

Get-PnPSearchCrawlLog: Error: Attempted to perform an unauthorized operation.. Make sure you are granted access to the crawl log via the SharePoint search admin center at https://<tenant>-admin.sharepoint.com/_layouts/15/searchadmin/crawllogreadpermission.aspx

Solution: Grant Crawl Log Permissions

To resolve this, ensure you have access to the crawl log:

  1. Go to the SharePoint Search Admin Center:
    https://<tenant>-admin.sharepoint.com/_layouts/15/searchadmin/crawllogreadpermission.aspx
  2. Grant yourself the necessary permissions.

Crawl permission

Once access is granted, you can successfully run the cmdlet.

Example: Exporting Crawl Log Data

Below is a sample script to connect to SharePoint Online, retrieve crawl log data for the past 24 hours, and export hourly crawl counts to a CSV file.

connect-PnPOnline -Url "https://contoso.sharepoint.com/sites/edrm"
$logPath = "CrawledLogResults.csv"
$numberofhourstogoback = 24

# Prepare log file
if (-not (Test-Path $logPath)) {
    "Hour,StartTime,EndTime,CrawledItemCount" | Out-File $logPath -Encoding utf8
}
$crawledItems = Get-PnPSearchCrawlLog -StartDate (Get-Date).AddDays(-1) -RowLimit 100000 -ContentSource Sites

for ($i = 1; $i -le $numberofhourstogoback; $i++) {
    $endTime = (Get-Date).AddHours(-($numberofhourstogoback - $i))
    $startTime = $endTime.AddHours(-1)

    $count = 0
    $crawledItems | ForEach-Object {
        if ($_.CrawlTime -ge $startTime -and $_.CrawlTime -lt $endTime) {
            $count++
        }
    }
    $logLine = "$i,$($startTime.ToString('yyyy-MM-dd HH:mm:ss')),$($endTime.ToString('yyyy-MM-dd HH:mm:ss')),$count"
    Add-Content -Path $logPath -Value $logLine
    Write-Host "Hour $($i) : $count crawled items from $startTime to $endTime" -ForegroundColor Green
}

Output

I started reindexing of a libray around 19:00 on the 18th July 2025 which showed higher activity during that time.

Crawl permission log error

Additional Errors

You may also see timeout errors if the crawl log is large or the query takes too long:

Crawl permission log error

Get-PnPSearchCrawlLog: Error: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

If this occurs, try using filters or row limit. I managed to return up to 150k records. I tried to filter down the results to return only the crawled items within the last hour, however it was still returning items from 24 hours. In the backgroung PnP PowerShell calls the method CrawlLog.GetCrawledUrls method. Anyone who knows how to reliably return all items crawled in the last hour and with higher row limit do reach out.

$crawledItems = Get-PnPSearchCrawlLog -StartDate (Get-Date).AddHours(-1) -RowLimit 1000000

References