Upgrade Pagefind to 1.3.0 and configure Pagefind logging levels #2728
Conversation
|
✅ Deploy Preview for astro-starlight ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
size-limit report 📦
|
|
I’ve repeated @HiDeoo’s exercise from #1750 of comparing results for a selection of search terms in the Starlight docs. For results that changed rank, I’ve added the difference in ranking compared to Pagefind v1 to help highlight what changed. Bolded entries are results that I’ve subjectively decided are “good” or “best” results, to help track those across columns. I’ve also added a subjective “impact” assessment for each query to summarize my impressions. Query dataQuery:
|
| Rank | 1.0 | 1.3, termFrequency: 1 |
1.3, termFrequency: 0.5 |
1.3, termFrequency: 0 |
|---|---|---|---|---|
| 1 | Manual Setup | +1 Plugins Reference |
+1 Plugins Reference |
+1 Plugins Reference |
| 2 | Plugins Reference | -1 Manual Setup |
-1 Manual Setup |
-1 Manual Setup |
| 3 | Using components | Using components | Using components | +2 Customizing Starlight |
| 4 | Getting Started | Getting Started | Getting Started | -1 Using components |
| 5 | Customizing Starlight | Customizing Starlight | Customizing Starlight | -1 Getting Started |
Query: installation
Impact: Neutral — we don’t have clear content for this query, closest being “Manual Setup”
| Rank | 1.0 | 1.3, termFrequency: 1 |
1.3, termFrequency: 0.5 |
1.3, termFrequency: 0 |
|---|---|---|---|---|
| 1 | Site Search | +1 Authoring Content in Markdown |
+1 Authoring Content in Markdown |
+1 Authoring Content in Markdown |
| 2 | Authoring Content in Markdown | -1 Site Search |
-1 Site Search |
+1 Customizing Starlight |
| 3 | Customizing Starlight | Customizing Starlight | Customizing Starlight | -2 Site Search |
| 4 | CSS & Styling | CSS & Styling | CSS & Styling | CSS & Styling |
| 5 | Manual Setup | Manual Setup | Manual Setup | +1 Configuration Reference |
Query: page
Impact: 👎 Negative — wildly different results, which in the worst case removes the “Pages” result entirely
| Rank | 1.0 | 1.3, termFrequency: 1 |
1.3, termFrequency: 0.5 |
1.3, termFrequency: 0 |
|---|---|---|---|---|
| 1 | Pages | +15 Internationalization (i18n) |
+15 Internationalization (i18n) |
+5 Overrides Reference |
| 2 | Site Search | +4 Overrides Reference |
+4 Overrides Reference |
+6 Frontmatter Reference |
| 3 | Overriding Components | -1 Site Search |
-1 Site Search |
-2 Pages |
| 4 | Customizing Starlight | +4 Frontmatter Reference |
+4 Frontmatter Reference |
+7 Configuration Reference |
| 5 | Eco-friendly docs | +6 Configuration Reference |
+6 Configuration Reference |
+11 Internationalization (i18n) |
| (Pages ranked №8) | (Pages ranked №8) |
Query: markdown
Impact: 👎 Negative — downranks the obvious result, which is not ideal
| Rank | 1.0 | 1.3, termFrequency: 1 |
1.3, termFrequency: 0.5 |
1.3, termFrequency: 0 |
|---|---|---|---|---|
| 1 | Authoring Content in Markdown | +9 Overrides Reference |
+9 Overrides Reference |
Authoring Content in Markdown |
| 2 | Link Cards | -1 Authoring Content in Markdown |
-1 Authoring Content in Markdown |
+8 Overrides Reference |
| 3 | Steps | -1 Link Cards |
-1 Link Cards |
+2 Pages |
| 4 | Card Grids | +1 Pages |
+1 Pages |
Card Grids |
| 5 | Pages | -1 Card Grids |
-1 Card Grids |
-4 Link Cards |
Query: component
Impact: 👍 Positive — surfaces the same key content, plus adds “Pages” which does include page component docs
| Rank | 1.0 | 1.3, termFrequency: 1 |
1.3, termFrequency: 0.5 |
1.3, termFrequency: 0 |
|---|---|---|---|---|
| 1 | Using components | Using components | Using components | +2 Overrides Reference |
| 2 | Overriding Components | +19 Pages |
+19 Pages |
-1 Using components |
| 3 | Overrides Reference | Overrides Reference | Overrides Reference | -1 Overriding Components |
| 4 | Eco-friendly docs | -2 Overriding Components |
-2 Overriding Components |
+2 File Tree |
| 5 | Code | +1 File Tree |
+1 File Tree |
+3 Card Grids |
Query: css
Impact: Neutral — no significant changes for the most part
| Rank | 1.0 | 1.3, termFrequency: 1 |
1.3, termFrequency: 0.5 |
1.3, termFrequency: 0 |
|---|---|---|---|---|
| 1 | CSS & Styling | CSS & Styling | CSS & Styling | CSS & Styling |
| 2 | Icons | Icons | Icons | +2 Customizing Starlight |
| 3 | Starlight Showcase | +1 Customizing Starlight |
+1 Customizing Starlight |
+2 Configuration Reference |
| 4 | Customizing Starlight | -1 Starlight Showcase |
-1 Starlight Showcase |
-2 Icons |
| 5 | Configuration Reference | Configuration Reference | Configuration Reference | -2 Starlight Showcase |
Query: language
Impact: Neutral — over promotes “Overrides Reference”, but otherwise OK
| Rank | 1.0 | 1.3, termFrequency: 1 |
1.3, termFrequency: 0.5 |
1.3, termFrequency: 0 |
|---|---|---|---|---|
| 1 | Internationalization (i18n) | +4 Overrides Reference |
+4 Overrides Reference |
Internationalization (i18n) |
| 2 | Link Cards | -1 Internationalization (i18n) |
-1 Internationalization (i18n) |
+2 Configuration Reference |
| 3 | Make your docs shine with Starlight | -1 Link Cards |
+1 Configuration Reference |
+2 Overrides Reference |
| 4 | Configuration Reference | Configuration Reference | -2 Link Cards |
+3 Authoring Content in Markdown |
| 5 | Overrides Reference | +1 Pages |
+1 Pages |
+3 Sidebar Navigation |
Query: sidebar
Impact: 👎 Negative — only the termFrequency: 0 variant returns similarly appropriate results
| Rank | 1.0 | 1.3, termFrequency: 1 |
1.3, termFrequency: 0.5 |
1.3, termFrequency: 0 |
|---|---|---|---|---|
| 1 | Sidebar Navigation | +4 Configuration Reference |
+4 Configuration Reference |
Sidebar Navigation |
| 2 | Overrides Reference | Overrides Reference | Overrides Reference | Overrides Reference |
| 3 | Pages | Pages | Pages | +1 Frontmatter Reference |
| 4 | Frontmatter Reference | -3 Sidebar Navigation |
-3 Sidebar Navigation |
+1 Configuration Reference |
| 5 | Configuration Reference | -1 Frontmatter Reference |
-1 Frontmatter Reference |
-2 Pages |
Query: lastUpdated
Impact: Neutral — no change
| Rank | 1.0 | 1.3, termFrequency: 1 |
1.3, termFrequency: 0.5 |
1.3, termFrequency: 0 |
|---|---|---|---|---|
| 1 | Overrides Reference | Overrides Reference | Overrides Reference | Overrides Reference |
| 2 | Frontmatter Reference | Frontmatter Reference | Frontmatter Reference | Frontmatter Reference |
| 3 | Configuration Reference | Configuration Reference | Configuration Reference | Configuration Reference |
Query: plugin
Impact: Neutral — no significant change (reference page second would be nice, but hard to expect the search index to distinguish between guides and reference without assistance)
| Rank | 1.0 | 1.3, termFrequency: 1 |
1.3, termFrequency: 0.5 |
1.3, termFrequency: 0 |
|---|---|---|---|---|
| 1 | Plugins and Integrations | +1 Plugins Reference |
+1 Plugins Reference |
+1 Plugins Reference |
| 2 | Plugins Reference | -1 Plugins and Integrations |
-1 Plugins and Integrations |
-1 Plugins and Integrations |
| 3 | Configuration Reference | CSS & Styling | CSS & Styling | Configuration Reference |
| 4 | CSS & Styling | +1 Site Search |
+1 Site Search |
CSS & Styling |
| 4 | Site Search | -1 Configuration Reference |
-1 Configuration Reference |
Site Search |
Assessment
I find it a bit hard to assess what this data means both for us and our docs results, and more broadly how it is likely to impact our users.
From one perspective results are largely unchanged for many of these queries, and while they may be subtly different, we may not have complained about these results if they had been presented to us out of context.
On the other hand, several of these queries (page and sidebar) seem to suffer from much worse results after the 1.3 update. In both those cases, only setting termFrequency to 0 resulted in a more resonable result. However, there are other more marginal queries like component and installation where termFrequency: 0 seems to slightly worsen results. It’s a bit hard to know whether essentially disabling frequency weighting like this would be an extreme decision or not.
Notes
- Like @HiDeoo did in Update
pagefindto version1.1.0#1750, I also experimented withtermSaturation, but it appeared to have a near-zero impact. A few of the results shifted up/down one place, but nothing significant. Even extreme values liketermSaturation: 0did not budge the needle on some of the problematic results.
Description
pagefindto version1.1.0#1750 and some of the bundle size changes, but wanted to put this up as a PR before I forget about the logging level feature.