More flexibility in the Content Summarization experiment

[dkotter]

What?

Instead of using a paragraph block variation, use a group block variation to output our summary. Add some basic styles to make a clear distinction between this section and the rest of the content.

Why?

At the moment the Content Summarization experiment outputs a summary within a paragraph block variation. This means if you want a summary that is multiple paragraphs or a bulleted list instead of paragraphs, you'd have to manually do that and then you lose the connection to the AI Summary block variation. This breaks styling and breaks any regeneration of the summary.

A better approach is taken in this PR where we use a group block variation instead, which allows you to have any number of blocks within it. So we output the summary as one or more paragraphs but then a user can convert those to list items, add extra paragraphs or add a heading to the section. And none of these changes lose that AI Summary block connection.

We could look to add additional settings in the future to output as list items instead of paragraphs but for now, this at least allows users to do this themselves.

Also adding in some basic styling, both in the editor and the front-end, to make a clear distinction between these summary section and the rest of the content. Kept this really basic so sites can override those styles easily with their own.

How?

• Create a Group block variation to replace our Paragraph block variation
• Update our system instructions a bit to support outputting multiple paragraphs and ensure our sanitization also supports this
• Add some basic styles and ensure this is loaded in the editor and front-end

Use of AI Tools

AI assistance: Yes
Tool(s): Claude Code
Model(s): Sonnet 4.6
Used for: Review various approaches (block variations, patterns, custom blocks) and putting a plan together once we settled on an approach. It executed on that plan and then refining and testing done by me

Testing Instructions

1. Checkout this PR and run npm i && npm run build
2. Enable the Content Summarization experiment
3. Ensure you have an AI Connector in place
4. Create a post or edit an existing post
5. Find the Generate Summary block in the sidebar and click it
6. Ensure a summary is added to the top of the post. It likely will have two paragraphs but can be more or less
7. Ensure this summary has basic styles
8. Save the post and view the front-end. Ensure styles show there
9. In the editor, click into the block and make some edits (add a heading, remove a paragraph, convert a paragraph to a list)
10. Ensure styles still show properly
11. Click the Regenerate Summary button and ensure that overwrites the block. You will lose any edits you made

Screenshots or screencast

Basic summary block	Summary block with human edits (heading, list items)

Changelog Entry

Added - Basic styles for the Content Summary block.
Changed - Update the Content Summary experiment to render the summary in a Group variation block instead of a Paragraph variation block.

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: dkotter <dkotter@git.wordpress.org>
Co-authored-by: jeffpaul <jeffpaul@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[codecov]

Codecov Report

❌ Patch coverage is 42.85714% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.30%. Comparing base (4736d6f) to head (cccd9e1).

Files with missing lines	Patch %	Lines
...des/Abilities/Summarization/system-instruction.php	33.33%	2 Missing ⚠️
...cludes/Experiments/Summarization/Summarization.php	33.33%	2 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##             develop     #510      +/-   ##
=============================================
+ Coverage      67.23%   67.30%   +0.07%     
- Complexity      1054     1055       +1     
=============================================
  Files             65       65              
  Lines           5024     5026       +2     
=============================================
+ Hits            3378     3383       +5     
+ Misses          1646     1643       -3     

Flag	Coverage Δ
unit	67.30% <42.85%> (+0.07%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[jeffpaul]

Works great, nice improvement!

Only tweak I'd make is updating the group block name to "Content Summary" and the description to "A generated summary of the post content."

[dkotter]

Only tweak I'd make is updating the group block name to "Content Summary" and the description to "A generated summary of the post content."

Updated both of these (as well as cleaned up some outdated and/or unneeded docs)

[jeffpaul]

In testing with mostly lorem ipsum post content, I'm getting some content in the summary that's probably not useful via OAI:

Perhaps we tweak the prompt there to avoid including post meta in the summary?

[dkotter]

In testing with mostly lorem ipsum post content, I'm getting some content in the summary that's probably not useful via OAI:

Perhaps we tweak the prompt there to avoid including post meta in the summary?

I've tweaked it a bit and seems better to me now though still not always perfect, open to any suggestions there, noting we can't fully control what the LLM decides to return.

[jeffpaul]

Seems to work better with copy output now, thanks!