The Scrape and summarize webpages with AI template, explained: how it pulls essay links and text, which selectors and URLs to swap, and how to send the summaries somewhere useful.
"Scrape and summarize webpages with AI" is an n8n team template with more than 390,000 views on n8n.io. The sample reads the three latest essays from Paul Graham's website and summarizes each one. The pattern applies to any site with a list page and article pages: competitor blogs, industry news, regulator announcements, vendor changelogs.
This template needs one service: OpenAI. The work is in the selectors and in what you do with the output.
A Manual Trigger starts the run. An HTTP Request node fetches paulgraham.com/articles.html, the page that lists every essay. An HTML node extracts the link of each essay with the CSS selector table table a, and a Split Out node turns that list into one item per link. A Limit node keeps the first three.
For each link, a second HTTP Request fetches the essay page. Two HTML nodes extract the page title and the body text (the selector body, with images and nav elements skipped). A Summarization Chain, backed by the OpenAI chat model gpt-4o-mini, a default data loader and a recursive text splitter, writes the summary. A Merge node and a Set node tidy the output into title plus summary.
The workflow ends there. Nothing sends the summaries anywhere, and you see them only in the n8n editor.
Replace the first HTTP Request URL with the page that lists the articles you want, such as a competitor's blog index.
The selector table table a fits the layout of one 1990s-style website. Open your target page in a browser, right-click a link, choose Inspect, and write a selector that matches only article links. Something like article h2 a or .post-title a is common. Run the node and check the output before you continue.
The second HTTP Request builds its address as a hard-coded domain plus the link: http://www.paulgraham.com/{{ $json.essay }}. Most modern sites return full URLs or root-relative paths. Remove the hard-coded domain if the extracted links are already complete. Keep and update it if they are relative.
The body selector grabs everything on the page, including menus and footers on a typical modern site. Narrow it to the article container, for example article or .entry-content, and add header,footer,aside to the skip list.
Raise or remove the Limit node when you want more than three items. Replace the Manual Trigger with a Schedule Trigger for a daily or weekly run.
The template re-reads the same pages each run. Add a Google Sheet or a data table with the URLs you processed, and filter them out before summarizing. Without it, a daily schedule sends you the same three summaries every morning.
Add a Slack, Gmail, or Notion node after the Set node, and map the title and summary into the message. A Slack channel named for the topic works well for a team.
The default summarization prompt is generic. Open the chain's options and add your own: the length, the audience, and the facts to pull out, such as prices, product launches, or policy changes.
Check the rules before you scrape. Read the site's robots.txt and terms before you automate requests. Add a Wait node between page fetches, run on a schedule instead of in bursts, and keep to public pages. Sites built with heavy JavaScript return empty HTML to a plain HTTP request. For those, look for an RSS feed or use a headless-browser service.
The page may build its content with JavaScript after loading, so the HTTP Request node sees an empty shell. Check for an RSS or sitemap URL, which often lists the same articles in plain XML.
Yes. Fetch the PDF with the HTTP Request node, add an Extract from File node set to PDF, and feed the text to the same Summarization Chain.
With gpt-4o-mini, summarizing a typical article costs a small fraction of a cent. A hundred articles a day stays in single-digit dollars per month. Check OpenAI's current pricing page, since rates change.
Tell us which workflow you need. We set up the credentials, adapt the template to your business, and document it so your team can run it.
Start a Project →