Configuration
The URL field supports Insert Variable. Below that, the settings sheet lets you choose which output types to return: HTML, Markdown, Links, and Subpages.
Use Crawl Mode to control how Fetch Hive retrieves the page:
- Preferred (the default for new steps) tries a live crawl first, then falls back to cache.
- Always always uses a live crawl.
- Fallback uses cache first, then crawls if needed.
- Never only uses cache, so it fails on pages that were never stored. Steps created before this default changed keep the mode they were saved with.
Output
Click Run in the step header to test the step. Fetch Hive shows the scrape result in Output after the run completes. Use the variable picker in a later step to insert the exact output path available for that run. The base reference is:Example
Add Website Scrape from the Research group in Search steps…. Set Name to something likeScrape product page.
Paste the page into URL. If the URL comes from an earlier workflow step, click Insert Variable and add that reference.
Turn on the outputs you need. For example, enable Markdown for clean content, Links for extracted links, and Subpages if you want the step to read more of the site than the main page. Set Subpage Count (1 to 5) and, optionally, Subpage Focus with words like about, services, pricing so the pages you care about are read first. Each subpage adds to the step’s cost.
Choose a Crawl Mode, then set Max Characters, Max Retries, and Timeout (ms) for the run.
If you need a visual capture, turn Screenshot on and choose Viewport or Full Page in Screenshot Type.
Click Run and review the scraped result in Output before sending it to later workflow steps.
Notes
- The returned result depends on which output toggles you enable, so inspect the variable picker after a run if you need exact field names.
- The editor shows a warning for direct LinkedIn URLs. LinkedIn URLs are not supported for scraping.
- Use Markdown when you want cleaner page content, HTML when you need raw markup, and Links when you only need extracted URLs.
- If the page can’t be read the first time, the step makes a second attempt with a backup reader, using the same options (output formats, links, subpages, timeout, character limit). If both attempts fail, the error shows why each one failed.
- Crawl Mode decides whether the page is read live or from a stored copy. Never only uses stored copies and fails on pages that were never stored, so choose Preferred (live first, stored copy if the live read fails) or Always for pages that change or that you haven’t scraped before.
- When this step is inside an Iteration, set the Iteration’s When the step fails to Continue because some sites block. A scrape failure still stops that loop item; Continue on the Iteration is what keeps the rest of the workflow running.