JavaScript Rendering

SpiderPro Pro includes a full Chromium engine for JavaScript rendering.

When to Use JavaScript Rendering

Enable JavaScript rendering when crawling:

• React applications
• Vue.js sites
• Angular projects
• Next.js with client-side rendering
• Any site with dynamic content loading

How It Works

1. SpiderPro loads the page in a headless Chromium browser
2. Waits for JavaScript to execute
3. Captures the fully rendered DOM
4. Extracts SEO elements from rendered content

Configuration

Enable JavaScript

In Settings > Crawl, toggle JavaScript Rendering ON.

Wait Time

Time to wait for JavaScript execution (default: 3 seconds).

Increase for:

• Slow-loading sites
• Heavy JavaScript applications
• API-dependent content

AJAX Mode

Wait for all network requests to complete before capturing.

TIP

Use AJAX mode for sites that load data from APIs.

Capture Options

Rendered HTML

Save the fully rendered HTML for each page.

Useful for:

• Debugging rendering issues
• Comparing raw vs rendered content
• Archive purposes

Screenshots

Capture visual screenshots of each page.

Performance Considerations

JavaScript rendering is slower than standard crawling:

Mode	Speed	Resource Usage
Standard	~50 pages/min	Low
JavaScript	~10 pages/min	High

Tips to Optimize

1. Use filters - Only render pages that need it
2. Reduce wait time - If content loads quickly
3. Disable screenshots - Unless needed
4. Crawl in batches - Break large sites into sections

Troubleshooting

Content not rendering

• Increase wait time
• Enable AJAX mode
• Check for JavaScript errors in console

Crawl is too slow

• Reduce concurrent requests to 1-2
• Disable screenshots
• Use faster hardware

Memory issues

• Close other applications
• Reduce max pages per session
• Enable "Low memory mode" in settings