You are helping me build the following AI agent workflow. ## Goal Web Data Extraction with Bright Data: Submit URLs to automatically scrape web content via Bright Data, analyze it with Google Gemini, and save the results to disk. ## Specification - What it does: Submit URLs to automatically scrape web content via Bright Data, then have Google Gemini analyze the data so you can save clean results directly to your disk without manual cleaning or parsing. - Trigger: Run manually (Manual ยท on demand) - Autonomy: Fully hands-off - Expected setup effort: a few focused days - Tools/services involved: - Bright Data MCP Server: Web scraping and data extraction - Google Gemini Chat Model: AI processing of extracted content - HTTP Request: Sending webhook notifications ## Known pitfalls, handle each one explicitly in your implementation 1. Webhook endpoints may reject requests without proper authentication headers, so configure your server to accept Bearer tokens. 2. Large HTML responses can exceed token limits for the Gemini model, so implement a chunking strategy before sending data to the AI agent. 3. Dynamic JavaScript-rendered pages may return empty content if Bright Data does not use headless browsing, so enable the headless browser option in your scraper settings. ## Reference implementation https://n8n.io/workflows/3778 (workflow template) Fetch and inspect this before building. If it matches my stack, adapt it; if not, rebuild the pattern with my tools. ## Process requirements 1. Before building: ask me which of the listed tools I actually use and what my platform is (n8n / Make / code / other). Do not assume. 2. Adapt the pattern to my answers; do not force the reference stack. 3. Address every pitfall above; tell me how you handled each. 4. Provide a test plan I can run before letting this touch real data. 5. Ask before any step that sends messages, modifies data, or spends money. Source: https://usecasesforagents.com/use-case/automated-web-data-extraction-with-bright-data/ via usecasesforagents.com