Competitor Price Monitoring with Web Scraping
Daily agent that scrapes competitor prices from a master list and alerts you via Telegram whenever a price changes.
What it does
Get instant Telegram alerts whenever a competitor changes their prices for products on your master list. The agent automatically checks each item's current price against your Google Sheets data so you never miss a shift in the market.
Step by step
- Parse the master list from Google Sheets to extract product URLs and last known prices.
- Send an HTTP GET request to each product URL with standard headers to fetch the HTML content.
- Use a CSS selector or XPath expression to locate the price element within the scraped HTML.
- Compare the newly extracted price string against the stored value in Google Sheets.
Where the LLM does the work
- Generate robust CSS selectors that can adapt to minor changes in competitor website layouts.
- Create natural language alert messages for Telegram that clearly explain significant price drops or hikes.
- Implement error handling logic to gracefully manage pages with dynamic content or anti-bot measures.
Watch out for
Competitor sites often block automated requests, so you must rotate user agents and add random delays between calls.
Price formats vary across sites like '$10.99' versus '10,99 EUR', so normalize strings before numerical comparison.
Dynamic JavaScript rendering may hide the price in the initial HTML response, requiring a headless browser solution.
Tools that fit
HTTP Request API Fetch web pages for scraping
HTML Service Extract price data via CSS selectors
The agent brief
Everything your agent needs, including the gotchas. Copy it and go.
You are helping me build the following AI agent workflow. ## Goal Competitor Price Monitoring with Web Scraping: Daily agent that scrapes competitor prices from a master list and alerts you via Telegram whenever a price changes. ## Specification - What it does: Get instant Telegram alerts whenever a competitor changes their prices for products on your master list. The agent automatically checks each item's current price against your Google Sheets data so you never miss a shift in the market. - Trigger: On a schedule (Scheduled · daily 8:00 AM) - Autonomy: Fully hands-off - Expected setup effort: under an hour - Tools/services involved: - Google Sheets: Store product data and log price history - HTTP Request: Fetch web pages for scraping - HTML: Extract price data via CSS selectors - Telegram: Send alerts for price changes ## Known pitfalls, handle each one explicitly in your implementation 1. Competitor sites often block automated requests, so you must rotate user agents and add random delays between calls. 2. Price formats vary across sites like '$10.99' versus '10,99 EUR', so normalize strings before numerical comparison. 3. Dynamic JavaScript rendering may hide the price in the initial HTML response, requiring a headless browser solution. ## Reference implementation https://n8n.io/workflows/4640 (workflow template) Fetch and inspect this before building. If it matches my stack, adapt it; if not, rebuild the pattern with my tools. ## Process requirements 1. Before building: ask me which of the listed tools I actually use and what my platform is (n8n / Make / code / other). Do not assume. 2. Adapt the pattern to my answers; do not force the reference stack. 3. Address every pitfall above; tell me how you handled each. 4. Provide a test plan I can run before letting this touch real data. 5. Ask before any step that sends messages, modifies data, or spends money. Source: https://usecasesforagents.com/use-case/competitor-price-monitoring-with-web-scraping/ via usecasesforagents.com
Frequently asked questions
Can I use a different Spreadsheet than Google Sheets?
Yes. Google Sheets is only the example spreadsheet in this recipe. The same flow works with Airtable, Baserow and Excel. Swap the spreadsheet connection and keep the rest of the setup as written.
Want this running in your business?
This is what I do. I design and build AI agents like this one, and keep them running. If you want it set up for your team instead of doing it yourself, get in touch.
Who it's for
Also fits E-commerce Operator.
Founders and execs get the same output without having to ask someone else to run it.