A price tracker
Watch a page, alert on change, without hammering anyone's site.
Build time ~4 hrs
Checking a page daily and telling you when a number changes is a small, genuinely useful automation — and a good exercise in doing scraping politely.
1. Check politely and infrequently
Prices don't change every minute. Once or twice a day is plenty, and it's the difference between a reasonable tool and getting blocked.
Build a price tracker. Watches <n> product pages, checks <how often>. - Check robots.txt and honour it. Real User-Agent with contact info. - One request at a time, with a delay. Exponential backoff on errors, and stop entirely after repeated 403 or 429. - Extract the price with a selector per site, stored in config so I can fix one without touching code. - If the selector finds nothing, that's an ERROR to alert me about — never a price of zero, and never silently skipped. - Store price history, not just the current price. - Alert on: price drops below a threshold, any change, or back in stock. Configurable per item.
2. A missing selector is an error
The most important rule. Sites change their markup, your selector breaks, and the naive implementation records nothing and tells you nothing. Weeks later you notice you never got an alert.
An extraction that fails must be loud. Alert on it exactly as you'd alert on a price drop.
3. Validate what you extracted
A price should parse as a number, be within an order of magnitude of the last recorded value, and be positive. A sudden change from £40 to £4 is more likely a parsing error than a sale — flag it for a look rather than firing an alert.
4. Store the raw HTML
At least for the most recent check. When extraction breaks, you can fix the selector against the saved page rather than fetching repeatedly while you debug.
5. Alerts people read
Include the old price, the new price, the percentage change, and a direct link. An alert that just says "price changed" makes you go and look, which defeats the purpose.
Add a maximum alerts-per-day cap. A site that starts returning random values shouldn't be able to send you sixty messages overnight.