You are helping me build the following AI agent workflow.

## Goal
Web Scraping And Data Enrichment: Triggers a background web crawl via Scrapyd, then parses the JSONL output to deduplicate items and extract structured product.

## Specification
- What it does: You get clean, structured product details with part numbers and prices automatically extracted from web pages. The process runs a background crawl via Scrapyd, removes duplicate entries, and delivers the final JSON results ready for your use.
- Trigger: Run manually (Manual · on demand)
- Autonomy: Fully hands-off
- Expected setup effort: about an afternoon
- Tools/services involved:
  - Scrapyd: Job scheduling and status tracking for web spiders
  - n8n: Workflow orchestration using HTTP, Wait, Code, and Aggregate nodes
  - Code: Data enrichment including parsing JSONL, deduplication, and normalization

## Known pitfalls, handle each one explicitly in your implementation
1. Scrapyd may return empty responses if jobs fail silently so you must check the HTTP status code and job logs before processing data.
2. JSONL files can contain malformed lines that crash the parser so wrap your JSON parsing in try-catch blocks to skip invalid entries.
3. n8n polling intervals might overwhelm Scrapyd if set too aggressively so add a random jitter or fixed delay between status checks.

## Reference implementation
https://n8n.io/workflows/8552 (workflow template)
Fetch and inspect this before building. If it matches my stack, adapt it;
if not, rebuild the pattern with my tools.

## Process requirements
1. Before building: ask me which of the listed tools I actually use and
   what my platform is (n8n / Make / code / other). Do not assume.
2. Adapt the pattern to my answers; do not force the reference stack.
3. Address every pitfall above; tell me how you handled each.
4. Provide a test plan I can run before letting this touch real data.
5. Ask before any step that sends messages, modifies data, or spends money.

Source: https://usecasesforagents.com/use-case/automated-web-scraping-and-data-enrichment/ via usecasesforagents.com