# Web Scraper App Requirements

## 1) Objective
Build a lightweight scraping tool that:
- accepts a target URL,
- allows users to define what data to capture,
- supports targeting by `id`, `class`, `css`, or `xpath`,
- previews page layout,
- shows hovered element HTML,
- returns extracted values in a structured format.

## 2) Tech Stack
- Frontend: HTML + JavaScript + Bootstrap 5 + Font Awesome
- Backend: PHP 8+
- Endpoint: `api/scrape-page.php`

## 3) Functional Requirements
1. URL input with validation (`http/https`).
2. Rule builder fields:
   - field name
   - selector type (`id`, `class`, `css`, `xpath`)
   - selector value
   - extract mode (`text`, `html`, `attr`)
   - attribute name (when `attr` is selected)
   - multiple-match toggle
3. Add/remove rules dynamically.
4. Fetch and scrape button.
5. Preview-only mode.
6. JSON result panel for extracted values.
7. Preview panel showing fetched page layout.
8. Hover in preview displays:
   - derived selector path
   - element HTML snippet.

## 4) Backend Requirements
1. Accept JSON POST body:
   - `url` (string)
   - `rules` (array)
   - `includeHtml` (bool)
2. Fetch page with cURL (timeouts + redirects).
3. Parse HTML with `DOMDocument` + `DOMXPath`.
4. Resolve selector types:
   - id/class directly
   - xpath directly
   - css via simple converter (limited).
5. Return extraction results as JSON.
6. Optionally return sanitized HTML for preview.

## 5) Security / Reliability
- Only `http`/`https` URLs allowed.
- Timeout and max size limits.
- Graceful errors with user-facing messages.
- Remove script tags from preview HTML response.
- Do not execute arbitrary server-side code.

## 6) Non-Functional Requirements
- Single-page UI, no build step.
- Responsive layout for desktop and tablet.
- Clear status messages.
- Easy extension for additional selector strategies.

## 7) Known Limits (Prototype)
- CSS selector parsing is intentionally limited on backend.
- Some websites block scraping via anti-bot rules.
- Authenticated pages are not supported.