Southerncharms Ivy Complete Siteriprar Better

| Step | UI Element | Description | |------|------------|-------------| | 1 | URL Input (TextField) | Real‑time validation (https:// + domain). | | 2 | Destination Picker (File‑system dialog via Electron or “Download to server” toggle) | | 3 | Options (Accordion)
• Concurrency Slider
• Filter Builder (URL regex, MIME checkboxes, size range)
• Retry/Timeout fields | | 4 | Summary + Confirm | Shows estimated # of requests (

  "$id": "https://southerncharms.io/schemas/job-options.json",
  "type": "object",
  "properties": 
    "concurrency":  "type": "integer", "minimum": 1, "maximum": 32, "default": 8 ,
    "retryLimit":  "type": "integer", "minimum": 0, "default": 5 ,
    "filters": 
      "type": "object",
      "properties": 
        "urlRegex":  "type": "string", "format": "regex" ,
        "mimeTypes":  "type": "array", "items":  "type": "string"  ,
        "minSize":  "type": "integer", "minimum": 0 ,
        "maxSize":  "type": "integer", "minimum": 0 ,
        "dateRange": 
          "type": "object",
          "properties": 
            "from":  "type": "string", "format": "date-time" ,
            "to":    "type": "string", "format": "date-time" 
          ,
          "required": ["from","to"]
,
      "additionalProperties": false
    ,
    "checksumAlgo":  "type": "string", "enum": ["md5","sha1","sha256"], "default":"sha256" 
  ,
  "required": ["concurrency"]

┌───────────────────────────────────────┐
│            Front‑End (React)           │
│  ──► Job Builder (Wizard)               │
│  ──► Job List / History                  │
│  ──► Live Progress Dashboard            │
└───────────────▲───────────────▲─────────┘
                │               │
                │   REST API    │
                ▼               ▼
┌───────────────────────────────────────┐
│               Back‑End (Node.js)       │
│  ──► Express Router (REST)              │
│  ──► Scheduler (BullMQ / Redis)         │
│  ──► Workers (Downloader, Parser)       │
│  ──► Storage Service (S3 / local FS)    │
│  ──► DB (PostgreSQL) – Jobs, Logs, …   │
└───────────────▲───────────────▲─────────┘
                │               │
                │   Message Bus │
                ▼               ▼
┌───────────────────────────────────────┐
│               Workers (Node)           │
│  ──► Parallel HTTP fetch (got/axios)   │
│  ──► Content‑type detection            │
│  ──► Checksum + deduplication           │
│  ──► Plugin hooks (parser, filter)      │
└───────────────────────────────────────────┘