SkillHub

search-cluster

v3.5.1

Aggregated search aggregator using Google CSE, GNews RSS, Wikipedia, Reddit, and Scrapling.

Sourced from ClawHub, Authored by azzar budiyanto

Installation

Please help me install the skill `search-cluster` from SkillHub official store. npx skills add 1999AZZAR/search-cluster

Search Cluster (Industrial Standard v3.1)

A multi-provider search aggregator designed for high-availability and security.

Installation

The scrapling provider requires a dedicated virtual environment. 1. Create a venv: python3 -m venv venv/scrapling 2. Install scrapling: venv/scrapling/bin/pip install scrapling 3. Provide the path to the venv binary in SCRAPLING_PYTHON_PATH.

Security Posture

  • Subprocess Isolation: Query inputs are passed as arguments to stealth_fetch.py.
  • Strict TLS: Mandatory SSL verification on all providers.
  • Sanitization: Integrated native internal scrubber (Path Neutral).

Requirements and Environment

Declare these variables in your environment or vault:

Variable Requirement Description
GOOGLE_API_KEY Optional API Key for Google Custom Search.
GOOGLE_CSE_ID Optional Search Engine ID for Google CSE.
SCRAPLING_PYTHON_PATH Optional Path to the scrapling venv python binary.
REDIS_HOST Optional Host for result caching.
REDIS_PORT Optional Port for result caching (Default: 6379).
SEARCH_USER_AGENT Optional Custom User-Agent string.

Providers

  • google: Official Google Custom Search.
  • wiki: Wikipedia OpenSearch API.
  • reddit: Reddit JSON search API.
  • gnews: Google News RSS aggregator.
  • scrapling: Headless stealth scraping (via DuckDuckGo).

Included Scripts

  • scripts/search-cluster.py: Main entry point.
  • scripts/stealth_fetch.py: Scrapling fetcher (REQUIRED for scrapling provider).

Workflow

  1. Execute: scripts/search-cluster.py all ""
  2. Output is structured JSON with source, title, link, and sanitized snippet.