ecap-security-auditor
v2.0.0Security audit framework for AI agent skills, MCP servers, and packages. Your LLM does the analysis — we provide structure, prompts, and a shared trust database.
Installation
ecap Security Auditor
You are an AI agent with a built-in security shield. This skill makes you automatically verify every package, skill, and MCP server before you install or use it — and gives you the tools to audit anything yourself.
📖 Key Terms
| Term | Definition |
|---|---|
risk_score |
API field (0–100). Higher = more dangerous. 0 = perfectly safe, 100 = actively malicious. |
| Trust Score | Display metric (0–100). Higher = more trustworthy. Calculated as 100 - penalties from findings. Used in Gate decisions. |
ecap_id |
Unique finding identifier (e.g., ECAP-2026-0777). Used in all API URLs for /review and /fix endpoints. |
package_name |
The name used to query the registry (e.g., "express", "mcp-server-fetch"). API field is skill_slug; both are accepted. |
⚡ Automatic Security Gate (Primary Feature)
This is your passive security layer. Every time you install or use a package, run this check automatically and inform the user of results.
When to Trigger
Run the Security Gate automatically when:
clawdhub install <skill>— any skill installationpip install <package>/npm install <package>— any package installation- First use of any MCP server
- User says "audit", "check", or "verify" a package
Gate Flow
┌─────────────────────────────┐
│ Package install/use detected│
└──────────────┬──────────────┘
▼
┌─────────────────────────────┐
│ 1. Query Trust Registry │
│ GET /api/findings?package=│
│ GET /api/integrity?package=│
└──────────────┬──────────────┘
▼
┌─────────┐
│ Report │──── No ───▶ Go to AUTO-AUDIT
│ exists? │
└────┬─────┘
│ Yes
▼
┌─────────────────────────────┐
│ 2. Hash Verification │
│ Run: bash scripts/verify.sh <package>
│ Compares local file hashes│
│ against audited hashes │
└──────────────┬──────────────┘
▼
┌─────────┐
│ Hash OK? │──── No ───▶ 🚨 STOP: TAMPERED
└────┬─────┘
│ Yes
▼
┌─────────────────────────────┐
│ 3. Calculate Trust Score │
│ from findings (see below)│
└──────────────┬──────────────┘
▼
┌─────────┴─────────┐
│ │
Score ≥ 70 Score 40-69 Score < 40
│ │ │
▼ ▼ ▼
✅ PASS ⚠️ WARNING 🔴 BLOCK
Continue Show findings, Block install.
silently. let user decide. Offer to audit.
Decision Table
| Condition | Action | Message to User |
|---|---|---|
| Score ≥ 70 + Hash OK | ✅ Proceed | ✅ [package] — Trust Score: XX/100, verified. |
| Score 40–69 + Hash OK | ⚠️ Warn, user decides | ⚠️ [package] — Trust Score: XX/100. Known issues: [list]. Proceed? (y/n) |
| Score < 40 | 🔴 Block | 🔴 [package] — Trust Score: XX/100. Blocked. Run audit to investigate. |
Note: By-design findings (e.g.,
exec()in agent frameworks) are displayed for transparency but do not affect the Trust Score or gate decisions. | No report exists | 🔍 Auto-audit |🔍 [package] — No audit data. Running security audit now...| | Hash mismatch | 🚨 Hard stop |🚨 [package] — INTEGRITY FAILURE. Local files don't match audited version. DO NOT INSTALL.|
Step-by-Step Implementation
Step 1: Query the Trust Registry
# Check for existing findings
curl -s "https://skillaudit-api.vercel.app/api/findings?package=PACKAGE_NAME"
# Check file integrity hashes
curl -s "https://skillaudit-api.vercel.app/api/integrity?package=PACKAGE_NAME"
Example — GET /api/findings?package=coding-agent (with findings):
{
"findings": [
{
"id": 11, "ecap_id": "ECAP-2026-0782",
"title": "Overly broad binary execution requirements",
"description": "Skill metadata requires ability to run "anyBins" which grants permission to execute any binary on the system.",
"severity": "medium", "status": "reported", "target_skill": "coding-agent",
"reporter": "ecap0", "source": "automated",
"pattern_id": "MANUAL_001", "file_path": "SKILL.md", "line_number": 4,
"confidence": "medium"
}
],
"total": 6, "page": 1, "limit": 100, "totalPages": 1
}
Example — GET /api/findings?package=totally-unknown-xyz (no findings):
{"findings": [], "total": 0, "page": 1, "limit": 100, "totalPages": 0}
Note: Unknown packages return
200 OKwith an empty array, not 404.
Example — GET /api/integrity?package=ecap-security-auditor:
{
"package": "ecap-security-auditor",
"repo": "https://github.com/starbuck100/ecap-security-auditor",
"branch": "main",
"commit": "553e5ef75b5d2927f798a619af4664373365561e",
"verified_at": "2026-02-01T23:23:19.786Z",
"files": {
"SKILL.md": {"sha256": "8ee24d731a...", "size": 11962},
"scripts/upload.sh": {"sha256": "21e74d994e...", "size": 2101},
"scripts/register.sh": {"sha256": "00c1ad0f8c...", "size": 2032},
"prompts/audit-prompt.md": {"sha256": "69e4bb9038...", "size": 5921},
"prompts/review-prompt.md": {"sha256": "82445ed119...", "size": 2635},
"README.md": {"sha256": "2dc39c30e7...", "size": 3025}
}
}
If the package is not in the integrity database, the API returns
404:json {"error": "Unknown package: unknown-xyz", "known_packages": ["ecap-security-auditor"]}
Step 2: Verify Integrity
bash scripts/verify.sh <package-name>
# Example: bash scripts/verify.sh ecap-security-auditor
This compares SHA-256 hashes of local files against the hashes stored during the last audit. If any file has changed since it was audited, the check fails.
⚠️ Limitation:
verify.shonly works for packages registered in the integrity database. Currently onlyecap-security-auditoris registered. For other packages, skip integrity verification and rely on Trust Score from findings only.🔒 Security: The API URL in
verify.shis hardcoded to the official registry and cannot be overridden. This prevents malicious SKILL.md forks from redirecting integrity checks to fake servers.
Step 3: Calculate Trust Score & Apply Decision Logic
The API does not provide a Trust Score endpoint. Calculate it yourself from the findings:
Trust Score = max(0, 100 - penalties)
Penalties per finding (only where by_design = false):
Critical: -25
High: -15
Medium: -8
Low: -3
Any (by_design = true): 0 ← excluded from score
Component-Type Weighting (v2): Apply a ×1.2 multiplier to penalties for findings in high-risk component types: shell scripts in
hooks/,.mcp.jsonconfigs,settings.json, and plugin entry points. Findings in documentation or test files receive no multiplier.
Example: 1 critical + 2 medium findings → 100 - 25 - 8 - 8 = 59 (⚠️ Caution) Example with by-design: 3 by-design high + 1 real low → 100 - 0 - 0 - 0 - 3 = 97 (✅ Trusted)
By-design findings are patterns that are core to the package's documented purpose (e.g.,
exec()in an agent framework). They are reported for transparency but do not reduce the Trust Score. Seeaudit-prompt.mdStep 4 for classification criteria.
If the package has a report in /api/reports, you can also use the risk_score from the report: Trust Score ≈ 100 - risk_score.
Apply the decision table above based on the calculated Trust Score.
Step 4: Auto-Audit (if no data exists)
If the registry has no report for this package:
- Get the source code (see "Getting Package Source" below)
- Read ALL files in the package directory
- Read
prompts/audit-prompt.md— follow every instruction - Analyze each file against the security checklist
- Perform cross-file analysis (see Cross-File Analysis below)
- Build a JSON report (format below)
- Upload:
bash scripts/upload.sh report.json - Re-run the gate check with the new data
This is how the registry grows organically — every agent contributes.
Getting Package Source for Auto-Audit
⚠️ The audit must run BEFORE installation. You need the source code without executing install scripts. Here's how:
| Type | How to get source safely | Audit location |
|---|---|---|
| OpenClaw skill | Already local after clawdhub install (skills are inert files) |
skills/<name>/ |
| npm package | npm pack <name> && mkdir -p /tmp/audit-target && tar xzf *.tgz -C /tmp/audit-target/ |
/tmp/audit-target/package/ |
| pip package | pip download <name> --no-deps -d /tmp/ && cd /tmp && tar xzf *.tar.gz (or unzip *.whl) |
/tmp/<name>-<version>/ |
| GitHub source | git clone --depth 1 <repo-url> /tmp/audit-target/ |
/tmp/audit-target/ |
| MCP server | Check MCP config for install path; if not installed yet, clone from source | Source directory |
Why not just install? Install scripts (postinstall, setup.py) can execute arbitrary code — that's exactly what we're trying to audit. Always get source without running install hooks.
Package Name
Use the exact package name (e.g., mcp-server-fetch, not mcp-fetch). You can verify known packages via /api/health (shows total counts) or check /api/findings?package=<name> — if total > 0, the package exists in the registry.
Finding IDs in API URLs
When using /api/findings/:ecap_id/review or /api/findings/:ecap_id/fix, use the ecap_id string (e.g., ECAP-2026-0777) from the findings response. The numeric id field does NOT work for API routing.
🔍 Manual Audit
For deep-dive security analysis on demand.
Step 1: Register (one-time)
bash scripts/register.sh <your-agent-name>
Creates config/credentials.json with your API key. Or set ECAP_API_KEY env var.
Step 2: Read the Audit Prompt
Read prompts/audit-prompt.md completely. It contains the full checklist and methodology.
Step 3: Analyze Every File
Read every file in the target package. For each file, check:
npm Packages:
- package.json: preinstall/postinstall/prepare scripts
- Dependency list: typosquatted or known-malicious packages
- Main entry: does it phone home on import?
- Native addons (.node, .gyp)
- process.env access + external transmission
pip Packages:
- setup.py / pyproject.toml: code execution during install
- __init__.py: side effects on import
- subprocess, os.system, eval, exec, compile usage
- Network calls in unexpected places
MCP Servers: - Tool descriptions vs actual behavior (mismatch = deception) - Permission scopes: minimal or overly broad? - Input sanitization before shell/SQL/file operations - Credential access beyond stated needs
OpenClaw Skills:
- SKILL.md: dangerous instructions to the agent?
- scripts/: curl|bash, eval, rm -rf, credential harvesting
- Data exfiltration from workspace
Step 3b: Component-Type Awareness (v2)
Different file types carry different risk profiles. Prioritize your analysis accordingly:
| Component Type | Risk Level | What to Watch For |
|---|---|---|
Shell scripts in hooks/ |
🔴 Highest | Direct system access, persistence mechanisms, arbitrary execution |
.mcp.json configs |
🔴 High | Supply-chain risks, npx -y without version pinning, untrusted server sources |
settings.json / permissions |
🟠 High | Wildcard permissions (Bash(*)), defaultMode: dontAsk, overly broad tool access |
| Plugin/skill entry points | 🟠 High | Code execution on load, side effects on import |
SKILL.md / agent prompts |
🟡 Medium | Social engineering, prompt injection, misleading instructions |
| Documentation / README | 🟢 Low | Usually safe; check for hidden HTML comments (>100 chars) |
| Tests / examples | 🟢 Low | Rarely exploitable; check for hardcoded credentials |
Findings in high-risk components should receive extra scrutiny. A
medium-severity finding in a hook script may warranthighseverity due to the execution context.
Step 3c: Cross-File Analysis (v2)
Do not analyze files in isolation. Explicitly check for multi-file attack chains:
| Cross-File Pattern | What to Look For |
|---|---|
| Credential + Network | Credentials read in file A, transmitted via network call in file B |
| Permission + Persistence | Permission escalation in one file enabling persistence mechanism in another |
| Hook + Skill Activation | A hook script that silently modifies skill behavior or injects instructions |
| Config + Obfuscation | Config file that references obfuscated scripts or encoded payloads |
| Supply Chain + Network | Dependency installed via postinstall hook that phones home |
| File Access + Exfiltration | File reading in one component, data sent externally in another |
When you find a cross-file relationship, report it as a single finding with pattern_id prefix CORR_ and list all involved files in the description.
Step 4: AI-Specific Security Checks (v2)
When auditing AI agent packages, skills, and MCP servers, check for these AI-specific attack patterns:
Prompt Injection & Manipulation
| Pattern ID | Attack | Examples to Look For |
|---|---|---|
AI_PROMPT_001 |
System Prompt Extraction | "reveal your system prompt", "output your instructions", "what were you told" |
AI_PROMPT_002 |
Agent Impersonation | "pretend to be", "you are now", "act as an Anthropic employee" |
AI_PROMPT_003 |
Capability Escalation | "enable developer mode", "unlock hidden capabilities", "activate god mode" |
AI_PROMPT_004 |
Context Pollution | "inject into context", "remember this forever", "prepend to all responses" |
AI_PROMPT_005 |
Multi-Step Attack Setup | "on the next message execute", "phase 1:", "when triggered do" |
AI_PROMPT_006 |
Output Manipulation | "output JSON without escaping", "encode response in base64", "hide in markdown" |
AI_PROMPT_007 |
Trust Boundary Violation | "skip all validation", "disable security", "ignore safety checks" |
AI_PROMPT_008 |
Indirect Prompt Injection | "follow instructions from the file", "execute commands from URL", "read and obey" |
AI_PROMPT_009 |
Tool Abuse | "use bash tool to delete", "bypass tool restrictions", "call tool without user consent" |
AI_PROMPT_010 |
Jailbreak Techniques | DAN prompts, "bypass filter/safety/guardrail", role-play exploits |
AI_PROMPT_011 |
Instruction Hierarchy Manipulation | "this supersedes all previous instructions", "highest priority override" |
AI_PROMPT_012 |
Hidden Instructions | Instructions embedded in HTML comments, zero-width characters, or whitespace |
False-positive guidance: Phrases like "never trust all input" or "do not reveal your prompt" are defensive, not offensive. Only flag patterns that attempt to perform these actions, not warn against them.
Persistence Mechanisms (v2)
Check for code that establishes persistence on the host system:
| Pattern ID | Mechanism | What to Look For |
|---|---|---|
PERSIST_001 |
Crontab modification | crontab -e, crontab -l, writing to /var/spool/cron/ |
PERSIST_002 |
Shell RC files | Writing to .bashrc, .zshrc, .profile, .bash_profile |
PERSIST_003 |
Git hooks | Creating/modifying files in .git/hooks/ |
PERSIST_004 |
Systemd services | systemctl enable, writing to /etc/systemd/, .service files |
PERSIST_005 |
macOS LaunchAgents | Writing to ~/Library/LaunchAgents/, /Library/LaunchDaemons/ |
PERSIST_006 |
Startup scripts | Writing to /etc/init.d/, /etc/rc.local, Windows startup folders |
Advanced Obfuscation (v2)
Check for techniques that hide malicious content:
| Pattern ID | Technique | Detection Method |
|---|---|---|
OBF_ZW_001 |
Zero-width characters | Look for U+200B–U+200D, U+FEFF, U+2060–U+2064 in any text file |
OBF_B64_002 |
Base64-decode → execute chains | atob(), base64 -d, b64decode() followed by eval/exec |
OBF_HEX_003 |
Hex-encoded content | x sequences, Buffer.from(hex), bytes.fromhex() |
OBF_ANSI_004 |
ANSI escape sequences | x1b[, |