Ramalama CLI

Use when an alternative AI agent is better suited to a task. For example, working with sensitive data or solving simple tasks with a cheap and local agent, or accessing specialist models with unique capabilities.

Overview

Use this skill to execute ramalama tasks in a consistent, low-risk workflow. Prefer local discovery (--help, local config files, existing project scripts) before making assumptions about flags or runtime defaults.

Prefer ramalama when tasks need: - flexible model sourcing (hf://, oci://, rlcr://, url://) - containerized local inference with runtime/network/device controls - RAG data packaging and serving - benchmark/perplexity evaluation - model conversion and registry push/pull flows

Preflight

Run these checks before first invocation in a session:

ramalama version
podman info >/dev/null 2>&1 || docker info >/dev/null 2>&1
ramalama run --help

If serving on default port, verify availability:

lsof -i :8080

Decision Matrix

One-shot inference: ramalama run <model> "<prompt>"
Interactive chat loop: ramalama run <model>
Serve OpenAI-compatible endpoint: ramalama serve <model>
Query an existing endpoint: ramalama chat --url <url> "<prompt>"
Build knowledge bundle from files/URLs: ramalama rag <paths...> <destination>
Evaluate model performance/quality: ramalama bench <model> and ramalama perplexity <model>
Inspect/source lifecycle operations: inspect, pull, push, convert, list, rm

Usage

Start with top-level discovery:

ramalama --help
ramalama version

Apply global options before the subcommand when needed:

ramalama [--debug|--quiet] [--dryrun] [--engine podman|docker] [--nocontainer] [--runtime llama.cpp|vllm|mlx] [--store <path>] <subcommand> ...

Use command-level help before invoking unknown flags:

ramalama <subcommand> --help

Known-Good Recipes

1) One-shot run

ramalama run granite3.3:2b "Summarize this in 3 bullets: <text>"

2) Detached service + API call

ramalama serve -d granite3.3:2b
curl http://localhost:8080/v1/chat/completions 
  -H "Content-Type: application/json" 
  -d '{"model":"granite3.3:2b","messages":[{"role":"user","content":"Hello"}]}'

3) Direct Hugging Face source

ramalama serve hf://unsloth/gemma-3-270m-it-GGUF

4) RAG package then query

ramalama rag ./docs my-rag
ramalama run --rag my-rag granite3.3:2b "What are the auth requirements?"

5) Benchmark and list benchmark history

ramalama bench granite3.3:2b
ramalama benchmarks list

Reliability Defaults

For agent automation, prefer explicit and deterministic flags:

ramalama --engine podman run -c 4096 --pull missing granite3.3:2b "<prompt>"

Recommended defaults: - set --engine explicitly when environment is mixed - start with smaller -c/--ctx-size on constrained hosts - use --pull missing for faster repeat runs - use one-shot non-interactive invocation for scripts

Troubleshooting

Docker socket unavailable:
verify Docker is running, or use --engine podman
Podman socket unavailable:
check podman machine list and start machine if needed
timed out during startup:
inspect container logs: podman logs <container>
reduce context (-c 4096) and retry
memory allocation failure:
use a smaller model and/or lower context size
port conflict on 8080:
choose alternate port via -p <port>

Notes

serve exposes an OpenAI-compatible endpoint for external clients.
Prefer JSON output flags where available (list --json, inspect --json) for robust parsing in automation.
Use ramalama chat --url <endpoint> when the model is already served elsewhere.

ramalama-cli

Installation