CI Healer started as a weekend experiment. The premise was simple: what if an AI agent could watch your GitHub repositories, detect CI failures, and open a pull request with the fix — without you ever touching a terminal?
Four days later, it was live. Here's exactly how I built it.
The Problem I Was Actually Solving
I've lost track of how many hours I've spent reading CI logs. Not learning from them — just reading them. Finding the one line that says "ModuleNotFoundError: No module named 'xyz'" in 800 lines of build output, then doing the obvious fix, then waiting four minutes for CI to re-run to confirm.
That workflow is broken. The information is all there — the log, the diff, the repository context. It just needs to be assembled by something that can process it faster than a human reading green text.
The Architecture
GitHub Webhook → FastAPI → LangGraph Agent Graph → GitHub API (PR)
↓
PostgreSQL (job log)
↓
Next.js Dashboard (live stream)
The whole system runs as three services orchestrated by Docker Compose:
- FastAPI backend — receives webhooks, manages job state, runs the LangGraph agent
- Next.js frontend — live dashboard that streams agent decisions in real time
- PostgreSQL — job history, repository config, audit log
Why LangGraph Instead of a Simple LLM Call
The naive approach is obvious: take the CI log, throw it at GPT-4, ask it for a fix. I tried this first. It produces plausible-looking patches that frequently don't compile, address the wrong root cause, or modify too many files.
LangGraph lets you build a stateful directed graph where each node has a single responsibility. My graph has five nodes:
from langgraph.graph import StateGraph, END
from typing import TypedDict, Annotated
import operator
class RepairState(TypedDict):
repo_url: str
branch: str
log: str
error_classification: str
patch: str
test_result: str
iterations: int
builder = StateGraph(RepairState)
builder.add_node("classify_error", classify_error)
builder.add_node("clone_and_reproduce", clone_and_reproduce)
builder.add_node("generate_patch", generate_patch)
builder.add_node("validate_patch", validate_patch)
builder.add_node("open_pr", open_pr)
builder.add_edge("classify_error", "clone_and_reproduce")
builder.add_edge("clone_and_reproduce", "generate_patch")
# Conditional: if patch passes validation, open PR. If not, retry up to 3 times.
builder.add_conditional_edges(
"validate_patch",
route_after_validation,
{
"success": "open_pr",
"retry": "generate_patch",
"give_up": END,
}
)
The conditional edge after validation is what makes this actually reliable. If the first patch fails to fix the issue (determined by running the tests locally in a Docker container), the agent loops back to generate_patch with the test output added to the context — so it has more information on the second attempt.
Three attempts maximum. If it can't fix it in three tries, it creates a GitHub issue instead of a PR, with the analysis and failed patches attached for context.
The Classification Step
Before writing any code, the agent classifies the failure type:
async def classify_error(state: RepairState) -> dict:
prompt = f"""
Analyse this CI build log and classify the failure:
Log:
{state["log"][-3000:]} # Last 3000 chars — where failures usually appear
Categories:
- DEPENDENCY_CONFLICT: version incompatibility between packages
- TEST_FAILURE: test assertion failed (not a build error)
- TYPE_ERROR: TypeScript/mypy type checking failure
- IMPORT_ERROR: missing module or incorrect import path
- CONFIG_ERROR: misconfigured CI yaml or environment variable
- BUILD_ERROR: compilation failure
- UNKNOWN: cannot be classified
Return JSON: {{"category": "...", "root_cause_summary": "...", "affected_files": [...]}}
"""
result = await llm.ainvoke([HumanMessage(content=prompt)])
parsed = json.loads(result.content)
return {"error_classification": parsed["category"], ...}
The category determines how subsequent nodes approach the fix. A DEPENDENCY_CONFLICT triggers a different patch strategy than a TYPE_ERROR. Getting this classification right is the single biggest factor in patch quality.
Reproducing the Error Locally
The most important step that most "AI code fix" demos skip: actually running the failing code before generating a patch.
async def clone_and_reproduce(state: RepairState) -> dict:
with tempfile.TemporaryDirectory() as tmpdir:
# Clone at the failing commit
subprocess.run(
["git", "clone", "--depth=1", state["repo_url"], tmpdir],
check=True, capture_output=True
)
# Run the failing step in a container matching CI
result = subprocess.run(
["docker", "run", "--rm", "-v", f"{tmpdir}:/app", "-w", "/app",
"python:3.12-slim", "sh", "-c",
"pip install -r requirements.txt && pytest --tb=short 2>&1"],
capture_output=True, text=True, timeout=120
)
return {
"reproduction_output": result.stdout + result.stderr,
"reproduction_successful": result.returncode != 0 # We want it to fail
}
Running the actual tests gives the patch generator concrete test output — not just the CI log (which is often truncated or formatted differently), but the actual error traceback from running the same code in the same environment.
Generating the Patch
async def generate_patch(state: RepairState) -> dict:
# Read relevant source files (guided by error_classification + affected_files)
context_files = await read_relevant_files(state)
prompt = f"""
You are fixing a CI failure. Generate a minimal unified diff patch.
Error category: {state["error_classification"]}
Test output showing the failure:
{state["reproduction_output"]}
Previous patch attempts that didn't work:
{state.get("failed_attempts", "None")}
Relevant source files:
{context_files}
Rules:
- Generate a unified diff (--- a/file.py / +++ b/file.py format)
- Change as little as possible
- Do not add features or refactor — only fix the failure
- Do not add comments explaining what you changed
"""
result = await llm.ainvoke([HumanMessage(content=prompt)])
return {"patch": result.content}
The constraint "change as little as possible" is critical. Without it, LLMs tend to "improve" code while fixing bugs — renaming variables, adding type hints, restructuring logic — which makes the diff harder to review and introduces new surface area for bugs.
The GitHub PR
If the patch passes local validation:
async def open_pr(state: RepairState) -> dict:
g = Github(settings.GITHUB_TOKEN)
repo = g.get_repo(state["repo_full_name"])
branch_name = f"ci-healer/fix-{state['job_id'][:8]}"
# Apply patch to a new branch
base_sha = repo.get_branch(repo.default_branch).commit.sha
repo.create_git_ref(f"refs/heads/{branch_name}", sha=base_sha)
apply_patch_to_branch(repo, branch_name, state["patch"])
# Open PR with structured description
pr = repo.create_pull(
title=f"fix: resolve {state['error_classification'].lower().replace('_', ' ')}",
body=f"""## 🤖 CI Healer Auto-Fix
**Failure type:** `{state["error_classification"]}`
**Root cause:** {state["root_cause_summary"]}
**How this was fixed:** {state["patch_explanation"]}
---
*Generated by [CI Healer](https://ci-healer.rahulkapoor.dev) after {state["iterations"]} attempt(s).*
*Verify by reviewing the diff carefully before merging.*
""",
head=branch_name,
base=repo.default_branch,
)
return {"pr_url": pr.html_url}
The Live Dashboard
The Next.js frontend connects to a Server-Sent Events (SSE) endpoint on FastAPI:
// app/jobs/[id]/page.tsx
export default function JobPage({ params }: { params: { id: string } }) {
const [events, setEvents] = useState<AgentEvent[]>([]);
useEffect(() => {
const es = new EventSource(`/api/jobs/${params.id}/stream`);
es.onmessage = (e) => {
const event = JSON.parse(e.data);
setEvents((prev) => [...prev, event]);
};
return () => es.close();
}, [params.id]);
return <EventTimeline events={events} />;
}
Every agent decision — which node was executed, what the classification returned, whether the patch passed tests — is streamed to the dashboard in real time. Users can watch the agent think through the problem.
What I'd Do Differently
Use a proper vector database for codebase context. Currently I do keyword-based file selection. For large repositories, semantic search over embedded code would find relevant files much more reliably.
Add a confidence threshold. Some failures are genuinely ambiguous. I'd add a confidence score to the classification step and skip to human escalation when confidence is below 0.7.
Parallel patch generation. For complex failures, generating three different patch strategies simultaneously and running them in parallel would dramatically improve the hit rate.
The Lesson
Agentic systems are not about making LLMs smarter — they're about giving LLMs the right structure to apply their capabilities reliably. A single "fix this bug" prompt fails 40% of the time. A structured graph that classifies, reproduces, patches, validates, and retries fails 8% of the time on the same bugs.
The intelligence is in the scaffolding.