Sales teams describe their ideal customers in natural language: "fintech startups in Southeast Asia with 50–200 employees that raised Series A in the last 18 months." Translating that into a qualified lead list has historically meant hours of manual searching, filtering, and data enrichment.
At AventIQ, I built a workflow that does it in under 10 seconds.
The Problem With Traditional Lead Tools
Most lead generation tools are query builders. You select filters — industry, size, geography, funding stage — from dropdowns, and the tool executes a database search against its contact index. The UX is a form. The output is a CSV.
This works for simple requirements. It breaks down when requirements are nuanced: "companies that recently launched a mobile app and are likely hiring for growth marketing" isn't a filter dropdown. It's a reasoning problem.
LLMs are good at reasoning problems. The insight behind AI Lead Discovery is straightforward: use an LLM to interpret the requirement, structure it into a search query, enrich the results with research, and rank them by fit.
The Architecture
Natural Language Input
↓
Requirement Parser (GPT-4o)
↓
Structured Query Builder
↓
Lead Database Search (internal index + Apollo API)
↓
Enrichment Agent (web research + LinkedIn data)
↓
Fit Scoring (GPT-4o with scoring rubric)
↓
Ranked Lead List (structured JSON)
Each stage is a Lambda function. The pipeline is orchestrated by an AWS Step Functions state machine — which means each stage can fail and retry independently, the pipeline state is durable, and long-running enrichment steps don't hit Lambda's 15-minute limit because they're modelled as separate executions.
Stage 1: Requirement Parsing
The first LLM call converts free-text input into a structured search specification:
// src/leads/discovery/parser.ts
import OpenAI from "openai";
import { z } from "zod";
const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const RequirementSchema = z.object({
industries: z.array(z.string()),
geographies: z.array(z.string()),
companySize: z.object({
min: z.number().nullable(),
max: z.number().nullable(),
}),
fundingStage: z.array(z.string()).nullable(),
fundingRecency: z.string().nullable(), // "last 6 months", "2024", etc.
keywords: z.array(z.string()),
exclusions: z.array(z.string()),
signals: z.array(z.string()), // "recently hired CRO", "launched mobile app"
});
export async function parseRequirement(
naturalLanguage: string
): Promise<z.infer<typeof RequirementSchema>> {
const response = await client.beta.chat.completions.parse({
model: "gpt-4o-2024-08-06",
messages: [
{
role: "system",
content: `You are a B2B sales intelligence analyst. Parse the user's
lead requirement into a structured search specification. Be precise:
- Extract explicit criteria as stated
- Infer implicit criteria (e.g. "fast-growing startup" implies small company, recent funding)
- Identify buying signals mentioned
- Note any exclusion criteria`,
},
{ role: "user", content: naturalLanguage },
],
response_format: zodResponseFormat(RequirementSchema, "requirement"),
});
return response.choices[0].message.parsed!;
}
client.beta.chat.completions.parse with zodResponseFormat is OpenAI's structured output API — the model is constrained to emit valid JSON matching the Zod schema. No JSON parsing errors, no missing fields, no hallucinated properties.
The benefit over function calling: structured output guarantees the schema at the API level. The model can't return an array where you expect a string. The Zod validation after the API call is a safety net, not the primary validation mechanism.
Stage 2: Lead Database Search
The parsed specification is used to query our internal lead index and the Apollo.io API:
// src/leads/discovery/search.ts
export async function searchLeads(
spec: RequirementSpec,
limit = 100
): Promise<RawLead[]> {
// Build Apollo.io query from structured spec
const apolloQuery = {
organization_industry_tag_ids: await resolveIndustryIds(spec.industries),
organization_num_employees_ranges: buildEmployeeRanges(spec.companySize),
organization_locations: spec.geographies,
funding_stage: spec.fundingStage ?? undefined,
// ... other Apollo filters
};
const [apolloResults, internalResults] = await Promise.all([
fetchFromApollo(apolloQuery, limit),
queryInternalIndex(spec, limit),
]);
// Deduplicate by domain
return deduplicateByDomain([...apolloResults, ...internalResults]);
}
Parallel fetching from both sources halves the latency of this stage. Deduplication by domain (company website domain) ensures the same company doesn't appear twice if it's in both indexes.
Stage 3: Enrichment
Raw leads from a database contain basic firmographic data — name, industry, size, location. Enrichment adds the signals that make leads actually useful:
// src/leads/discovery/enrichment.ts
export async function enrichLead(lead: RawLead): Promise<EnrichedLead> {
// Parallel research — don't wait for one to finish before starting the next
const [newsResult, linkedInResult, techStackResult] = await Promise.allSettled([
researchRecentNews(lead.domain), // Recent press mentions, product launches
fetchLinkedInSignals(lead.domain), // Recent hires, job postings
detectTechStack(lead.domain), // BuiltWith/Wappalyzer API
]);
const news = newsResult.status === "fulfilled" ? newsResult.value : [];
const linkedIn = linkedInResult.status === "fulfilled" ? linkedInResult.value : null;
const techStack = techStackResult.status === "fulfilled" ? techStackResult.value : [];
// Use LLM to synthesise the research into a structured signal summary
const signals = await synthesiseSignals({ lead, news, linkedIn, techStack });
return { ...lead, signals, enrichedAt: new Date().toISOString() };
}
async function synthesiseSignals(data: EnrichmentData): Promise<LeadSignals> {
const response = await openai.beta.chat.completions.parse({
model: "gpt-4o-mini", // Faster, cheaper — this runs per-lead
messages: [
{
role: "system",
content: "Extract structured buying signals from the research data. Focus on signals that indicate the company is likely to buy sales engagement software.",
},
{
role: "user",
content: JSON.stringify(data),
},
],
response_format: zodResponseFormat(LeadSignalsSchema, "signals"),
});
return response.choices[0].message.parsed!;
}
Using Promise.allSettled instead of Promise.all is deliberate. If the LinkedIn API is down or rate-limits the request, the enrichment continues with partial data rather than failing entirely. A partially enriched lead is more useful than no lead.
Using gpt-4o-mini for the per-lead signal synthesis (rather than gpt-4o) cuts both latency and cost. When you're synthesising data for 50 leads in a single discovery run, model selection has a meaningful cost impact.
Stage 4: Fit Scoring
The final stage ranks enriched leads by fit against the original requirement:
// src/leads/discovery/scoring.ts
export async function scoreLead(
lead: EnrichedLead,
originalRequirement: string,
parsedSpec: RequirementSpec
): Promise<ScoredLead> {
const response = await openai.beta.chat.completions.parse({
model: "gpt-4o",
messages: [
{
role: "system",
content: `You are a senior sales intelligence analyst. Score this lead's
fit against the requirement on a 0-100 scale.
Scoring rubric:
- Industry fit (30 points): Does this company operate in the target industries?
- Size fit (20 points): Is the company in the target size range?
- Signal relevance (30 points): Do the enrichment signals match what the buyer is looking for?
- Geographic fit (10 points): Is the company in the target geography?
- Timing (10 points): Are there signals of active buying intent?
Provide a score for each dimension and an overall score.`,
},
{
role: "user",
content: JSON.stringify({
requirement: originalRequirement,
parsedSpec,
lead: {
name: lead.name,
industry: lead.industry,
employees: lead.employees,
location: lead.location,
signals: lead.signals,
},
}),
},
],
response_format: zodResponseFormat(ScoringSchema, "score"),
});
const score = response.choices[0].message.parsed!;
return {
...lead,
fitScore: score.overall,
scoreBreakdown: score,
scoredAt: new Date().toISOString(),
};
}
The explicit rubric in the system prompt is essential. Without it, the model produces arbitrary scores with no interpretable basis. With it, scores are consistent across runs — the same lead scored against the same requirement will reliably produce similar scores.
The Company Lookalike Search
The lookalike search is a variant of the discovery flow with a different Stage 1:
// src/leads/lookalike/parser.ts
export async function parseLookalikeTarget(
companyDomain: string
): Promise<RequirementSpec> {
// Fetch the target company's profile
const targetProfile = await enrichCompany(companyDomain);
// Use the target's profile as the search specification
const response = await openai.beta.chat.completions.parse({
model: "gpt-4o",
messages: [
{
role: "system",
content: `Given a company profile, generate a search specification for
finding similar companies. Focus on the characteristics that make this
company similar to others that would benefit from the same sales engagement tools.`,
},
{
role: "user",
content: JSON.stringify(targetProfile),
},
],
response_format: zodResponseFormat(RequirementSchema, "spec"),
});
return response.choices[0].message.parsed!;
}
Once the lookalike spec is generated, it flows through the same search → enrichment → scoring pipeline. The user inputs one domain; they get 20 companies that look like it.
Frontend Data Fetching Optimisation
One thing I fixed beyond the backend: the leads dashboard was making three separate API calls on load — one for the lead list, one for pipeline stats, and one for recent activity. All three were loading independently, causing a cascading layout shift as data arrived at different times.
I consolidated them into a single endpoint that returns all dashboard data in one response:
// Before: 3 separate fetches, sequential waterfall
const leads = await fetchLeads();
const stats = await fetchStats();
const activity = await fetchActivity();
// After: parallel fetch from one endpoint
const { leads, stats, activity } = await fetchDashboardData();
// The endpoint itself runs the three queries in parallel
export async function getDashboardData(userId: string) {
const [leads, stats, activity] = await Promise.all([
db.lead.findMany({ where: { userId }, orderBy: { fitScore: "desc" }, take: 20 }),
db.lead.groupBy({ by: ["stage"], _count: true }),
db.activity.findMany({ where: { userId }, orderBy: { createdAt: "desc" }, take: 10 }),
]);
return { leads, stats, activity };
}
Dashboard load time dropped from ~1.8s to ~400ms. The user sees a fully populated dashboard on first render rather than watching three sections load independently.
What I'd Build Next
Vector search for semantic lead matching. The current scoring uses LLM reasoning on individual leads. Embedding lead profiles and requirements as vectors would enable much faster approximate matching on large indexes — scoring 1000 leads with embeddings takes milliseconds; scoring them with GPT-4o takes minutes.
Feedback loop from CRM outcomes. When a lead converts (or doesn't), feeding that signal back into the scoring model would improve its calibration over time. The leads that scored 85 but didn't convert are telling you something about the rubric.
Streaming results to the frontend. Currently the user sees a loading state for the full pipeline duration. Streaming results as each lead is scored — showing leads progressively, sorted by score — would make the UX feel dramatically faster even with the same backend latency.
The Lesson
AI Lead Discovery isn't magic — it's a structured pipeline where LLMs handle the steps that are hard to express as rules: interpreting natural language, synthesising disparate research signals, and applying nuanced judgement. Every other step — database queries, API calls, deduplication — is conventional engineering.
The design principle is to use LLMs only where they're needed. Stage 2 (database search) is a query execution — no LLM involved. Stage 3 (enrichment) is API calls — LLM only for the final synthesis step. Stage 4 (scoring) needs reasoning — LLM with an explicit rubric.
Match the tool to the problem. Not every step needs an LLM.