Skip to content
J—S
All projects

May 1, 2025

Internal LLM Tooling

Production AI tooling with OpenAI and Anthropic Claude prompt pipelines, Zod-validated structured output, and BullMQ async inference.

  • OpenAI
  • Anthropic Claude
  • TypeScript
  • Zod
  • BullMQ
  • Redis
  • Retell AI

Problem

Internal teams needed reliable AI automation for content and workflow generation without blocking primary APIs on long inference jobs or accepting malformed model output.

Architecture decisions

  • Provider-agnostic prompt pipelines for OpenAI GPT-4o and Anthropic Claude.
  • Zod schema validation on every model response with reinforced retries on malformed JSON.
  • BullMQ workers on Redis for non-blocking async inference and backpressure.
  • Dynamic context assembly at job execution time so prompts use current data.
  • Retell AI evaluated for conversational capture with architecture prepared for rollout.

2

LLM providers integrated

OpenAI GPT-4o + Anthropic Claude (production)

BullMQ

Queue-backed inference jobs

Non-blocking async pipeline

Zod

Output validation layer

Structured JSON enforced on every model response

Retell AI

Voice AI integration

Conversational agent layer (prototype)

Role and impact

Designed and built the internal LLM tooling suite used to automate content and operational workflows at Netpace.

Tech stack

  • LLM: OpenAI API · Anthropic Claude API · LangChain · Retell AI
  • Runtime: TypeScript · Zod · BullMQ · Redis

Related projects