Skip to content
TwoFour_

~/work/aiassist $ cat case-study.md

RAG-powered knowledge base chatbot with expert training loop

Outcomes

  • ▸RAG pipeline accurately answers questions from existing documentation
  • ▸Ground truth system improves accuracy over time with expert feedback
  • ▸Citation tracking ensures every answer is verifiable
  • ▸Automated KB sync keeps the chatbot current without manual effort

$ cat overview.txt

Overview

AiAssist is a Retrieval-Augmented Generation (RAG) chatbot that syncs with a company's knowledge base, creates vector embeddings, and generates AI-powered answers with source citations. It includes an expert review workflow where corrected answers become ground truth for future queries, making the system smarter over time.

$ cat challenge.txt

The problem

Support teams spend hours answering the same questions repeatedly, and knowledge base articles go unread. We wanted to build a chatbot that could actually answer questions accurately from existing documentation and get smarter over time as subject matter experts correct its responses.

$ cat solution.txt

The approach

We built a RAG pipeline that syncs knowledge base articles on a schedule, chunks them into ~400-token segments with semantic overlap, and generates vector embeddings using OpenAI's text-embedding-3-large model. When a user asks a question, the system performs cosine similarity search to find relevant context, then generates an answer with GPT-4o, always citing the source articles.

The key differentiator is the ground truth system. When an expert reviews a conversation and corrects an answer, that correction is embedded and stored as ground truth. Future queries that match closely (>0.95 similarity) return the expert-verified answer directly, creating a feedback loop that continuously improves accuracy.

aiassist/screen-01.tsx
aiassist/screen-02.tsx

$ ls features/

What shipped

  • +Automatic knowledge base sync (every 3 hours)
  • +Text chunking with semantic overlap (~400 tokens)
  • +Vector embeddings via OpenAI text-embedding-3-large
  • +Semantic similarity search for relevant context
  • +GPT-4o answer generation with source citations
  • +Ground truth system for expert-corrected answers
  • +Admin dashboard with review queue and statistics
  • +AI Training Mode for subject matter experts
  • +Conversation logging and audit trail
  • +Data export capabilities
twofour.os · /new-project

Let's build something that ships.

Book a discovery call and we'll map your problem, scope a first sprint, and tell you honestly whether we're the right team.

▸ book_discovery_call