Slack Bot + LangChain RAG Chatbot Practical Guide — Building an Internal Document Search Bot

• Introduction
• Architecture Overview
• Project Setup
  • Installing Dependencies
  • Environment Variables
  • Project Structure
• Document Indexing
  • Loading and Chunking Documents
• Building the RAG Chain
• Slack Bot Integration
  • Slack App Configuration
  • Slack Bot Implementation
• What Shows Up in the Logs on the First Run
• The 3-Second Rule — This Bot's Structural Problem
• Why the Answer Lands in the Channel Instead of the Thread
• Where Answers Get Cut — 4,000, 3,000 and 40,000 Characters
• From One Question to One Answer — All the Way Through Once
• Docker Deployment
• Performance Optimization
  • Embedding Caching
  • Conversation History (Thread Context)
• A Version Check as of August 2026
• Failure Cases and Traps
• When Not to Use This
• Conclusion
• References
• Quiz

Introduction

"Where's the deployment procedure doc in Confluence?" "How do I access the Kubernetes cluster?"

Instead of having someone answer these questions every time, let's build an AI chatbot that searches internal documents. We'll create a production-level chatbot using the combination of LangChain + RAG (Retrieval-Augmented Generation) + Slack Bot.

Architecture Overview

# Indexing Pipeline (Offline)
# Documents → Chunking → Embedding → Vector DB (ChromaDB)

# Query Pipeline (Online)
# Slack Message → Embedding → Vector Search → LLM Generation → Slack Response

Project Setup

Installing Dependencies

mkdir slack-rag-bot && cd slack-rag-bot

# Virtual environment
python -m venv .venv
source .venv/bin/activate

# Dependencies
pip install \
  langchain==0.2.16 \
  langchain-openai==0.1.25 \
  langchain-community==0.2.16 \
  chromadb==0.5.3 \
  slack-bolt==1.20.0 \
  python-dotenv==1.0.1 \
  unstructured==0.15.0 \
  tiktoken==0.7.0

Environment Variables

# .env
OPENAI_API_KEY=sk-xxx
SLACK_BOT_TOKEN=xoxb-xxx
SLACK_APP_TOKEN=xapp-xxx
SLACK_SIGNING_SECRET=xxx
CHROMA_PERSIST_DIR=./chroma_db
DOCS_DIR=./documents

Project Structure

slack-rag-bot/
├── .env
├── main.py              # Slack Bot entry point
├── indexer.py           # Document indexing
├── rag_chain.py         # RAG chain
├── config.py            # Configuration
├── documents/           # Internal documents (Markdown, PDF, etc.)
│   ├── deployment-guide.md
│   ├── k8s-access.md
│   └── onboarding.pdf
└── chroma_db/           # Vector DB storage

Document Indexing

Loading and Chunking Documents

# indexer.py
import os
from pathlib import Path
from langchain_community.document_loaders import (
    DirectoryLoader,
    UnstructuredMarkdownLoader,
    PyPDFLoader,
    TextLoader
)
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma
from dotenv import load_dotenv

load_dotenv()


def load_documents(docs_dir: str):
    """Load documents in various formats"""
    documents = []

    # Markdown files
    md_loader = DirectoryLoader(
        docs_dir,
        glob="**/*.md",
        loader_cls=UnstructuredMarkdownLoader,
        show_progress=True
    )
    documents.extend(md_loader.load())

    # PDF files
    pdf_loader = DirectoryLoader(
        docs_dir,
        glob="**/*.pdf",
        loader_cls=PyPDFLoader,
        show_progress=True
    )
    documents.extend(pdf_loader.load())

    # Text files
    txt_loader = DirectoryLoader(
        docs_dir,
        glob="**/*.txt",
        loader_cls=TextLoader,
        show_progress=True
    )
    documents.extend(txt_loader.load())

    print(f"Total {len(documents)} documents loaded")
    return documents


def split_documents(documents):
    """Split documents into chunks"""
    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000,
        chunk_overlap=200,
        length_function=len,
        separators=["\n## ", "\n### ", "\n\n", "\n", " ", ""]
    )

    chunks = text_splitter.split_documents(documents)
    print(f"Total {len(chunks)} chunks created")
    return chunks


def create_vectorstore(chunks, persist_dir: str):
    """Create vector DB"""
    embeddings = OpenAIEmbeddings(
        model="text-embedding-3-small",
        chunk_size=500
    )

    vectorstore = Chroma.from_documents(
        documents=chunks,
        embedding=embeddings,
        persist_directory=persist_dir,
        collection_metadata={"hnsw:space": "cosine"}
    )

    print(f"Vector DB created at: {persist_dir}")
    return vectorstore


def index_documents():
    """Full indexing pipeline"""
    docs_dir = os.getenv("DOCS_DIR", "./documents")
    persist_dir = os.getenv("CHROMA_PERSIST_DIR", "./chroma_db")

    # Load → Chunk → Embed → Store
    documents = load_documents(docs_dir)
    chunks = split_documents(documents)
    vectorstore = create_vectorstore(chunks, persist_dir)

    return vectorstore


if __name__ == "__main__":
    index_documents()

# Run indexing
python indexer.py

Building the RAG Chain

# rag_chain.py
import os
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_community.vectorstores import Chroma
from langchain.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser
from dotenv import load_dotenv

load_dotenv()


class RAGChain:
    def __init__(self):
        self.embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
        self.vectorstore = Chroma(
            persist_directory=os.getenv("CHROMA_PERSIST_DIR", "./chroma_db"),
            embedding_function=self.embeddings
        )
        self.retriever = self.vectorstore.as_retriever(
            search_type="mmr",  # Maximum Marginal Relevance
            search_kwargs={
                "k": 5,
                "fetch_k": 20,
                "lambda_mult": 0.7
            }
        )
        self.llm = ChatOpenAI(
            model="gpt-4o-mini",
            temperature=0.1,
            max_tokens=2000
        )
        self.chain = self._build_chain()

    def _build_chain(self):
        """Build the RAG chain"""
        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are an internal document-based Q&A assistant.
Answer questions based on the context below.

Rules:
1. Use only the information in the context.
2. If unsure, answer "I could not find related documents."
3. Include source documents in your answer.
4. Format code or commands in code blocks.

Context:
{context}"""),
            ("human", "{question}")
        ])

        def format_docs(docs):
            formatted = []
            for i, doc in enumerate(docs):
                source = doc.metadata.get("source", "unknown")
                formatted.append(f"[Document {i+1}] ({source})\n{doc.page_content}")
            return "\n\n---\n\n".join(formatted)

        chain = (
            {"context": self.retriever | format_docs, "question": RunnablePassthrough()}
            | prompt
            | self.llm
            | StrOutputParser()
        )

        return chain

    def ask(self, question: str) -> dict:
        """Answer a question"""
        # Search for relevant documents
        docs = self.retriever.invoke(question)

        # LLM generation
        answer = self.chain.invoke(question)

        # Source document information
        sources = list(set(
            doc.metadata.get("source", "unknown") for doc in docs
        ))

        return {
            "answer": answer,
            "sources": sources,
            "num_docs": len(docs)
        }

    def refresh_index(self):
        """Refresh the index"""
        from indexer import index_documents
        self.vectorstore = index_documents()
        self.retriever = self.vectorstore.as_retriever(
            search_type="mmr",
            search_kwargs={"k": 5, "fetch_k": 20, "lambda_mult": 0.7}
        )
        self.chain = self._build_chain()

Slack Bot Integration

Slack App Configuration

1. Create a new app at https://api.slack.com/apps
2. Enable Socket Mode
3. Add Bot Token Scopes:
   - app_mentions:read
   - chat:write
   - im:history
   - im:read
   - im:write
4. Enable Event Subscriptions:
   - app_mention
   - message.im
5. Install to workspace

Slack Bot Implementation

# main.py
import os
import logging
from slack_bolt import App
from slack_bolt.adapter.socket_mode import SocketModeHandler
from rag_chain import RAGChain
from dotenv import load_dotenv

load_dotenv()
logging.basicConfig(level=logging.INFO)

# Initialize Slack App
app = App(token=os.environ["SLACK_BOT_TOKEN"])

# Initialize RAG Chain
rag = RAGChain()


@app.event("app_mention")
def handle_mention(event, say, client):
    """Receive questions via @mention"""
    user = event["user"]
    text = event["text"]
    channel = event["channel"]
    thread_ts = event.get("thread_ts", event["ts"])

    # Remove bot mention
    question = text.split(">", 1)[-1].strip()

    if not question:
        say(
            text="Please enter a question! Example: `@DocBot tell me about the deployment process`",
            thread_ts=thread_ts
        )
        return

    # Loading message
    loading_msg = client.chat_postMessage(
        channel=channel,
        thread_ts=thread_ts,
        text=":mag: Searching documents..."
    )

    try:
        # RAG query
        result = rag.ask(question)

        # Format response
        response = f"<@{user}>\n\n{result['answer']}"

        if result["sources"]:
            sources_text = "\n".join(f"• `{s}`" for s in result["sources"])
            response += f"\n\n:page_facing_up: *Reference documents:*\n{sources_text}"

        # Update loading message
        client.chat_update(
            channel=channel,
            ts=loading_msg["ts"],
            text=response
        )

    except Exception as e:
        logging.error(f"RAG error: {e}")
        client.chat_update(
            channel=channel,
            ts=loading_msg["ts"],
            text=f"Sorry, an error occurred: {str(e)}"
        )


@app.event("message")
def handle_dm(event, say):
    """Receive questions via DM"""
    if event.get("channel_type") != "im":
        return
    if event.get("bot_id"):
        return

    question = event["text"]

    try:
        result = rag.ask(question)

        response = result["answer"]
        if result["sources"]:
            sources_text = "\n".join(f"• `{s}`" for s in result["sources"])
            response += f"\n\n:page_facing_up: *Reference documents:*\n{sources_text}"

        say(text=response)

    except Exception as e:
        say(text=f"An error occurred: {str(e)}")


@app.command("/docbot-reindex")
def handle_reindex(ack, say):
    """Refresh index via slash command"""
    ack()
    say("Refreshing the index... :hourglass_flowing_sand:")

    try:
        rag.refresh_index()
        say("Index refresh complete! :white_check_mark:")
    except Exception as e:
        say(f"Index refresh failed: {str(e)}")


if __name__ == "__main__":
    handler = SocketModeHandler(app, os.environ["SLACK_APP_TOKEN"])
    print("Slack RAG Bot started!")
    handler.start()

What Shows Up in the Logs on the First Run

Installing the app into the workspace and running python main.py is where things really begin, and most tutorials end exactly there.

The Socket Mode handler calls apps.connections.open with the app-level token, gets a WebSocket URL back, and receives events over that socket. No public URL, no inbound port. In the documentation's own words, "When using Socket Mode, your app does not need a Request URL to use the Events API."

The three tokens are confusing. SLACK_BOT_TOKEN starts with xoxb-, SLACK_APP_TOKEN starts with xapp-, and SLACK_SIGNING_SECRET is only used in HTTP mode. Without connections:write on the app-level token the socket will not open at all; receiving app_mention needs app_mentions:read, and posting an answer needs chat:write.

# Run
python main.py

# If the logs we write ourselves come out in this order, things are fine.
# (What Bolt prints on its own varies by version, so do not use it as a baseline.)
[INFO] socket mode handler started
[INFO] event=app_mention channel=C123ABC456 user=U061F7AUR ts=1515449522.000016
[INFO] question='tell me the deployment process' thread_ts=None
[INFO] retrieved 5 chunks in 0.42s
[INFO] llm answered in 6.1s len=1842
[INFO] chat_update ok

To print these logs you only need to take one more listener argument. Bolt injects listener function arguments by name, so you write down only what you need.

ack       returns an acknowledgement to the Slack server
say       calls chat.postMessage with the associated channel ID
respond   uses the associated response_url
body      the entire parsed request body
payload   the core data stripped out of the request body
event / message / command / action / shortcut / view / options
          aliases for payload in each listener
client    a WebClient instance with a valid token
logger    the logger
context   a BoltContext instance
next      moves on to the next step of the middleware chain

# Arguments are injected by name. Neither the order nor taking all of them matters.
@app.event("app_mention")
def handle_mention(event, say, client, logger):
    logger.info(
        "event=app_mention channel=%s user=%s ts=%s thread_ts=%s",
        event["channel"],
        event["user"],
        event["ts"],
        event.get("thread_ts"),
    )

That one line is enough to identify most of the problems below from the logs alone. The last field in particular, whether thread_ts prints as None or as a value, is the crux of the next two sections.

The 3-Second Rule — This Bot's Structural Problem

Start with the symptom. The bot answers the same question twice, and in bad cases four times. But the duplicates have a rhythm. The first is almost immediate, the next about a minute later, the last about five minutes later. The moment you see those intervals the cause narrows to one thing.

The Events API documentation puts it this way: "Your app should respond to the event request with an HTTP 2xx within three seconds." The Bolt documentation is blunter: "We recommend calling ack() right away before initiating any time-consuming processes… since you only have 3 seconds to respond before Slack registers a timeout error."

A RAG pipeline does not finish in 3 seconds. Query embedding, vector search, LLM generation. The last one alone is usually 3 to 15 seconds. With the defaults left as they are, the structure in this article is bound to time out, which makes it a structural problem rather than an accidental bug.

When it times out, Slack retries. In the documentation's own words, "retrying a failed request up to 3 times in a gradually increasing timetable", where the first retry is almost immediate, the second a minute later, and the last five minutes later. Retried requests carry an x-slack-retry-num header whose value is one of "1", "2" or "3", with the reason in x-slack-retry-reason. If you do not want it again, put x-slack-no-retry: 1 on a non-200 response; the documentation explains this as "we'll understand it to mean you'd rather this specific event not be re-delivered".

14:02:10.114  event=app_mention ts=1515449522.000016   answer generation starts
14:02:10.140  event=app_mention ts=1515449522.000016   <- retry 1
14:03:10.203  event=app_mention ts=1515449522.000016   <- retry 2
14:08:10.377  event=app_mention ts=1515449522.000016   <- retry 3

# The ts values are all identical. The user asked once, and we answered four times.

One point worth pinning down. What the documentation firmly requires an ack() acknowledgement for is actions, commands, shortcuts, options requests and view submissions. Events are not on that list, and the framework handles acknowledging events. That does not make the 3 seconds go away, because the 3 seconds is an Events API rule, not a Bolt one. There is only one reliable approach: return from the listener immediately and run the actual work on another thread.

import threading

@app.event("app_mention")
def handle_mention(event, client, logger):
    # The listener returns immediately. RAG does not run here.
    threading.Thread(
        target=answer_in_background,
        args=(event, client, logger),
        daemon=True,
    ).start()


@app.command("/docbot-reindex")
def handle_reindex(ack, say):
    ack()  # For slash commands the documentation explicitly requires ack().
    threading.Thread(target=reindex_in_background, args=(say,), daemon=True).start()

Even this does not stop retries that already went out. On days when the LLM is slow duplicates still arrive, so we add one more layer.

from collections import OrderedDict
import threading

_seen = OrderedDict()
_seen_lock = threading.Lock()


def already_handled(body) -> bool:
    """True when the same event_id comes back. Filters out Slack's re-delivery."""
    event_id = body.get("event_id")
    if not event_id:
        return False
    with _seen_lock:
        if event_id in _seen:
            return True
        _seen[event_id] = True
        while len(_seen) > 5000:
            _seen.popitem(last=False)
    return False


@app.event("app_mention")
def handle_mention(body, event, client, logger):
    if already_handled(body):
        logger.info("duplicate event_id=%s skipped", body.get("event_id"))
        return
    threading.Thread(
        target=answer_in_background, args=(event, client, logger), daemon=True
    ).start()

Let me be straight about this. The retry schedule and the header names are documented behaviour, but the code above that filters on event_id is not an official recipe from the Bolt documentation; it is an idiom I use. The same goes for the cruder approach of skipping any request carrying x-slack-retry-num outright in HTTP mode. And I could not confirm from the documentation whether Socket Mode re-delivery behaves exactly like the table above. Defending on event_id is safe in both modes.

If you run more than one process, an in-memory dictionary is not enough. Put the event_id into shared storage such as Redis with a TTL.

Why the Answer Lands in the Channel Instead of the Thread

An app_mention payload looks like this. It is the example straight from the documentation.

{
  "type": "app_mention",
  "user": "U061F7AUR",
  "text": "<@U0LAN0Z89> is it everything a river should be?",
  "ts": "1515449522.000016",
  "channel": "C123ABC456",
  "event_ts": "1515449522000016"
}

The important part is that there is no thread_ts key. Mention the bot at the top level of a channel and there is no thread concept at all, so the key simply does not arrive, and writing event["thread_ts"] raises a KeyError right there. The logs keep only the exception and the user sees no reaction at all. This is usually where the report "the bot seems dead" comes from.

Conversely, mention it inside an existing thread and thread_ts arrives filled with the parent message's ts. That is how the idiom settled into this shape.

# For a top-level mention, this message itself becomes the start of the thread.
# For a mention inside a thread, the parent's ts is already in thread_ts.
thread_ts = event.get("thread_ts") or event["ts"]

say(text=answer, thread_ts=thread_ts)

The thread_ts documentation for chat.postMessage says: "Provide another message's ts value to make this message a reply. Avoid using a reply's ts value; use its parent instead." The one line above follows exactly that rule. Passing a reply's own timestamp inside a thread amounts to trying to open a thread on a reply, and the answer does not attach where the user expected it. The original code's event.get("thread_ts", event["ts"]) gives the same result.

There is one thing to disclose. I could not find the exact call shape for passing thread_ts to say in the Bolt documentation as such. It is the idiom you get by combining the description that say "calls chat.postMessage API with the associated channel ID" with the rule above.

Where Answers Get Cut — 4,000, 3,000 and 40,000 Characters

RAG answers are long. With a source list appended they get longer. But Slack has three different upper bounds.

• chat.postMessage — recommends limiting text to 4,000 characters for best results. Past 40,000 characters Slack truncates.
• chat.update — the error string nails it down. "Message text is too long. The text field cannot exceed 4,000 characters." markdown_text goes up to 12,000.
• Block Kit section blocks — text has a minimum length of 1 and a maximum of 3,000. Items in the fields array are 2,000 each.

The trap is the last line. The moment you put an answer into a section block to make it look nice, the limit drops to 3,000. An ordinary RAG answer going past 3,000 characters is very common, so an answer that was fine as plain text starts getting cut the moment you prettify it.

SECTION_LIMIT = 3000   # text limit of a Block Kit section block
TEXT_LIMIT = 4000      # recommended limit of chat.postMessage / chat.update


def chunk_for_slack(answer: str, limit: int = TEXT_LIMIT) -> list[str]:
    """Cut on paragraph boundaries. Cutting anywhere breaks code blocks."""
    parts, buf = [], ""
    for para in answer.split("\n\n"):
        if len(buf) + len(para) + 2 > limit:
            if buf:
                parts.append(buf)
                buf = ""
            while len(para) > limit:
                parts.append(para[:limit])
                para = para[limit:]
            buf = para
        else:
            buf = f"{buf}\n\n{para}" if buf else para
    if buf:
        parts.append(buf)
    return parts

Next comes call frequency. The documentation says chat.postMessage "generally allows posting one message per second per channel", and a workspace-wide limit applies alongside it. chat.update is Tier 3, that is 50 or more per minute (the tiers are 1 for one per minute, 2 for 20, 3 for 50, and 4 for 100 or more). The pattern of posting a loading message and swapping it out with chat.update is fine in itself, but calling it per token to fake streaming runs head-on into Tier 3. Go over and, in the documentation's own words, "Slack will return a HTTP 429 Too Many Requests error, and a Retry-After HTTP header containing the number of seconds until you can retry."

symptom     the answer stops dead in the middle of a sentence
check       log len(answer)                ->  3214
diagnosis   it went into a Block Kit section block (3,000). The text field (4,000) would have passed.

symptom     the answer appears but updates stop a few seconds later
check       log the response code and headers  ->  429 / Retry-After: 12
diagnosis   chat.update was called several times per second. Tier 3 (50+/min) was exceeded.

Bolt also has streaming surfaces such as say_stream and WebClient.chat_stream. I could not confirm their full arguments and behaviour, so check the exact API in the documentation for the version you are using.

From One Question to One Answer — All the Way Through Once

Gathering everything so far into a single handler gives you this. There are no new concepts; the ordering is the whole thing.

[channel #dev-help]
jiwoo    @DocBot tell me the staging deployment process   14:02:10

DocBot   :mag: Searching documents...                     14:02:10
         (the same message turns into the answer 6 seconds later)

DocBot   @jiwoo                                           14:02:16
         Staging deployment goes in this order.
         1. Merge into the main branch
         2. Confirm CI passes
         3. Run the deploy-staging workflow manually

         :page_facing_up: Reference documents:
         - documents/deployment-guide.md
         - documents/ci-cd.md

# main.py — with acknowledgement / deduplication / threading / length all reflected
import os
import threading
import logging

from slack_bolt import App
from slack_bolt.adapter.socket_mode import SocketModeHandler
from rag_chain import RAGChain

logging.basicConfig(level=logging.INFO)
app = App(token=os.environ["SLACK_BOT_TOKEN"])
rag = RAGChain()


def answer_in_background(event, client, logger):
    channel = event["channel"]
    thread_ts = event.get("thread_ts") or event["ts"]
    question = event["text"].split(">", 1)[-1].strip()
    logger.info("question=%r thread_ts=%s", question, event.get("thread_ts"))

    placeholder = client.chat_postMessage(
        channel=channel,
        thread_ts=thread_ts,
        text=":mag: Searching documents...",
    )
    try:
        result = rag.ask(question)
        body = f"<@{event['user']}>\n\n{result['answer']}"
        if result["sources"]:
            lines = "\n".join(f"- `{s}`" for s in result["sources"])
            body += f"\n\n:page_facing_up: *Reference documents:*\n{lines}"

        parts = chunk_for_slack(body)
        client.chat_update(channel=channel, ts=placeholder["ts"], text=parts[0])
        for extra in parts[1:]:
            client.chat_postMessage(channel=channel, thread_ts=thread_ts, text=extra)
        logger.info("answered len=%d parts=%d", len(body), len(parts))
    except Exception:
        # Do not put the exception string into the channel as is. Details go to the log.
        logger.exception("rag failed")
        client.chat_update(
            channel=channel,
            ts=placeholder["ts"],
            text="I could not produce an answer. Please try again shortly.",
        )


@app.event("app_mention")
def handle_mention(body, event, client, logger):
    if already_handled(body):
        return
    threading.Thread(
        target=answer_in_background, args=(event, client, logger), daemon=True
    ).start()


if __name__ == "__main__":
    SocketModeHandler(app, os.environ["SLACK_APP_TOKEN"]).start()

For one pass over the same question the logs look like this.

[INFO] event=app_mention channel=C123ABC456 user=U061F7AUR ts=1515449522.000016 thread_ts=None
[INFO] question='tell me the staging deployment process' thread_ts=None
[INFO] retrieved 5 chunks in 0.42s
[INFO] llm answered in 6.1s
[INFO] answered len=1842 parts=1

thread_ts=None means it was a top-level mention, so that message itself becomes the start of the thread. If a value printed, the question came from inside a thread and the answer attaches to that thread. This one line tells you immediately whether the problem from the previous section occurred.

Breaking the time apart matters too. With retrieval at 0.4 seconds and generation at 6.1, the bottleneck is clearly generation. If retrieval goes past 2 seconds, suspect k or fetch_k first. parts=1 means the answer went in one piece; if 2 or more is frequent, it is better to limit answer length in the prompt.

Docker Deployment

# Dockerfile
FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

# Start bot after indexing
CMD ["python", "main.py"]

# docker-compose.yml
version: '3.8'

services:
  slack-rag-bot:
    build: .
    env_file: .env
    volumes:
      - ./documents:/app/documents
      - ./chroma_db:/app/chroma_db
    restart: unless-stopped

# Build and run
docker compose up -d

# Check logs
docker compose logs -f

Performance Optimization

Embedding Caching

from langchain.storage import LocalFileStore
from langchain.embeddings import CacheBackedEmbeddings

store = LocalFileStore("./embedding_cache")
cached_embeddings = CacheBackedEmbeddings.from_bytes_store(
    underlying_embeddings=OpenAIEmbeddings(model="text-embedding-3-small"),
    document_embedding_cache=store,
    namespace="text-embedding-3-small"
)

Conversation History (Thread Context)

from langchain.memory import ConversationBufferWindowMemory

# Per-thread memory management
thread_memories = {}

def get_memory(thread_ts: str) -> ConversationBufferWindowMemory:
    if thread_ts not in thread_memories:
        thread_memories[thread_ts] = ConversationBufferWindowMemory(
            k=5,
            memory_key="chat_history",
            return_messages=True
        )
    return thread_memories[thread_ts]

A Version Check as of August 2026

The code in this article is current as of March 2026. Install it as it stands today and half of it is somewhere else. slack_bolt 1.30.0 came out on 15 July 2026 and supports Python 3.7 through 3.14. On the LangChain side it is langchain 1.3.15 (11 August 2026), langchain-core 1.5.5, langchain-classic 1.0.8 and langchain-community 0.4.2.

The problem is langchain-community. The PyPI banner reads "langchain-community is being sunset. See #674 for details and guidance.", and the issue body says "We are making the decision to sunset the langchain-community package… This sunset will take effect immediately." It is dated 22 May 2026 and no end date is stated, so do not plan a schedule around a deadline that does not exist. In v1, langchain shrank to agents, messages, tools, chat_models and embeddings, and the old chains and memory went to langchain-classic.

# Installing again on a 2026-08 basis
pip install \
  "langchain==1.3.15" \
  "langchain-core==1.5.5" \
  "langchain-classic==1.0.8" \
  "langchain-openai" \
  "langchain-text-splitters" \
  "langchain-chroma>=0.1.2" \
  "slack-bolt==1.30.0" \
  "python-dotenv"

# Imports in this article -> where they live as of 2026-08
# langchain.memory.ConversationBufferMemory
#   -> langchain_classic.memory.buffer.ConversationBufferMemory
# langchain.embeddings.CacheBackedEmbeddings
#   -> langchain_classic.embeddings.cache.CacheBackedEmbeddings
# langchain.storage.LocalFileStore
#   -> langchain_classic.storage.file_system.LocalFileStore
# langchain.chains.RetrievalQA
#   -> langchain_classic.chains.retrieval_qa.base.RetrievalQA

# The text splitters shipped as their own package.
from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=200,
    add_start_index=True,
)

# Chroma's own package is the official one too.
# (langchain_community.vectorstores.Chroma has been deprecated since community 0.2.9)
from langchain_chroma import Chroma

# The LCEL pieces remain in langchain-core exactly as they were.
from langchain_core.runnables import RunnablePassthrough, RunnableLambda
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.documents import Document

# Retriever options, summarized
#   search_type   : 'similarity'(default) / 'mmr' / 'similarity_score_threshold'
#   search_kwargs : k(default 4), score_threshold, fetch_k(default 20),
#                   lambda_mult(default 0.5), filter
#   The lambda_mult=0.7 used in this article weighs relevance over diversity.

Anyone who has migrated once is more surprised. create_retrieval_chain, create_stuff_documents_chain and create_history_aware_retriever, the ones presented as the standard back in v0.2 and v0.3, are in langchain-classic in v1 as well. FAISS is worse off. It is still from langchain_community.vectorstores import FAISS and there is no package called langchain-faiss. It is the one path left inside a package with an announced sunset and nowhere to move to.

The LCEL pieces are alive in langchain-core as they were, and the Runnable documentation still says "Any chain constructed this way will automatically have sync, async, batch, and streaming support." What disappeared from the v1 documentation is the name "LCEL". It was not discarded; the documentation just no longer teaches it under that name.

What v1 puts front and center is create_agent. "create_agent is the standard way to build agents", and memory became a checkpointer. "To add short-term memory (thread-level persistence) to an agent, you need to specify a checkpointer when creating an agent."

from langchain.agents import create_agent
from langgraph.checkpoint.memory import InMemorySaver

agent = create_agent(
    model="openai:gpt-5.5",
    tools=[search_internal_docs],
    checkpointer=InMemorySaver(),
)

# This is the point where it lines up exactly with a Slack bot.
# One thread = one conversation. Use thread_ts directly as thread_id.
thread_ts = event.get("thread_ts") or event["ts"]
thread_config = {"configurable": {"thread_id": thread_ts}}

The correspondence is clean. The checkpointer takes over what the thread_memories dictionary was doing, and Slack is already handing you the key. In operation, use a persistent checkpointer such as PostgresSaver.

One note on async as well. The async methods on VectorStore are not genuinely asynchronous but thread-pool wrappers.

# Part of the VectorStore implementation in langchain-core
async def asimilarity_search(self, query: str, k: int = 4, **kwargs: Any) -> list[Document]:
    return await run_in_executor(None, self.similarity_search, query, k=k, **kwargs)


# Import paths on the async Bolt side, noted here too.
# from slack_bolt.app.async_app import AsyncApp          (short alias: slack_bolt.async_app)
# from slack_bolt.adapter.socket_mode.async_handler import AsyncSocketModeHandler
#   (also available at slack_bolt.adapter.socket_mode.aiohttp)
# await handler.start_async()

The source comments call this a set of "temporary workarounds" and note that "The proper solution is to make the similarity search asynchronous in the vector store implementations." Since Chroma in langchain_chroma has not a single async def, switching to async will not make retrieval any faster.

Finally, I could not confirm the full argument list of CacheBackedEmbeddings.from_bytes_store in the current langchain-classic reference. That page carries a work-in-progress marker. Check the exact API in the documentation for the version you are using.

Failure Cases and Traps

The symptom comes first, with the diagnostic order attached.

The bot answers its own answers — the @app.event("message") handler also receives messages the bot posted. Without a guard, a loop runs where it reads its own answer as a new question. The two lines if event.get("bot_id"): return in the original code are that guard. Do not delete them.

One mention gets two answers — it may be double subscription rather than a retry. Subscribe to both app_mention and message and a single channel mention wakes two listeners. Logging the event name separates the two. The same name twice is a retry; two different names once each is double subscription.

Indexing supposedly worked but retrieval comes back empty — check three things in order. Whether the embedding model at index time and at query time is the same (if not, the vector spaces differ and similarity becomes meaningless), whether persist_directory points at the same place (the most common case is forgetting the volume and looking at an empty directory inside the container), and whether the collection name is the same. Do not touch chunking or the prompt before these three.

It always brings back the wrong document — suspect chunk boundaries. With chunk_size=1000 and chunk_overlap=200, tables and code blocks get cut in the middle. The cut fragment is meaningless, yet it embeds perfectly well. Turning on add_start_index=True leaves the position in the original as metadata.

Calling the re-index command freezes the whole bot — slash commands live under the same 3-second rule. The original code calling ack() first is correct, but running a full re-index synchronously on the same thread is the problem. In a workspace where it takes three minutes, every question that arrives during those three minutes times out, gets retried, and comes back as a burst of answers much later.

A question asked during re-indexing gets a strange answer — while refresh_index() swaps the vector store, the retriever and the chain one after another, other threads are reading those objects. A new index gets mixed with an old chain. Build everything on a temporary path and then swap a single reference.

The bill grows quietly — with k=5 and 1,000-character chunks, more than 5,000 characters go into the prompt per question. Attach history and it grows further. Logging the prompt length makes it visible the same day rather than at the end of the month.

Exception strings get pasted into the channel — the original code turns the exception into a string and posts it in a message. If a connection string or a fragment of a key is mixed in, that ends up sitting in a public channel. Generic guidance goes to the channel; the details go to the log.

symptom                        look at first          then
same answer 2-4 times          duplicate event_id     over 3 seconds -> retry
two answers to one mention     event name in the log  app_mention + message double subscription
retrieval comes back empty     embedding model match  persist_directory / collection name
answer stops mid-sentence      answer length          3,000 (Block Kit) vs 4,000 (text)
answer lands outside a thread  logged thread_ts       whether a reply's ts is being used as is

# Extend ask() by one line so it also returns the prompt length,
# and the cost problem shows up in the log rather than on the bill.
try:
    result = rag.ask(question)
    logger.info("prompt_chars=%d chunks=%d", result["prompt_chars"], result["num_docs"])
except Exception:
    logger.exception("rag failed question=%r", question[:200])
    client.chat_update(
        channel=channel,
        ts=placeholder["ts"],
        text="I could not produce an answer. Please try again shortly.",
    )

When Not to Use This

There are clear cases where this combination is not the answer.

When read permissions differ per document. This is the dangerous one. A Slack bot answers with its own document access rights, not with the rights of the person asking. Every document the indexing script could read becomes the bot's knowledge, and anyone in the workspace pulls that knowledge out with a single mention. If one folder of performance review documents accidentally sits in the documents directory, someone without permission gets an answer just by asking. The vector DB stores the raw fragments as they are, so the comfort of "only summaries go out, so it is fine" does not hold either.

# When a person opens a document
user -> document store -> permission check -> access allowed or denied

# When the bot answers
user -> bot -> vector DB already indexed with the bot's rights -> answer
             (who asked has no influence at all at this point)

For a corpus where permissions differ, the design comes first: split the index by tier, or apply per-user filters at the retrieval step. If that is too much, put only documents anyone may see into this bot.

When pasting it all in would be enough. If the documents run to about ten pages, there is no reason to stand up a vector DB. Put all of it into the prompt. The indexing pipeline, the re-index command, the volume mounts and the embedding cost all disappear at once. RAG is the tool for when it will not all fit in the context.

Questions Slack search does better. A question like "where did that thread go" belongs to Slack's own search rather than to RAG. Conversation history is not a document, and embedding it and summarizing it actually throws the context away.

When you plan to list on the Marketplace. The Socket Mode documentation states: "Apps using Socket Mode are not currently allowed in the public Slack Marketplace." That is no problem for internal use, but if you intend to sell it as a product, go with HTTP mode from the start. The shape changes to passing a signing secret and exposing a Request URL, and it is a fairly annoying decision to change later.

One more thing. In areas where being wrong is a problem, such as compliance or legal, a bot that returns only source links rather than composing an answer is better. Writing "use only the information in the context" does not stop the LLM from breaking that rule occasionally, and if the occasional case is subject to audit, the shape of the tool itself has to change.

Conclusion

Key points for a Slack RAG chatbot:

1. Document chunking: Semantic-unit splitting with RecursiveCharacterTextSplitter
2. Vector search: Diverse document retrieval with MMR (Maximum Marginal Relevance)
3. Prompting: Design to cite sources and answer honestly when uncertain
4. Slack integration: Socket Mode + app_mention/DM event handling
5. Re-indexing: Reflect document updates via slash command
6. The 3-second rule: Return from the listener immediately and run RAG on a separate thread. Defend against retry duplicates with event_id
7. Length limits: 4,000 characters for text, 3,000 for a Block Kit section block. The cut happens in different places
8. Versions: langchain-community has an announced sunset, and the old chains moved to langchain-classic

References

All checked as of 2026-08-16.

• Bolt for Python — Socket Mode — initialization and starting the handler
• Bolt for Python — Acknowledging requests — the acknowledgement recommendation
• Bolt for Python — listener arguments — the source of the list above
• Slack Events API — the 3-second rule, the retry schedule and headers
• Using Socket Mode — no Request URL needed, the Marketplace restriction
• app_mention event — the payload example
• Web API rate limits — tiers and the 429 response
• langchain-classic reference — the relocated module paths

The old address tools.slack.dev/bolt-python/ 301-redirects to docs.slack.dev/tools/bolt-python/. If it is still in your internal wiki, update it. For message length limits and Block Kit field specifications, the method and block pages on docs.slack.dev are authoritative.

Quiz