Deb8flow: Orchestrating Autonomous AI Debates with LangGraph and GPT-4o

I’ve been fascinated by debates—the strategic framing, the sharp retorts, and the rigorously timed comebacks. Debates aren’t simply entertaining; they’re structured battles of concepts, pushed by logic and proof. Lately, I began questioning: may we replicate that dynamic utilizing AI brokers—having them debate one another autonomously, full with real-time fact-checking and moderation? The end result was Deb8flow, an autonomous AI debating surroundings powered by LangGraph, OpenAI’s GPT-4o mannequin, and the brand new built-in Web Search function.

In Deb8flow, two brokers—Professional and Con—sq. off on a given matter whereas a Moderator manages turn-taking. A devoted Truth Checker critiques each declare in actual time utilizing GPT-4o’s new searching capabilities, and a ultimate Choose evaluates the arguments for high quality and coherence. If an agent repeatedly makes factual errors, they’re routinely disqualified—making certain the talk stays grounded in reality.

This text affords an in-depth take a look at the superior structure and dynamic workflows that energy autonomous AI debates. I’ll stroll you thru how Deb8flow’s modular design leverages LangGraph’s state administration and conditional routing, alongside GPT-4o’s capabilities.

Even in case you’re new to AI brokers or LangGraph (see assets [1] and [2] for primers), I’ll clarify the important thing ideas clearly. And in case you’d prefer to discover additional, the total venture is offered on GitHub: iason-solomos/Deb8flow.

Able to see how AI brokers can debate autonomously in follow?

Let’s dive in.

Excessive-Stage Overview: Autonomous Debates with A number of Brokers

In Deb8flow, we orchestrate a formal debate between two AI brokers – one arguing Professional and one Con – full with a Moderator, a Truth Checker, and a ultimate Choose. The talk unfolds autonomously, with every agent enjoying a task in a structured format.

At its core, Deb8flow is a LangGraph-powered agent system, constructed atop LangChain, utilizing GPT-4o to energy every position—Professional, Con, Choose, and past. We use GPT-4o’s preview mannequin with searching capabilities to allow real-time fact-checking. In essence, the Professional and Con brokers debate; after every assertion, a fact-checker agent makes use of GPT-4o’s internet search to catch any hallucinations or inaccuracies in that assertion in actual time.​ The talk solely continues as soon as the assertion is verified. The entire course of is coordinated by a LangGraph-defined workflow that ensures correct turn-taking and conditional logic.

The talk workflow goes by means of these levels:

• Matter Era: A Matter Generator agent produces a nuanced, debatable matter for the session (e.g. “Ought to AI be utilized in classroom training?”).
• Opening: The Professional Argument Agent makes a gap assertion in favor of the subject, kicking off the talk.
• Rebuttal: The Debate Moderator then offers the ground to the Con Argument agent, who rebuts the Professional’s opening assertion.
• Counter: The Moderator offers the ground again to the Professional agent, who counters the Con agent’s factors.
• Closing: The Moderator switches the ground to the Con agent one final time for a closing argument.
• Judgment: Lastly, the Choose agent critiques the total debate historical past and evaluates either side based mostly on argument high quality, readability, and persuasiveness. Essentially the most convincing facet wins.

After each single speech, the Truth Checker agent steps in to confirm the factual accuracy of that assertion​. If a debater’s declare doesn’t maintain up (e.g. cites a fallacious statistic or “hallucinates” a reality), the workflow triggers a retry: the speaker has to appropriate or modify their assertion. (If both debater accumulates 3 fact-check failures, they’re routinely disqualified for repeatedly spreading inaccuracies, and their opponent wins by default.) This mechanism retains our AI debaters trustworthy and grounded in actuality!

Stipulations and Setup

Earlier than diving into the code, be sure to have the next in place:

• Python 3.12+ put in.
• An OpenAI API key with entry to the GPT-4o mannequin. You’ll be able to create your personal API key right here: https://platform.openai.com/settings/organization/api-keys
• Challenge Code: Clone the Deb8flow repository from GitHub (git clone https://github.com/iason-solomos/Deb8flow.git). The repo features a necessities.txt for all required packages. Key dependencies embody LangChain/LangGraph (for constructing the agent graph) and the OpenAI Python shopper.
• Set up Dependencies: In your venture listing, run: pip set up -r necessities.txt to put in the mandatory libraries.
• Create a .env file within the venture root to carry your OpenAI API credentials. It ought to be of the shape: OPENAI_API_KEY_GPT4O = "sk-…"
• You may as well at any time take a look at the README file: https://github.com/iason-solomos/Deb8flow in case you merely wish to run the completed app.

As soon as dependencies are put in and the surroundings variable is ready, try to be able to run the app. The venture construction is organized for readability:

Deb8flow/
├── configurations/
│ ├── debate_constants.py
│ └── llm_config.py
├── nodes/
│ ├── base_component.py
│ ├── topic_generator_node.py
│ ├── pro_debater_node.py
│ ├── con_debater_node.py
│ ├── debate_moderator_node.py
│ ├── fact_checker_node.py
│ ├── fact_check_router_node.py
│ └── judge_node.py
├── prompts/
│ ├── topic_generator_prompts.py
│ ├── pro_debater_prompts.py
│ ├── con_debater_prompts.py
│ └── … (prompts for different brokers)
├── checks/ (accommodates unit and entire workflow checks)
└── debate_workflow.py

A fast tour of this construction:

configurations/ holds fixed definitions and LLM configuration courses.

nodes/ accommodates the implementation of every agent or practical node within the debate (every of those is a module defining one agent’s habits).

prompts/ shops the immediate templates for the language mannequin (so every agent is aware of tips on how to immediate GPT-4o for its particular activity).

debate_workflow.py ties every little thing collectively by defining the LangGraph workflow (the graph of nodes and transitions).

debate_state.py defines the shared knowledge construction that the brokers can be utilizing on every run.

checks/ contains some primary checks and instance runs that can assist you confirm every little thing is working.

Underneath the Hood: State Administration and Workflow Setup

To coordinate a posh multi-turn debate, we want a shared state and a well-defined stream. We’ll begin by how Deb8flow defines the debate state and constants, after which see how the LangGraph workflow is constructed.

Defining the Debate State Schema (debate_state.py)

Deb8flow makes use of a shared state (https://langchain-ai.github.io/langgraph/concepts/low_level/#state ) within the type of a Python TypedDict that every one brokers can learn from and replace. This state tracks the talk’s progress and context – issues like the subject, the historical past of messages, whose flip it’s, and so forth. By centralizing this data, every agent node could make choices based mostly on the present state of the talk.

Hyperlink: debate_state.py

from typing import TypedDict, Checklist, Dict, Literal


DebateStage = Literal["opening", "rebuttal", "counter", "final_argument"]

class DebateMessage(TypedDict):
    speaker: str  # e.g. professional or con
    content material: str  # The message every speaker produced
    validated: bool  # Whether or not the FactChecker okay’d this message
    stage: DebateStage # The stage of the talk when this message was produced

class DebateState(TypedDict):
    debate_topic: str
    positions: Dict[str, str]
    messages: Checklist[DebateMessage]
    opening_statement_pro_agent: str
    stage: str  # "opening", "rebuttal", "counter", "final_argument"
    speaker: str  # "professional" or "con"
    times_pro_fact_checked: int # The variety of occasions the professional agent has been fact-checked. If it reaches 3, the professional agent is disqualified.
    times_con_fact_checked: int # The variety of occasions the con agent has been fact-checked. If it reaches 3, the con agent is disqualified.

Key fields that we have to have within the DebateState embody:

• debate_topic (str): The subject being debated.
• messages (Checklist[DebateMessage]): An inventory of all messages exchanged to date. Every message is a dictionary with fields for speaker (e.g. "professional" or "con" or "fact_checker"), the message content material (textual content), a validated flag (whether or not it handed fact-check), and the stage of the talk when it was produced.
• stage (str): The present debate stage (considered one of "opening", "rebuttal", "counter", "final_argument").
• speaker (str): Whose flip it’s presently ("professional" or "con").
• times_pro_fact_checked / times_con_fact_checked (int): Counters for what number of occasions both sides has been caught with a false declare. (In our guidelines, if a debater fails fact-check 3 occasions, they could possibly be disqualified or routinely lose.)
• positions (Dict[str, str]): (Optionally available) A mapping of every facet’s basic stance (e.g., "professional": "In favor of the subject").

By structuring the talk’s state, brokers discover it straightforward to entry the dialog historical past or verify the present stage, and the management logic can replace the state between turns. The state is basically the reminiscence of the talk.

Constants and Configuration

To keep away from “magic strings” scattered within the code, we outline some constants in debate_constants.py. For instance, constants for stage names (STAGE_OPENING = "opening", and so forth.), speaker identifiers (SPEAKER_PRO = "professional", SPEAKER_CON = "con", and so forth.), and node names (NODE_PRO_DEBATER = "pro_debater_node", and so forth.). These make the code simpler to take care of and browse.

debate_constants.py:

# Stage names
STAGE_OPENING = "opening"
STAGE_REBUTTAL = "rebuttal"
STAGE_COUNTER = "counter"
STAGE_FINAL_ARGUMENT = "final_argument"
STAGE_END = "finish"

# Audio system
SPEAKER_PRO = "professional"
SPEAKER_CON = "con"
SPEAKER_JUDGE = "decide"

# Node names
NODE_PRO_DEBATER = "pro_debater_node"
NODE_CON_DEBATER = "con_debater_node"
NODE_DEBATE_MODERATOR = "debate_moderator_node"
NODE_JUDGE = "judge_node"

We additionally arrange LLM configuration in llm_config.py. Right here, we outline courses for OpenAI or Azure OpenAI configs after which create a dictionary llm_config_map mapping mannequin names to their config. As an illustration, we map "gpt-4o" to an OpenAILLMConfig that holds the mannequin title and API key. This fashion, every time we have to initialize a GPT-4o agent, we will simply do llm_config_map["gpt-4o"] to get the proper config. All our principal brokers (debaters, matter generator, decide) use this similar GPT-4o configuration.

import os
from dataclasses import dataclass
from typing import Union

@dataclass
class OpenAILLMConfig:
    """
    A knowledge class to retailer configuration particulars for OpenAI fashions.

    Attributes:
        model_name (str): The title of the OpenAI mannequin to make use of.
        openai_api_key (str): The API key for authenticating with the OpenAI service.
    """
    model_name: str
    openai_api_key: str


llm_config_map = {
    "gpt-4o": OpenAILLMConfig(
        model_name="gpt-4o",
        openai_api_key=os.getenv("OPENAI_API_KEY_GPT4O"),
    )
}

Constructing the LangGraph Workflow (debate_workflow.py)

With state and configs in place, we assemble the debate workflow graph. LangGraph’s StateGraph is the spine that connects all our agent nodes within the order they need to execute. Right here’s how we set it up:

class DebateWorkflow:

    def _initialize_workflow(self) -> StateGraph:
        workflow = StateGraph(DebateState)
        # Nodes
        workflow.add_node("generate_topic_node", GenerateTopicNode(llm_config_map["gpt-4o"]))
        workflow.add_node("pro_debater_node", ProDebaterNode(llm_config_map["gpt-4o"]))
        workflow.add_node("con_debater_node", ConDebaterNode(llm_config_map["gpt-4o"]))
        workflow.add_node("fact_check_node", FactCheckNode())
        workflow.add_node("fact_check_router_node", FactCheckRouterNode())
        workflow.add_node("debate_moderator_node", DebateModeratorNode())
        workflow.add_node("judge_node", JudgeNode(llm_config_map["gpt-4o"]))

        # Entry level
        workflow.set_entry_point("generate_topic_node")

        # Movement
        workflow.add_edge("generate_topic_node", "pro_debater_node")
        workflow.add_edge("pro_debater_node", "fact_check_node")
        workflow.add_edge("con_debater_node", "fact_check_node")
        workflow.add_edge("fact_check_node", "fact_check_router_node")
        workflow.add_edge("judge_node", END)
        return workflow



    async def run(self):
        workflow = self._initialize_workflow()
        graph = workflow.compile()
        # graph.get_graph().draw_mermaid_png(output_file_path="workflow_graph.png")
        initial_state = {
            "matter": "",
            "positions": {}
        }
        final_state = await graph.ainvoke(initial_state, config={"recursion_limit": 50})
        return final_state

Let’s break down what’s occurring:

• We initialize a brand new StateGraph with our DebateState sort because the state schema.
• We add every node (agent) to the graph with a reputation. For nodes that want an LLM, we go within the GPT-4o config. For instance, "pro_debater_node" is added as ProDebaterNode(llm_config_map["gpt-4o"]), that means the Professional debater agent will use GPT-4o as its underlying mannequin.
• We set the entry level of the graph to "generate_topic_node". This implies step one of the workflow is to generate a debate matter.
• Then we add directed edges to attach nodes. The sides above encode the first sequence: matter -> professional’s flip -> fact-check -> (then a routing resolution) -> … finally -> decide -> END. We don’t join the Moderator or Truth Test Router with static edges, since these nodes use dynamic instructions to redirect the stream. The ultimate edge connects the decide to an END marker to terminate the graph.

When the workflow runs, management will go alongside these edges so as, however every time we hit a router or moderator node, that node will output a command telling the graph which node to go to subsequent (overriding the default edge). That is how we create conditional loops: the fact_check_router_node would possibly ship us again to a debater node for a retry, as an alternative of following a straight line. LangGraph helps this by permitting nodes to return a particular Command object with goto directions.

In abstract, at a excessive degree we’ve outlined an agentic workflow: a graph of autonomous brokers the place management can department and loop based mostly on the brokers’ outputs. Now, let’s discover what every of those agent nodes truly does.

Agent Nodes Breakdown

Every stage or position within the debate is encapsulated in a node (agent). In LangGraph, nodes are sometimes easy capabilities, however I wished a extra object-oriented method for readability and reusability. So in Deb8flow, each node is a class with a __call__ methodology. All the principle agent courses inherit from a typical BaseComponent for shared performance. This design makes the system modular: we will simply swap out or lengthen brokers by modifying their class definitions, and every agent class is accountable for its piece of the workflow.

Let’s undergo the important thing brokers one after the other.

BaseComponent – A Reusable Agent Base Class

Most of our agent nodes (just like the debaters and decide) share frequent wants: they use an LLM to generate output, they could must retry on errors, and they need to observe token utilization. The BaseComponent class (outlined in <a href="https://github.com/iason-solomos/Deb8flow/blob/principal/nodes/base_component.py">nodes/base_component.py</a>) offers these frequent options so we don’t repeat code.

class BaseComponent:
    """
    A foundational class for managing LLM-based workflows with token monitoring.
    Can deal with each Azure OpenAI (AzureChatOpenAI) and OpenAI (ChatOpenAI).
    """

    def __init__(
        self,
        llm_config: Optionally available[LLMConfig] = None,
        temperature: float = 0.0,
        max_retries: int = 5,
    ):
        """
        Initializes the BaseComponent with elective LLM configuration and temperature.

        Args:
            llm_config (Optionally available[LLMConfig]): Configuration for both Azure or OpenAI.
            temperature (float): Controls the randomness of LLM outputs. Defaults to 0.0.
            max_retries (int): What number of occasions to retry on 429 errors.
        """
        logger = logging.getLogger(self.__class__.__name__)
        tracer = hint.get_tracer(__name__, tracer_provider=get_tracer_provider())

        self.logger = logger
        self.tracer = tracer
        self.llm: Optionally available[ChatOpenAI] = None
        self.output_parser: Optionally available[StrOutputParser] = None
        self.state: Optionally available[DebateState] = None
        self.prompt_template: Optionally available[ChatPromptTemplate] = None
        self.chain: Optionally available[RunnableSequence] = None
        self.paperwork: Optionally available[List] = None
        self.prompt_tokens = 0
        self.completion_tokens = 0
        self.max_retries = max_retries

        if llm_config will not be None:
            self.llm = self._init_llm(llm_config, temperature)
            self.output_parser = StrOutputParser()

    def _init_llm(self, config: LLMConfig, temperature: float):
        """
        Initializes an LLM occasion for both Azure OpenAI or normal OpenAI.
        """
        if isinstance(config, AzureOpenAILLMConfig):
            # If it is Azure, use the AzureChatOpenAI class
            return AzureChatOpenAI(
                deployment_name=config.deployment_name,
                azure_endpoint=config.azure_endpoint,
                openai_api_version=config.openai_api_version,
                openai_api_key=config.openai_api_key,
                temperature=temperature,
            )
        elif isinstance(config, OpenAILLMConfig):
            # If it is normal OpenAI, use the ChatOpenAI class
            return ChatOpenAI(
                model_name=config.model_name,
                openai_api_key=config.openai_api_key,
                temperature=temperature,
            )
        else:
            increase ValueError("Unsupported LLMConfig sort.")

    def validate_initialization(self) -> None:
        """
        Ensures we've an LLM and an output parser.
        """
        if not self.llm:
            increase ValueError("LLM will not be initialized. Guarantee `llm_config` is supplied.")
        if not self.output_parser:
            increase ValueError("Output parser will not be initialized.")

    def execute_chain(self, inputs: Any) -> Any:
        """
        Executes the LLM chain, tracks token utilization, and retries on 429 errors.
        """
        if not self.chain:
            increase ValueError("No chain is initialized for execution.")

        retry_wait = 1  # Preliminary wait time in seconds

        for try in vary(self.max_retries):
            strive:
                with get_openai_callback() as cb:
                    end result = self.chain.invoke(inputs)
                    self.logger.data("Immediate Token utilization: %s", cb.prompt_tokens)
                    self.logger.data("Completion Token utilization: %s", cb.completion_tokens)
                    self.prompt_tokens = cb.prompt_tokens
                    self.completion_tokens = cb.completion_tokens

                return end result

            besides Exception as e:
                # If the error mentions 429, do exponential backoff and retry
                if "429" in str(e):
                    self.logger.warning(
                        f"Fee restrict reached. Retrying in {retry_wait} seconds... "
                        f"(Try {try + 1}/{self.max_retries})"
                    )
                    time.sleep(retry_wait)
                    retry_wait *= 2
                else:
                    self.logger.error(f"Sudden error: {str(e)}")
                    increase e

        increase Exception("API request failed after most variety of retries")

    def create_chain(
        self, system_template: str, human_template: str
    ) -> RunnableSequence:
        """
        Creates a sequence for unstructured outputs.
        """
        self.validate_initialization()
        self.prompt_template = ChatPromptTemplate.from_messages(
            [
                ("system", system_template),
                ("human", human_template),
            ]
        )
        self.chain = self.prompt_template | self.llm | self.output_parser
        return self.chain

    def create_structured_output_chain(
        self, system_template: str, human_template: str, output_model: Kind[BaseModel]
    ) -> RunnableSequence:
        """
        Creates a sequence that yields structured outputs (parsed right into a Pydantic mannequin).
        """
        self.validate_initialization()
        self.prompt_template = ChatPromptTemplate.from_messages(
            [
                ("system", system_template),
                ("human", human_template),
            ]
        )
        self.chain = self.prompt_template | self.llm.with_structured_output(output_model)
        return self.chain

    def build_return_with_tokens(self, node_specific_data: dict) -> dict:
        """
        Comfort methodology so as to add token utilization data into the return values.
        """
        return {
            **node_specific_data,
            "prompt_tokens": self.prompt_tokens,
            "completion_tokens": self.completion_tokens,
        }

    def __call__(self, state: DebateState) -> None:
        """
        Updates the node's native copy of the state.
        """
        self.state = state
        for key, worth in state.objects():
            setattr(self, key, worth)

Key options of BaseComponent:

• It shops an LLM shopper (e.g. an OpenAI ChatOpenAI occasion) initialized with a given mannequin and API key, in addition to an output parser.
• It offers a technique create_chain(system_template, human_template) which units up a LangChain immediate chain (a RunnableSequence) combining a system immediate and a human immediate. This chain is what truly generates outputs when run.
• It has an execute_chain(inputs) methodology that invokes the chain and contains logic to retry if the OpenAI API returns a rate-limit error (HTTP 429). That is finished with exponential backoff as much as a max_retries depend.
• It retains observe of token utilization (immediate tokens and completion tokens) for logging or evaluation.
• The __call__ methodology of BaseComponent (which every subclass will name through tremendous().__call__(state)) can carry out any setup wanted earlier than the node’s principal logic runs (like making certain the LLM is initialized).

By constructing on BaseComponent, every agent class can give attention to its distinctive logic (like what immediate to make use of and tips on how to deal with the state), whereas inheriting the heavy lifting of interacting with GPT-4o reliably.

Matter Generator Agent (GenerateTopicNode)

The Matter Generator (topic_generator_node.py) is the primary agent within the graph. Its job is to provide you with a debatable matter for the session. We give it a immediate that instructs it to output a nuanced matter that might fairly have a professional and con facet.

This agent inherits from BaseComponent and makes use of a immediate chain (system + human immediate) to generate one merchandise of textual content – the talk matter. When known as, it executes the chain (with no particular enter, simply utilizing the immediate) and will get again a topic_text. It then updates the state with:

• debate_topic: the generated matter (stripped of any additional whitespace),
• positions: a dictionary assigning the professional and con stances (by default we use "In favor of the subject" and "In opposition to the subject"),
• stage: set to "opening",
• speaker: set to "professional" (so the Professional facet will communicate first).

In code, the return would possibly appear to be:

return {
    "debate_topic": debate_topic,
    "positions": positions,
    "stage": "opening",
    "speaker": first_speaker  # "professional"
}

Listed below are the prompts for the subject generator:

SYSTEM_PROMPT = """
You're a brainstorming AI that means debate subjects.
You'll present a single, fascinating or well timed matter that may have two opposing views.
"""

HUMAN_PROMPT = """
Please counsel one debate matter for 2 AI brokers to debate.
For instance, it could possibly be about expertise, politics, philosophy, or any fascinating area.
Simply present the subject in a concise sentence.
"""

Then we go these prompts within the constructor of the category itself.

class GenerateTopicNode(BaseComponent):
    def __init__(self, llm_config, temperature: float = 0.7):
        tremendous().__init__(llm_config, temperature)
        # Create the immediate chain.
        self.chain: RunnableSequence = self.create_chain(
            system_template=SYSTEM_PROMPT,
            human_template=HUMAN_PROMPT
        )

    def __call__(self, state: DebateState) -> Dict[str, str]:
        """
        Generates a debate matter and assigns positions to the 2 debaters.
        """
        tremendous().__call__(state)

        topic_text = self.execute_chain({})

        # Retailer the subject and assign stances within the DebateState
        debate_topic = topic_text.strip()
        positions = {
            "professional": "In favor of the subject",
            "con": "In opposition to the subject"
        }

        
        first_speaker = "professional"
        self.logger.data("Welcome to our debate panel! As we speak's debate matter is: %s", debate_topic)
        return {
            "debate_topic": debate_topic,
            "positions": positions,
            "stage": "opening",
            "speaker": first_speaker
        }

It’s a sample we are going to repeat for all courses apart from these not utilizing LLMs and the actual fact checker.

Now we will implement the two stars of the present, the Professional and Con argument brokers!

Debater Brokers (Professional and Con)

Hyperlink: pro_debater_node.py

The 2 debater brokers are very related in construction, however every makes use of totally different immediate templates tailor-made to their position (professional vs con) and the stage of the talk.

The Professional debater, for instance, has to deal with an opening assertion and a counter-argument (countering the Con’s rebuttal). We additionally want logic for retries in case a press release fails fact-check. In code, the ProDebater class units up a number of immediate chains:

• opening_chain and an opening_retry_chain (utilizing barely totally different human prompts – the retry immediate would possibly instruct it to strive once more with out repeating any factually doubtful claims).
• counter_chain and counter_retry_chain for the counter-argument stage.

class ProDebaterNode(BaseComponent):
    def __init__(self, llm_config, temperature: float = 0.7):
        tremendous().__init__(llm_config, temperature)
        self.opening_chain = self.create_chain(SYSTEM_PROMPT, OPENING_HUMAN_PROMPT)
        self.opening_retry_chain = self.create_chain(SYSTEM_PROMPT, OPENING_RETRY_HUMAN_PROMPT)
        self.counter_chain = self.create_chain(SYSTEM_PROMPT, COUNTER_HUMAN_PROMPT)
        self.counter_retry_chain = self.create_chain(SYSTEM_PROMPT, COUNTER_RETRY_HUMAN_PROMPT)

    def __call__(self, state: DebateState) -> Dict[str, Any]:
        tremendous().__call__(state)

        debate_topic = state.get("debate_topic")
        messages = state.get("messages", [])
        stage = state.get("stage")
        speaker = state.get("speaker")

        # Test if retrying (final message was by professional and never validated)
        last_msg = messages[-1] if messages else None
        retrying = last_msg and last_msg["speaker"] == SPEAKER_PRO and never last_msg["validated"]

        if stage == STAGE_OPENING and speaker == SPEAKER_PRO:
            chain = self.opening_retry_chain if retrying else self.opening_chain # choose which chain we're triggering: the traditional one or the fact-cehcked one
            end result = chain.invoke({
                "debate_topic": debate_topic
            })
        elif stage == STAGE_COUNTER and speaker == SPEAKER_PRO:
            opponent_msg = self._get_last_message_by(SPEAKER_CON, messages)
            debate_history = get_debate_history(messages)
            chain = self.counter_retry_chain if retrying else self.counter_chain
            end result = chain.invoke({
                "debate_topic": debate_topic,
                "opponent_statement": opponent_msg,
                "debate_history": debate_history
            })
        else:
            increase ValueError(f"Unknown flip for ProDebater: stage={stage}, speaker={speaker}")
        new_message = create_debate_message(speaker=SPEAKER_PRO, content material=end result, stage=stage)
        self.logger.data("Speaker: %s, Stage: %s, Retry: %snMessage:npercents", speaker, stage, retrying, end result)
        return {
            "messages": messages + [new_message]
        }

    def _get_last_message_by(self, speaker_prefix, messages):
        for m in reversed(messages):
            if m.get("speaker") == speaker_prefix:
                return m["content"]
        return ""

When the ProDebater’s __call__ runs, it appears to be like on the present stage and speaker within the state to resolve what to do:

• If it’s the opening stage and the speaker is “professional”, it makes use of the opening_chain to generate a gap argument. If the final message from Professional was marked invalid (not validated), it is aware of this can be a retry, so it will use the opening_retry_chain as an alternative.
• If it’s the counter stage and speaker is “professional”, it generates a counter-argument to regardless of the opponent (Con) simply stated. It’s going to fetch the final message by the Con from the messages historical past, and feed that into the immediate (in order that the Professional can immediately counter it). Once more, if the final Professional message was invalid, it will swap to the retry chain.

After producing its argument, the Debater agent creates a brand new message entry (with speaker="professional", the content material textual content, validated=False initially, and the stage) and appends it to the state’s message checklist. That turns into the output of the node (LangGraph will merge this partial state replace into the worldwide state).

The Con Debater agent mirrors this logic for its levels:

It equally appends its message to the state.

It has a rebuttal and closing argument (ultimate argument) stage, every with a traditional and a retry chain.

It checks if it’s the rebuttal stage (speaker “con”) or ultimate argument stage (speaker “con”) and invokes the suitable chain, presumably utilizing the final Professional message for context when rebutting.

con_debater_node.py

Through the use of class-based implementation, our debaters’ code is simpler to take care of. We are able to clearly separate what the Professional does vs what the Con does, even when they share construction. Additionally, by encapsulating immediate chains inside the category, every debater can handle a number of potential outputs (common vs retry) cleanly.

Immediate design: The precise prompts (in prompts/pro_debater_prompts.py and con_debater_prompts.py) information the GPT-4o mannequin to tackle a persona (“You’re a debater arguing for/towards the subject…”) and produce the argument. Additionally they instruct the mannequin to maintain statements factual and logical. If a reality verify fails, the retry immediate could say one thing like: “Your earlier assertion had an unverified declare. Revise your argument to be factually appropriate whereas sustaining your place.” – encouraging the mannequin to appropriate itself.

With this, our AI debaters can have interaction in a multi-turn duel, and even recuperate from factual missteps.

Truth Checker Agent (FactCheckNode)

After every debater speaks, the Truth Checker agent swoops in to confirm their claims. This agent is carried out in <a href="https://github.com/iason-solomos/Deb8flow/blob/principal/nodes/fact_checker_node.py">fact_checker_node.py</a>, and curiously, it makes use of the GPT-4o mannequin’s searching skill moderately than our personal customized prompts. Basically, we delegate the fact-checking to OpenAI’s GPT-4 with internet search.

How does this work? The OpenAI Python shopper for GPT-4 (with searching) permits us to ship a consumer message and get a structured response. In FactCheckNode.__call__, we do one thing like:

completion = self.shopper.beta.chat.completions.parse(
            mannequin="gpt-4o-search-preview",
            web_search_options={},
            messages=[{
                "role": "user",
                "content": (
                        f"Consider the following statement from a debate. "
                        f"If the statement contains numbers, or figures from studies, fact-check it online.nn"
                        f"Statement:n"{claim}"nn"
                        f"Reply clearly whether any numbers or studies might be inaccurate or hallucinated, and why."
                        f"n"
                        f"If the statement doesn't contain references to studies or numbers cited, don't go online to fact-check, and just consider it successfully fact-checked, with a 'yes' score.nn"
                )
            }],
            response_format=FactCheck
        )

If the result’s “sure” (that means the declare appears truthful or a minimum of not factually fallacious), the Truth Checker will mark the final message’s validated area as True within the state, and output {"validated": True} with no additional adjustments. This indicators that the talk can proceed usually.

If the result’s “no” (that means it discovered the declare to be incorrect or doubtful), the Truth Checker will append a brand new message to the state with speaker="fact_checker" describing the discovering (or we may merely mark it, however offering a short observe like “(Truth Checker: The statistic cited couldn’t be verified.)” will be helpful). It’s going to additionally set validated: False and increment a counter for whichever facet made the declare. The output state from this node contains validated: False and an up to date times_pro_fact_checked or times_con_fact_checked depend.

We additionally use a Pydantic BaseModel to regulate the output of the LLM:

class FactCheck(BaseModel):
    """
    Pydantic mannequin for the actual fact checking the claims made by debaters.

    Attributes:
        binary_score (str): 'sure' if the declare is verifiable and truthful, 'no' in any other case.
    """

    binary_score: str = Area(
        description="Signifies if the declare is verifiable and truthful. 'sure' or 'no'."
    )
    justification: str = Area(
        description="Rationalization of the reasoning behind the rating."
    )

Debate Moderator Agent (DebateModeratorNode)

The Debate Moderator is the conductor of the talk. As a substitute of manufacturing prolonged textual content, this agent’s job is to handle turn-taking and stage development. Within the workflow, after a press release is validated by the Truth Checker, management passes to the Moderator node. The Moderator then points a Command that updates the state for the following flip and directs the stream to the suitable subsequent agent.

The logic in DebateModeratorNode.__call__ (see <a href="https://github.com/iason-solomos/Deb8flow/blob/principal/nodes/debate_moderator_node.py">nodes/debate_moderator_node.py</a>) goes roughly like this:

if stage == STAGE_OPENING and speaker == SPEAKER_PRO:
            return Command(
                replace={"stage": STAGE_REBUTTAL, "speaker": SPEAKER_CON},
                goto=NODE_CON_DEBATER
            )
        elif stage == STAGE_REBUTTAL and speaker == SPEAKER_CON:
            return Command(
                replace={"stage": STAGE_COUNTER, "speaker": SPEAKER_PRO},
                goto=NODE_PRO_DEBATER
            )
        elif stage == STAGE_COUNTER and speaker == SPEAKER_PRO:
            return Command(
                replace={"stage": STAGE_FINAL_ARGUMENT, "speaker": SPEAKER_CON},
                goto=NODE_CON_DEBATER
            )
        elif stage == STAGE_FINAL_ARGUMENT and speaker == SPEAKER_CON:
            return Command(
                replace={},
                goto=NODE_JUDGE
            )

        increase ValueError(f"Sudden stage/speaker combo: stage={stage}, speaker={speaker}")

Every conditional corresponds to some extent within the debate the place a flip simply ended, and units up the following flip. For instance, after the opening (Professional simply spoke), it units stage to rebuttal, switches speaker to Con, and directs the workflow to the Con debater node​. After the final_argument (Con’s closing), it directs to the Choose with no additional replace (the talk stage successfully ends).

Truth Test Router (FactCheckRouterNode)

That is one other management node (just like the Moderator) that introduces conditional logic. The Truth Test Router sits proper after the Truth Checker agent within the stream. Its goal is to department the workflow relying on the fact-check end result.

In <a href="https://github.com/iason-solomos/Deb8flow/blob/principal/nodes/fact_check_router_node.py">nodes/fact_check_router_node.py</a>, the logic is:

if pro_fact_checks >= 3 or con_fact_checks >= 3:
            disqualified = SPEAKER_PRO if pro_fact_checks >= 3 else SPEAKER_CON
            winner = SPEAKER_CON if disqualified == SPEAKER_PRO else SPEAKER_PRO

            verdict_msg = {
                "speaker": "moderator",
                "content material": (
                    f"Debate ended early as a result of extreme factual inaccuracies.nn"
                    f"DISQUALIFIED: {disqualified.higher()} (exceeded reality verify restrict)n"
                    f"WINNER: {winner.higher()}"
                ),
                "validated": True,
                "stage": "verdict"
            }
            return Command(
                replace={"messages": messages + [verdict_msg]},
                goto=END
            )
        if last_message.get("validated"):
            return Command(goto=NODE_DEBATE_MODERATOR)
        elif speaker == SPEAKER_PRO:
            return Command(goto=NODE_PRO_DEBATER)
        elif speaker == SPEAKER_CON:
            return Command(goto=NODE_CON_DEBATER)
        increase ValueError("Unable to find out routing in FactCheckRouterNode.")

First, the Truth Test Router checks if both facet’s fact-check depend has reached 3. In that case, it creates a Moderator-style message saying an early finish: the offending facet is disqualified and the opposite facet is the winner​. It appends this verdict to the messages and returns a Command that jumps to END, successfully terminating the talk with out going to the Choose (as a result of we already know the end result).

If we’re not ending the talk early, it then appears to be like on the Truth Checker’s end result for the final message (which is saved as validated on that message). If validated is True, we go to the talk moderator: Command(goto=debate_moderator_node).

Else if the assertion fails fact-check, the workflow goes again to the debater to supply a revised assertion (with the state counters up to date to mirror the failure). This loop can occur a number of occasions if wanted (as much as the disqualification restrict).

This dynamic management is the center of Deb8flow’s “agentic” nature – the flexibility to adapt the trail of execution based mostly on the content material of the brokers’ outputs. It showcases LangGraph’s energy: combining management stream with state. We’re basically encoding debate guidelines (like permitting retries for false claims, or ending the talk if somebody cheats too usually) immediately into the workflow graph.

Choose Agent (JudgeNode)

Final however not least, the Choose agent delivers the ultimate verdict based mostly on rhetorical ability, readability, construction, and general persuasiveness. Its system immediate and human immediate make this specific:

• System Immediate: “You’re an neutral debate decide AI. … Consider which debater introduced their case extra clearly, persuasively, and logically. You could give attention to communication expertise, construction of argument, rhetorical energy, and general coherence.”
• Human Immediate: “Right here is the total debate transcript. Please analyze the efficiency of each debaters—PRO and CON. Consider rhetorical efficiency—readability, construction, persuasion, and relevance—and resolve who introduced their case extra successfully.”

When the Choose node runs, it receives the complete debate transcript (all validated messages) alongside the unique matter. It then makes use of GPT-4o to look at how both sides framed their arguments, dealt with counterpoints, and supported (or didn’t help) claims with examples or logic. Crucially, the Choose is forbidden to guage which place is objectively appropriate (or who it thinks may be appropriate)—solely who argued extra persuasively.

Under is an instance ultimate verdict from a Deb8flow run on the subject:
“Ought to governments implement a common primary revenue in response to growing automation within the workforce?”

WINNER: PRO

REASON: The PRO debater introduced a extra compelling and rhetorically efficient case for common primary revenue. Their arguments had been well-structured, starting with a transparent assertion of the difficulty and the need of UBI in response to automation. They successfully addressed potential counterarguments by highlighting the unprecedented velocity and scope of present technological adjustments, which distinguishes the present scenario from previous technological shifts. The PRO additionally supplied empirical proof from UBI pilot packages to counter the CON's claims about work disincentives and financial inefficiencies, reinforcing their argument with real-world examples.

In distinction, the CON debater, whereas presenting legitimate issues about UBI, relied closely on historic analogies and assumptions about workforce adaptability with out adequately addressing the distinctive challenges posed by trendy automation. Their arguments concerning the fiscal burden and potential inefficiencies of UBI had been much less supported by particular proof in comparison with the PRO's rebuttals.

General, the PRO's arguments had been extra coherent, persuasive, and backed by empirical proof, making their case extra convincing to a impartial observer.

Langsmith Tracing

All through Deb8flow’s growth, I relied on LangSmith (LangChain’s tracing and observability toolkit) to make sure the complete debate pipeline was behaving appropriately. As a result of we’ve a number of brokers passing management between themselves, it’s straightforward for sudden loops or misrouted states to happen. LangSmith offers a handy solution to:

• Visualize Execution Movement: You’ll be able to see every agent’s immediate, the tokens consumed (so you may as well observe prices), and any intermediate states. This makes it a lot easier to substantiate that, say, the Con Debater is correctly referencing the Professional Debater’s final message, or that the Truth Checker is precisely receiving the declare to confirm.
• Debug State Updates: If the Moderator or Truth Test Router is sending the stream to the fallacious node, the hint will spotlight that mismatch. You’ll be able to hint which agent was invoked at every step and why, serving to you notice stage or speaker misalignments early.
• Monitor Immediate and Completion Tokens: With a number of GPT-4o calls, it’s helpful to see what number of tokens every stage is utilizing, which LangSmith logs routinely in case you allow tracing.

Integrating LangSmith is unexpectedly straightforward. You’ll simply want to supply these 3 keys in your .env file: LANGCHAIN_API_KEY

LANGCHAIN_TRACING_V2

LANGCHAIN_PROJECT

Then you possibly can open the LangSmith UI to see a structured hint of every run. This vastly reduces the guesswork concerned in debugging multi-agent techniques and is, in my expertise, important for extra advanced AI orchestration like ours. Instance of a single run:

Reflections and Subsequent Steps

Constructing Deb8flow was an eye-opening train in orchestrating autonomous agent workflows. We didn’t simply chain a single mannequin name – we created a whole debate simulation with AI brokers, every with a particular position, and allowed them to work together in line with a algorithm. LangGraph supplied a transparent framework to outline how knowledge and management flows between brokers, making the advanced sequence manageable in code. Through the use of class-based brokers and a shared state, we maintained modularity and readability, which is able to repay for any software program engineering venture in the long term.

An thrilling facet of this venture was seeing emergent habits. Although every agent follows a script (a immediate), the unscripted mixture – a debater attempting to deceive, a fact-checker catching it, the debater rephrasing – felt surprisingly lifelike! It’s a small step towards extra Agentic Ai techniques that may carry out non-trivial multi-step duties with oversight on one another.

There’s loads of concepts for enchancment:

• Person Interplay: At the moment it’s totally autonomous, however one may add a mode the place a human offers the subject and even takes the position of 1 facet towards an AI opponent.
• We are able to swap the order through which the Debaters speak.
• We are able to change the prompts, and thus to a superb diploma the habits of the brokers, and experiment with totally different prompts.
• Make the debaters additionally carry out internet search earlier than producing their statements, thus offering them with the newest data.

The broader implication of Deb8flow is the way it showcases a sample for composable AI brokers. By defining clear boundaries and interactions (similar to microservices in software program), we will have advanced AI-driven processes that stay interpretable and controllable. Every agent is sort of a cog in a machine, and LangGraph is the gear system making them work in unison.

I discovered this venture energizing, and I hope it conjures up you to discover multi-agent workflows. Whether or not it’s debating, collaborating on writing, or fixing issues from totally different knowledgeable angles, the mixture of GPT, instruments, and structured agentic workflows opens up a brand new world of prospects for AI growth. Comfortable hacking!

References

[1] D. Bouchard, “From Basics to Advanced: Exploring LangGraph,” Medium, Nov. 22, 2023. [Online]. Available: https://medium.com/data-science/from-basics-to-advanced-exploring-langgraph-e8c1cf4db787. [Accessed: Apr. 1, 2025].

[2] A. W. T. Ng, “Constructing a Analysis Agent that Can Write to Google Docs: Half 1,” In direction of Information Science, Jan. 11, 2024. [Online]. Obtainable: https://towardsdatascience.com/building-a-research-agent-that-can-write-to-google-docs-part-1-4b49ea05a292/. [Accessed: Apr. 1, 2025].