Using the ReAct Framework in LangChain

Both ways: Using off-the-shelf agents and LCEL

ReAct Framework for Prompting

ReAct, which stands for Reasoning + Acting, is a prompting technique developed by professors from Princeton University in collaboration with Google researchers.

The method is designed to enhance the reasoning capabilities of LLMs, allowing them to reason and act intelligently within simulated environments.

ReAct aims to enable LLMs to mimic human-like operations in the real world, where we reason verbally and take actions to gain information. This advancement holds significant potential for reshaping how we interact with language models.

The ReAct framework is flexible and designed to synergize the reasoning abilities of LLMs with actionable outputs.

It allows LLMs to interact with external tools, enhancing their decision-making processes. With React, LLMs can understand and generate text, make informed decisions and take actions based on their understanding.

How Does It Work?

ReAct combines reasoning and acting to solve complex language reasoning and decision-making tasks.

While Chain-of-thought (CoT) prompting has shown that LLMs can generate reasoning traces for tasks like arithmetic and commonsense reasoning, its lack of access to the external world can lead to issues like fact hallucination. ReAct addresses this by allowing LLMs to generate verbal reasoning traces and actions for a task.

Language models can generate responses based on their input, but ReAct enhances this by allowing the model to interact with its environment.

This interaction is achieved through text actions that the model can use to ask questions or perform tasks to gain more information and better understand a situation. For instance, when faced with a multi-hop reasoning question, ReAct might initiate multiple search actions, each potentially being a call to an external tool.

The results of these actions are then used to generate a final answer.

A ReAct prompt might include examples of actions, the observations resulting from those actions, and the transcribed thoughts of a human during the process.

By prompting the LLM to alternate between thinking and acting, ReAct transforms it into an active agent in its environment, capable of solving tasks in a human-aligned manner.

When Should This Be Used?

ReAct is ideal when a language model needs to interact with its environment to gather more information or when a task requires multiple reasoning steps.

It excels in knowledge-intensive reasoning tasks like multi-hop question answering using Wikipedia passages or fact verification. Additionally, ReAct is beneficial in decision-making tasks where the model navigates simulated environments, such as online shopping websites or text-based games.

One of the significant advantages of ReAct is its ability to integrate LLMs with external tools for real-world applications. For instance, Microsoft has integrated OpenAI LLMs with their Office applications in Microsoft 365 Copilot, showcasing its utility.

In scenarios like QA systems, where LLMs might not always provide correct answers, the interaction with an external search engine becomes crucial, and ReAct proves invaluable.

How Is It Different from Other Prompting Methods?

Traditional prompting methods predominantly rely on the LLM’s internal knowledge.

ReAct’s strength is also its potential weakness.

Its dynamic approach to prompting heavily relies on integrating external tools. This means that the effectiveness of ReAct is partly dependent on the capabilities and reliability of these external tools.

As with any system that relies on external data, there’s always a risk of introducing biases or inaccuracies from those sources.

Preliminaries

Let’s get this out of the way…

%%capture
!pip install langchain openai duckduckgo-search youtube_search wikipedia langchainhub

import os
import getpass

os.environ["OPENAI_API_KEY"] = getpass.getpass("Enter Your OpenAI API Key:")

from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType
from langchain.chat_models import ChatOpenAI

from langchain import hub
from langchain.agents.format_scratchpad import format_log_to_str
from langchain.agents.output_parsers import ReActSingleInputOutputParser
from langchain.tools.render import render_text_description

ReAct Agents in LangChain

The ReAct agent in LangChain is a versatile agent that utilizes the ReAct framework to select the appropriate tool based on its description.

This agent is the most general-purpose action agent available in LangChain and can be highly beneficial in situations where multiple tools are available, and selecting the right tool is time-consuming. The ReAct agent allows you to specify a description for each tool and then automatically selects the appropriate tool based on the description.

This feature simplifies determining which tool to use in a given situation.

How it works:

1. The system receives input from the user.
2. Based on the input, the agent decides whether to use a tool and, if so, what the input to that tool should be.
3. The tool is then called with the appropriate input, and the output of the tool is recorded as an observation.
4. The history of the tool, tool input, and observation is passed back into the agent, which decides the next step.
5. The above steps are repeated until the agent determines that no further tools are needed. At this point, it will respond directly to the user.

#  load the language model we're using to control the agent
llm = ChatOpenAI(model = "gpt-4-1106-preview",temperature=0)

# load some tools
tools = load_tools(['wikipedia', 'ddg-search','llm-math'], llm=llm)

# initialize agent with the tools, language model, and the type of agent
agent = initialize_agent(tools,
                         llm,
                         agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
                         verbose=True
                         )

You can inspect the prompt that the ReAct agent uses. It’s pretty much as described in the paper, but with no examples. That’s why it’s called ZERO_SHOT_REACT_DESCRIPTION.

print(agent.agent.llm_chain.prompt.template)

Answer the following questions as best you can. You have access to the following tools:

Wikipedia: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, facts, historical events, or other subjects. Input should be a search query.
duckduckgo_search: A wrapper around DuckDuckGo Search. Useful for when you need to answer questions about current events. Input should be a search query.
Calculator: Useful for when you need to answer questions about math.

Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [Wikipedia, duckduckgo_search, Calculator]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question

Begin!

Question: {input}
Thought:{agent_scratchpad}

If you wanted to, you could just write your prompt and have a few shot examples. It will improve the performance of your agent for your particular use case.

You would just rewrite the prompt and set it as show below:

prompt = """
Answer the following questions as best you can. You have access to the following tools:

Wikipedia: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, facts, historical events, or other subjects. Input should be a search query.
duckduckgo_search: A wrapper around DuckDuckGo Search. Useful for when you need to answer questions about current events. Input should be a search query.
Calculator: Useful for when you need to answer questions about math.

Use the following format:

Question: the input question you must answer
Thought: you should think step by step what actions you need to take
Action: the action to take, should be one of [Wikipedia, duckduckgo_search, Calculator]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question

Begin!

Question: {input}
Thought:{agent_scratchpad}

"""

# set the redefined prompt
agent.agent.llm_chain.prompt.template = prompt

tools[1].description = "Useful for when you need to to determine the current date."
tools[2].description = "Useful for when you need to perfom calculations."

agent.run("Who is Diljit Dosanjh and what is his age as of today? What is that raised to the 0.43 power?")

And the result:

> Entering new AgentExecutor chain...
To answer the first part of the question, I need to find out who Diljit Dosanjh is and his date of birth to calculate his current age.
Action: Wikipedia
Action Input: Diljit Dosanjh
Observation: Page: Diljit Dosanjh
Summary: Diljit Dosanjh (born 6 January 1984) is an Indian singer, songwriter, actor, film producer and television personality. He works in Punjabi Music and subsequently in Punjabi and Hindi cinema. Dosanjh entered Social 50 chart by Billboard in 2020. He is featured in various music charts including Canadian Albums Chart, UK Asian chart by Official Charts Company  and New Zealand Hot Singles. His movies, including Jatt & Juliet 2, Punjab 1984, Sajjan Singh Rangroot and Honsla Rakh are among the highest grossing Punjabi films in history.Hailing from Dosanjh Kalan, Jalandhar, Dosanjh started in 2002 and gained recognition in punjabi music with his album ‘Smile’ (2005) and ‘Chocolate’ (2008) followed by ‘The Next Level’ (2009) with Yo Yo Honey Singh. Subsequently he pursued his acting, starring in debut punjabi movie The Lion of Punjab in 2011.He made his Bollywood debut in 2016 with the crime thriller Udta Punjab for which he earned the Filmfare Award for Best Male Debut, in addition for a nomination for the Filmfare Award for Best Supporting Actor. This was followed by Good Newwz (2019), for which he received his second nomination for the Filmfare Award for Best Supporting Actor. As of 2020, he has won the most five PTC Award for Best Actor. He has also appeared as a judge in three seasons of the reality show Rising Star. In 2020, Dosanjh charted on the Social 50 chart by Billboard, with the release of his 11th album G.O.A.T.. The album, along with his next one MoonChild Era were listed in the Billboard Canadian Albums Chart. He became the first Punjabi artist to perform at Coachella Valley Music and Arts Festival in 2023.

Page: Diljit Dosanjh discography
Summary: Indian singer Diljit Dosanjh has released 13 studio albums, one extended plays, and 41 singles. In 2020, he entered the Social 50 chart by Billboard, following the release of his 11th album G.O.A.T. The album also entered the top 20 in the Canadian Albums Chart. His 12th album MoonChild Era charted at number 32 on the Canadian Albums Chart.

Page: G.O.A.T. (Diljit Dosanjh album)
Summary: G.O.A.T. is the eleventh studio album by Indian-Punjabi vocalist Diljit Dosanjh, released on July 30, 2020, by Famous Studios. Title track of the album and its music video was released a day prior to album release. Songs were written by Karan Aujla, Raj Ranjodh, Amrit Maan, and other artists. The album was produced by various artists including Desi Crew and The Kidd.
Thought:Diljit Dosanjh is an Indian singer, songwriter, actor, film producer, and television personality born on January 6, 1984. To calculate his current age, I need to subtract his birth year from the current year and then account for whether his birthday has passed this year or not.

Action: Calculator
Action Input: 2023 - 1984
Observation: Answer: 39
Thought:Since today is after January 6, 2023, Diljit Dosanjh has already celebrated his birthday this year, so he is 39 years old. Now I need to calculate 39 raised to the power of 0.43.

Action: Calculator
Action Input: 39^0.43
Observation: Answer: 4.832343312281572
Thought:I have calculated Diljit Dosanjh's age as 39 years old and raised that to the power of 0.43 to get the result.
Final Answer: Diljit Dosanjh is an Indian singer, songwriter, actor, film producer, and television personality, and as of today, he is 39 years old. The value of 39 raised to the 0.43 power is approximately 4.832.

> Finished chain.
Diljit Dosanjh is an Indian singer, songwriter, actor, film producer, and television personality, and as of today, he is 39 years old. The value of 39 raised to the 0.43 power is approximately 4.832.

ReAct Document Store

The DocstoreExplorer is a class in LangChain that is used for working with document stores.

It’s specifically designed to implement the ReAct logic for document stores.

You can use the DocstoreExplorer for tasks such as searching and looking up documents in the document store.

It provides two useful tools: “Search” and “Lookup”.

The “Search” tool is used when you need to ask a question with a search, while the “Lookup” tool is used when you need to ask a question with a lookup.

To use the DocstoreExplorer, you need to initialize an agent with the necessary tools and language model.

from langchain import OpenAI, Wikipedia
from langchain.document_loaders import WebBaseLoader
from langchain.agents import initialize_agent, Tool
from langchain.agents import AgentType
from langchain.agents.react.base import DocstoreExplorer

docstore = DocstoreExplorer(Wikipedia())

search_tool = Tool(name="Search",
                   func=docstore.search,
                   description="Useful for when you need to ask with search"
                   )

lookup_tool = Tool(name="Lookup",
                   func=docstore.lookup,
                   description="Useful for when yuo need to ask with lookup"
                   )

tools = [search_tool, lookup_tool]

llm = OpenAI(temperature=0)

react = initialize_agent(tools,
                         llm,
                         agent=AgentType.REACT_DOCSTORE,
                         verbose=True)

question = "Who did Tupac Shakur have beef with over east vs west coast?"
react.run(question)

> Entering new AgentExecutor chain...
Thought: I need to search Tupac Shakur and find who he had beef with over east vs west coast.
Action: Search[Tupac Shakur]
Observation: Tupac Amaru Shakur ( TOO-pahk shə-KOOR; born Lesane Parish Crooks; June 16, 1971 – September 13, 1996), also known by his stage names 2Pac and Makaveli, was an American rapper. He is widely considered one of the most influential and successful rappers of all time. Shakur is among the best-selling music artists, having sold more than 75 million records worldwide. Much of Shakur's music has been noted for addressing contemporary social issues that plagued inner cities.
Shakur was born in New York City to parents who were both political activists and Black Panther Party members. Raised by his mother, Afeni Shakur, he relocated to Baltimore in 1984 and to the San Francisco Bay Area in 1988. With the release of his debut album 2Pacalypse Now in 1991, he became a central figure in West Coast hip hop for his conscious rap lyrics. Shakur achieved further critical and commercial success with his follow-up albums Strictly 4 My N.I.G.G.A.Z... (1993) and Me Against the World (1995). His Diamond certified album All Eyez on Me (1996), the first double-length album in hip-hop history, abandoned his introspective lyrics for volatile gangsta rap. In addition to his music career, Shakur also found considerable success as an actor, with his starring roles in Juice (1992), Poetic Justice (1993), Above the Rim (1994), Bullet (1996), Gridlock'd (1997), and Gang Related (1997).
During the later part of his career, Shakur was shot five times in the lobby of a New York recording studio and experienced legal troubles, including incarceration. Shakur served eleven months in prison on sexual abuse charges, but was released pending an appeal of his conviction in 1995. Following his release, he signed to Marion "Suge" Knight's label Death Row Records and became heavily involved in the growing East Coast–West Coast hip hop rivalry. On September 7, 1996, Shakur was shot four times by an unidentified assailant in a drive-by shooting in Las Vegas; he died six days later. Following his murder, Shakur's friend-turned-rival, the Notorious B.I.G., was at first considered a suspect due to their public feud; he was also murdered in another drive-by shooting six months later in March 1997, while visiting Los Angeles.Shakur's double-length posthumous album Greatest Hits (1998) is one of his two releases—and one of only nine hip hop albums—to have been certified Diamond in the United States. Five more albums have been released since Shakur's death, including his critically acclaimed posthumous album The Don Killuminati: The 7 Day Theory (1996) under his stage name Makaveli, all of which have been certified Platinum in the United States. In 2002, Shakur was inducted into the Hip-Hop Hall of Fame. In 2017, he was inducted into the Rock and Roll Hall of Fame in his first year of eligibility. Rolling Stone ranked Shakur among the 100 Greatest Artists of All Time. In 2023, he was awarded a posthumous star on the Hollywood Walk of Fame.
Thought: Tupac Shakur had beef with Notorious B.I.G. over east vs west coast.
Action: Finish[Notorious B.I.G.]

> Finished chain.
Notorious B.I.G.

ReAct Agents in LCEL

It’s essentially the same pattern as above, but using the expression language.

prompt = hub.pull("hwchase17/react")

prompt = prompt.partial(
    tools=render_text_description(tools),
    tool_names=", ".join([t.name for t in tools]),
)

llm_with_stop = llm.bind(stop=["\nObservation"])

agent = (
    {
        "input": lambda x: x["input"],
        "agent_scratchpad": lambda x: format_log_to_str(x["intermediate_steps"]),
    }
    | prompt
    | llm_with_stop
    | ReActSingleInputOutputParser()
)

Explaining ReActSingleInputOutputParser

🛠️ Instantiation: When you use ReActSingleInputOutputParser(), you create an instance of this parser, all set to interpret outputs from a Large Language Model (LLM).

Functionality: It’s designed to understand two types of LLM outputs:

• 🎯 Action-oriented outputs: Identifies actions and their inputs, creating an AgentAction object.
• 🏁Final answer outputs: Recognizes conclusive answers, resulting in an AgentFinish object .

🕵️‍♂️Usage: You’ll typically pass the LLM’s text output to this parser. It then determines if the output is for an action or a final answer, extracting relevant details .

🚨Error Handling: If the LLM’s output doesn’t match expected formats or misses key components, the parser raises exceptions to alert you .

In essence, ReActSingleInputOutputParser() is your go-to tool for smartly parsing and interpreting LLM responses, especially when they’re formatted for specific actions or definitive answers.

from langchain.agents import AgentExecutor

agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

agent_executor.invoke(
    {
        "input": "Who is Diljit Dosanjh and what is his age as of today? What is that raised to the 0.43 power?"
    }
)

Entering new AgentExecutor chain...
 I need to find out who Diljit Dosanjh is and his age, then calculate the power.
Action: Search
Action Input: "Diljit Dosanjh"Diljit Dosanjh (born 6 January 1984) is an Indian singer, songwriter, actor, film producer and television personality. He works in Punjabi Music and subsequently in Punjabi and Hindi cinema. Dosanjh entered Social 50 chart by Billboard in 2020. He is featured in various music charts including Canadian Albums Chart, UK Asian chart by Official Charts Company  and New Zealand Hot Singles. His movies, including Jatt & Juliet 2, Punjab 1984, Sajjan Singh Rangroot and Honsla Rakh are among the highest grossing Punjabi films in history.Hailing from Dosanjh Kalan, Jalandhar, Dosanjh started in 2002 and gained recognition in punjabi music with his album ‘Smile’ (2005) and ‘Chocolate’ (2008) followed by ‘The Next Level’ (2009) with Yo Yo Honey Singh. Subsequently he pursued his acting, starring in debut punjabi movie The Lion of Punjab in 2011.He made his Bollywood debut in 2016 with the crime thriller Udta Punjab for which he earned the Filmfare Award for Best Male Debut, in addition for a nomination for the Filmfare Award for Best Supporting Actor. This was followed by Good Newwz (2019), for which he received his second nomination for the Filmfare Award for Best Supporting Actor. As of 2020, he has won the most five PTC Award for Best Actor. He has also appeared as a judge in three seasons of the reality show Rising Star. In 2020, Dosanjh charted on the Social 50 chart by Billboard, with the release of his 11th album G.O.A.T.. The album, along with his next one MoonChild Era were listed in the Billboard Canadian Albums Chart. He became the first Punjabi artist to perform at Coachella Valley Music and Arts Festival in 2023. I now know who Diljit Dosanjh is and his age.
Action: Lookup
Action Input: "Diljit Dosanjh age"No Results I need to calculate the power.
Action: Search
Action Input: "Raise to the 0.43 power"Survivor 43 is the forty-third season of the American reality television series Survivor. The show was filmed from May 2 through May 27, 2022, in Fiji, for an eleventh consecutive season; it premiered on September 21, 2022 on CBS in the United States and Global in Canada. The season concluded on December 14, 2022; Mike Gabler was named the winner of the season, defeating Cassidy Clark and Owen Knight in a 7–1–0 vote. Gabler, aged 51, was the second oldest winner, after Bob Crowley, who was 57 years old at the time of winning Survivor: Gabon. Gabler has stated he intends on donating the one million dollar prize to charity. This is the first season in Survivor history where the oldest contestant of the season was the winner. I now know the final answer.
Final Answer: Diljit Dosanjh is an Indian singer, songwriter, actor, film producer and television personality born on 6 January 1984. Raising his age to the 0.43 power results in Survivor 43, the forty-third season of the American reality television series Survivor.

> Finished chain.
{'input': 'Who is Diljit Dosanjh and what is his age as of today? What is that raised to the 0.43 power?',
 'output': 'Diljit Dosanjh is an Indian singer, songwriter, actor, film producer and television personality born on 6 January 1984. Raising his age to the 0.43 power results in Survivor 43, the forty-third season of the American reality television series Survivor.'}

Conclusion

To wrap up, this blog has explored the ReAct framework’s role in elevating LLMs’ reasoning and action capabilities.

We’ve seen how ReAct empowers LLMs to engage with their environment in a more human-like manner, enhancing decision-making and problem-solving. The integration of ReAct agents in LangChain, with their tool-selection intelligence, showcases practical applications in complex scenarios. The insights into the ReAct Document Store and the ReActSingleInputOutputParser highlight the framework’s effectiveness in processing and responding to complex queries.

Overall, ReAct represents a significant advancement in the functionality and versatility of language models, opening new possibilities for their application in various fields.