【虹科分享】基于Redis Enterprise，LangChain，OpenAI 构建一个电子商务聊天机器人

pip install langchain==0.0.123


pip install openai==0.27.2


pip install redis==4.5.3


pip install numpy


pip install pandas


pip install gdown

import pandas as pd


 


MAX_TEXT_LENGTH=1000  # Maximum num of text characters to use


 


def auto_truncate(val):


 


    """Truncate the given text."""


 


    return val[:MAX_TEXT_LENGTH]


 


# Load Product data and truncate long text fields


 


all_prods_df = pd.read_csv("product_data.csv", converters={


 


    'bullet_point': auto_truncate,


 


    'item_keywords': auto_truncate,


 


    'item_name': auto_truncate


 


})

# Replace empty strings with None and drop


 


all_prods_df['item_keywords'].replace('', None, inplace=True)


 


all_prods_df.dropna(subset=['item_keywords'], inplace=True)


 


# Reset pandas dataframe index


 


all_prods_df.reset_index(drop=True, inplace=True)

# Num products to use (subset)


NUMBER_PRODUCTS = 2500  


 


# Get the first 2500 products


product_metadata = (


    all_prods_df


     .head(NUMBER_PRODUCTS)


     .to_dict(orient='index')


)


 


# Check one of the products


product_metadata[0]

{'item_id': 'B07T2JY31Y',


 'marketplace': 'Amazon',


 'country': 'IN',


 'main_image_id': '71vX7qIEAIL',


 'domain_name': 'amazon.in',


 'bullet_point': '3D Printed Hard Back Case Mobile Cover for Sony Xperia Z1 L39H Easy to put & take off with perfect cutouts for volume buttons, audio & charging ports. Stylish design and appearance, express your unique personality. Extreme precision design allows easy access to all buttons and ports while featuring raised bezel to life screen and camera off flat surface. Slim Hard Back Cover No Warranty',


 'item_keywords': 'mobile cover back cover mobile case phone case mobile panel phone panel LG mobile case LG phone cover LG back case hard case 3D printed mobile cover mobile cover back cover mobile case phone case mobile panel phone panel Sony Xperia mobile case Sony Xperia phone cover Sony Xperia back case hard case 3D printed mobile cover mobile cover back cover mobile case phone case mobile panel phone panel Sony Xperia mobile case Sony Xperia phone cover Sony Xperia back case hard case 3D printed mobile cover mobile cove',


 'material': 'Wood',


 'brand': 'Amazon Brand - Solimo',


 'color': 'others',


 'item_name': 'Amazon Brand - Solimo Designer Leaf on Wood 3D Printed Hard Back Case Mobile Cover for Sony Xperia Z1 L39H',


 'model_name': 'Sony Xperia Z1 L39H',


 'model_number': 'gz8056-SL40528',


 'product_type': 'CELLULAR_PHONE_CASE'}

import os


 


from langchain.embeddings import OpenAIEmbeddings


from langchain.vectorstores.redis import Redis as RedisVectorStore


 


# set your openAI api key as an environment variable


os.environ['OPENAI_API_KEY'] = "YOUR OPENAI API KEY"


 


# data that will be embedded and converted to vectors


texts = [


    v['item_name'] for k, v in product_metadata.items()


]


 


# product metadata that we'll store along our vectors


metadatas = list(product_metadata.values())


 


# we will use OpenAI as our embeddings provider


embedding = OpenAIEmbeddings()


 


# name of the Redis search index to create


index_name = "products"


 


# assumes you have a redis stack server running on local host


redis_url = "redis://localhost:6379"

# create and load redis with documents


vectorstore = RedisVectorStore.from_texts(


    texts=texts,


    metadatas=metadatas,


    embedding=embedding,


    index_name=index_name,


    redis_url=redis_url


)

from langchain.callbacks.base import CallbackManager


from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler


from langchain.chains import (


    ConversationalRetrievalChain,


    LLMChain


)


from langchain.chains.question_answering import load_qa_chain


from langchain.llms import OpenAI


from langchain.prompts.prompt import PromptTemplate

template = """Given the following chat history and a follow up question, rephrase the follow up input question to be a standalone question.


Or end the conversation if it seems like it's done.


Chat History:\\"""


{chat_history}


\\"""


Follow Up Input: \\"""


{question}


\\"""


Standalone question:"""


 


condense_question_prompt = PromptTemplate.from_template(template)


 


template = """You are a friendly, conversational retail shopping assistant. Use the following context including product names, descriptions, and keywords to show the shopper whats available, help find what they want, and answer any questions.


 


It's ok if you don't know the answer.


Context:\\"""


 


{context}


\\"""


Question:\\"


\\"""


 


Helpful Answer:"""


 


qa_prompt= PromptTemplate.from_template(template)

# define two LLM models from OpenAI


llm = OpenAI(temperature=0)


 


streaming_llm = OpenAI(


    streaming=True,


    callback_manager=CallbackManager([


        StreamingStdOutCallbackHandler()


    ]),


    verbose=True,


    max_tokens=150,


    temperature=0.2


)


 


# use the LLM Chain to create a question creation chain


question_generator = LLMChain(


    llm=llm,


    prompt=condense_question_prompt


)


 


# use the streaming LLM to create a question answering chain


doc_chain = load_qa_chain(


    llm=streaming_llm,


    chain_type="stuff",


    prompt=qa_prompt


)

chatbot = ConversationalRetrievalChain(


    retriever=vectorstore.as_retriever(),


    combine_docs_chain=doc_chain,


    question_generator=question_generator


)

# create a chat history buffer


chat_history = []


# gather user input for the first question to kick off the bot


question = input("Hi! What are you looking for today?")


 


# keep the bot running in a loop to simulate a conversation


while True:


    result = chatbot(


        {"question": question, "chat_history": chat_history}


    )


    print("\\n")


    chat_history.append((result["question"], result["answer"]))


    question = input()

Hi! What are you looking for today?


 


>> gold-plated earrings


 


Hi there! I'm happy to help you find the perfect earrings. Do you have a preference for yellow gold plated sterling silver or platinum or gold-plated sterling silver?


 


>> My preference is the yellow gold plated sterling silver


 


Hi there! Are you looking for yellow gold-plated sterling silver earrings with Swarovski Zirconia or Topaz gemstones? We have a few options that might fit the bill. We have yellow gold-plated sterling silver Swarovski Zirconia fancy green stud earrings, yellow gold-plated sterling silver honey topaz stud earrings made with Swarovski Topaz gemstones, and yellow gold-plated sterling silver antique drop earrings set.

import json


from langchain.schema import BaseRetriever


from langchain.vectorstores import VectorStore


from langchain.schema import Document


from pydantic import BaseModel


 


class RedisProductRetriever(BaseRetriever, BaseModel):


    vectorstore: VectorStore


 


    class Config:


        arbitrary_types_allowed = True


 


    def combine_metadata(self, doc) -> str:


        metadata = doc.metadata


        return (


           "Item Name: " + metadata["item_name"] + ". " +


           "Item Description: " + metadata["bullet_point"] + ". " +


           "Item Keywords: " + metadata["item_keywords"] + "."


        )


 


    def get_relevant_documents(self, query):


        docs = []


        for doc in self.vectorstore.similarity_search(query):


            content = self.combine_metadata(doc)


            docs.append(Document(


                page_content=content,


                metadata=doc.metadata


            ))


 


        return docs

redis_product_retriever = RedisProductRetriever(vectorstore=vectorstore)


 


chatbot = ConversationalRetrievalChain(


    retriever=redis_product_retriever,


    combine_docs_chain=doc_chain,


    question_generator=question_generator


)

Hi! What are you looking for today?


 


>>> fancy footwear for going out


 


Hi there! We have a few great options for women's shoes and sandals. We have the Amazon Brand - The Fix Women's Giana Open Toe Bootie with Pearl Buckle, bright white leather, 9.5 B US, Flavia Women's Beige Fashion Sandals-7 UK (39 EU) (8 US) (FL/236/BEG), Flavia Women's Blue Fashion Sandals-8 UK (40 EU) (9 US) (FL/211/BLU), and The Fix Women's Faris Flat Slide Sandal with Pearls. All of these shoes feature a variety of styles and colors to choose from. Let me know if you have any questions about any of these items!


 


>>> These are nice. However, I am looking for men's shoes. Can you help me?


 


Hi there! We have a great selection of men's formal shoes available. We have Amazon Brand - Symbol Men's Formal Shoes, Amazon Brand - Symbol Men's Leather Formal Shoes, and more. All of our formal shoes are made from high quality materials and feature a variety of closure types, toe styles, and heel types. They also come with a manufacturer's warranty and care instructions to ensure they last. Let me know if you have any questions or need help finding the perfect pair of shoes for you!


 


>>>Can you show me some more men's options?


 


Hi there! I'm here to help you find the perfect item for you. We have a few options available for men's formal shoes. We have the Men's Stainless Steel Link Bracelet, the Amazon Brand - Arthur Harvey Men's Leather Formal Shoes, and the Amazon Brand - Symbol Men's Formal Derby shoes. All of these items feature a variety of features such as leather material, lace-up closure, pointed toe, block heel, and more. If you have any questions about any of these items, please let me know. I'm happy to help!


 


>>> Ok this looks great, thanks!