Skip to content

12.code.md

Latest commit

 

History

History
644 lines (538 loc) · 15.4 KB

12.code.md

File metadata and controls

644 lines (538 loc) · 15.4 KB

Context & Memory: Making AI More Real

The Problem: No Memory

No Context = Chaos of Randomness and Confusion

cat << 'EOF' > src/app.py
from api import client
import click

model = "gpt-3.5-turbo"
system_prompt = "You are helpful assistant."

base_messages = [
    {
        "role": "system",
        "content": system_prompt
    }
]

while True:
    messages = base_messages.copy()

    # read the user input
    request = input(
      click.style(
        "Input: (type 'exit' to quit): ", 
        fg="green"
      )
    )
    
    if request.lower() in ["exit", "quit"]:
        break    

    # add the user input to the messages
    messages.append(
        {
            "role": "user",
            "content": f"{request}"
        }
    )

    # send the messages to the API
    response = client.chat.completions.create(
        model=model,
        messages=messages,                  
    )
    
    # get the response
    content = response.choices[0].message.content.strip()    

    # Print the command in a nice way
    click.echo(
        click.style(
            "Output: ", 
            fg="yellow"
        ) + content
    )

    click.echo()
EOF
python src/app.py
Input: (type 'exit' to quit): Hey!
Output: Hello! How can I assist you today?

Input: (type 'exit' to quit): My cat is black.
Output: That's interesting! Black cats are often associated with mystery and superstition. What else can you share about your cat?

Input: (type 'exit' to quit): Why?                      
Output: Why what? Can you please provide more context or specify your question?

Input: (type 'exit' to quit): I was talking about the cat!
Output: Oh, I apologize for the confusion. What would you like to talk about in regard to the cat? Please provide more specific information or ask a question.

Input: (type 'exit' to quit): The color!
Output: Color is a visual perception that results from the interaction of light with objects. It is the specific wavelengths of light that are reflected, transmitted, or absorbed by an object that determine its color. The human eye has different types of color receptors called cones, which are sensitive to different ranges of wavelengths. This allows us to perceive a broad spectrum of colors.

History = Context

cat << 'EOF' > src/app.py
from api import client
import click

model = "gpt-3.5-turbo"
system_prompt = "You are helpful assistant."

history = [
    {
        "role": "system",
        "content": system_prompt
    }
]

while True:    

    # read the user input
    request = input(
      click.style(
        "Input: (type 'exit' to quit): ", 
        fg="green"
      )
    )
    
    if request.lower() in ["exit", "quit"]:
        break    

    # Add the message to the history
    history.append(
        {
            "role": "user",
            "content": f"{request}"
        }
    )

    # send the messages to the API
    response = client.chat.completions.create(
        model=model,
        messages=history,
    )
    
    # get the response
    content = response.choices[0].message.content.strip() 

    # Debug: print the history
    click.echo(
        click.style(
            "History: ", 
            fg="blue"
        ) + str(history)
    )   

    # Print the command in a nice way
    click.echo(
        click.style(
            "Output: ", 
            fg="yellow"
        ) + content
    )

    # add the the response to the history
    history.append(
        {
            "role": "assistant",
            "content": f"{content}"
        }
    )

    click.echo()
EOF
Input: (type 'exit' to quit): Hi there.
History: [{'role': 'system', 'content': 'You are helpful assistant.'}, {'role': 'user', 'content': 'Hi there.'}]
Output: Hello! How can I assist you today?

Input: (type 'exit' to quit): My cat is black!
History: [{'role': 'system', 'content': 'You are helpful assistant.'}, {'role': 'user', 'content': 'Hi there.'}, {'role': 'assistant', 'content': 'Hello! How can I assist you today?'}, {'role': 'user', 'content': 'My cat is black!'}]
Output: That's interesting! Black cats can be quite beautiful. Is there anything specific you'd like to know or discuss about your black cat?

Input: (type 'exit' to quit): Yes.
History: [{'role': 'system', 'content': 'You are helpful assistant.'}, {'role': 'user', 'content': 'Hi there.'}, {'role': 'assistant', 'content': 'Hello! How can I assist you today?'}, {'role': 'user', 'content': 'My cat is black!'}, {'role': 'assistant', 'content': "That's interesting! Black cats can be quite beautiful. Is there anything specific you'd like to know or discuss about your black cat?"}, {'role': 'user', 'content': 'Yes.'}]
Output: Sure, what would you like to know or discuss about your black cat? I'm here to help!

The Problem with Carrying Over History

Last In First Out (LIFO) Memory

cat << 'EOF' > src/app.py
from api import client
import click, json

# define the length of the history to consider
n = 2
model = "gpt-3.5-turbo"
system_prompt = "You are helpful assistant."

global_context = [
    {
        "role": "system",
        "content": system_prompt
    }
]

# Define the file path
hitory_file_path = 'context.txt'

# Open the file in 'w' mode 
# and close it immediately
with open(hitory_file_path, 'w') as file:
    pass

def save_history_to_file(history):
    """
    Save the history of interactions 
    to a file.
    """
    with open(hitory_file_path, "w") as f:
        # Use json to store history 
        # as a JSON string        
        f.write(json.dumps(history))

def load_history_from_file():
    """
    Load the history from a file.
    """
    with open(hitory_file_path, "r") as f:
        import json
        try:
            history = json.loads(f.read())
            # Return the last n items
            return history[-n:]
        # In case the file is empty or 
        # corrupted
        except json.JSONDecodeError:
            return []

full_history = []
while True:    
    # read the user input
    request = input(
      click.style(
        "Input: (type 'exit' to quit): ", 
        fg="green"
      )
    )
    
    if request.lower() in ["exit", "quit"]:
        break    

    # Add the message to the history
    history = {
            "role": "user",
            "content": request
    }
    

    # Load the history from the file 
    # and append the new messages
    full_history = load_history_from_file()
    full_history.append(history)
    messages = global_context + full_history
    # Send the messages to the API
    response = client.chat.completions.create(
        model=model,
        messages=messages,
        max_tokens=150,
        temperature=0.7,
    )

    # Debug: print the history    
    click.echo(
        click.style("History: ", fg="blue") + \
        str(json.dumps(
            messages, 
            indent=4
            )
        )
    )
    
    # Get the response
    content = response.choices[0].message.content.strip() 

    # Print the command in a nice way
    click.echo(
        click.style(
            "Output: ", 
            fg="yellow"
        ) + content
    )

    # Add the response to the history
    full_history.append(
        {
            "role": "assistant",
            "content": content
        }
    )

    # Save the history to a file
    save_history_to_file(full_history)
EOF
python src/app.py

The Problem with Last in First out Memory

Selective Context

embedding_model = "text-embedding-ada-002"
context_window = 5

def sort_history(history, prompt, context_window):
    """
    Sort the history of interactions based 
    on cosine similarity.
    Returns the top context_window segments.
    """
    sorted_history = []
    for segment in history:        
        content  = segment['content']
        preprocessed_content = preprocess_text(content)
        preprocessed_prompt = preprocess_text(prompt)
        embedding_model = "text-embedding-ada-002"
        embedding_content = get_embedding(
            preprocessed_content, 
            embedding_model
        )
        embedding_prompt = get_embedding(
            preprocessed_prompt, 
            embedding_model
        )
        similarity = cosine_similarity(
            embedding_content, 
            embedding_prompt
        )
        sorted_history.append(
            (segment, similarity)
        )
    sorted_history = sorted(
        sorted_history, 
        key=lambda x: x[1], 
        reverse=True
    )
    sorted_history = [
        x[0] for x in sorted_history
    ]
    return sorted_history[:context_window]
# defined in src/api.py
def get_embedding(text, model):
    text = text.replace("\n", " ")
    return client.embeddings.create(
        input = [text], 
        model=model
    ).data[0].embedding

# defined in src/utils.py
def preprocess_text(text):
    from nltk.corpus import stopwords
    from nltk.stem import PorterStemmer
    from nltk.tokenize import word_tokenize    
    # Tokenize text
    tokens = word_tokenize(text)

    # Convert to lower case
    tokens = [
        word.lower() for word in tokens
    ]

    # Remove punctuation
    words = [
        word for word in tokens if word.isalpha()
    ]

    # Filter out stop words
    stop_words = set(
        stopwords.words('english')
    )
    words = [
        word for word in words if word not in stop_words
    ]

    # Stemming
    stemmer = PorterStemmer()
    stemmed_words = [
        stemmer.stem(word) for word in words
    ]

    return ' '.join(stemmed_words)
cat << 'EOF' > src/app.py
from api import client, get_embedding
from utils import preprocess_text, cosine_similarity
import click, json

context_window = 2
model = "gpt-3.5-turbo"
system_prompt = "You are helpful assistant."
hitory_file_path = 'context.txt'
full_history = []

global_context = [
    {
        "role": "system",
        "content": system_prompt
    }
]

# Open the file in 'w' mode 
# and close it immediately
with open(hitory_file_path, 'w') as file:
    pass

def save_history_to_file(history):
    """
    Save the history of interactions to a file.
    """
    with open(hitory_file_path, "w") as f:
        f.write(json.dumps(history))

def load_history_from_file():
    """
    Load the history from a file.
    """
    with open(hitory_file_path, "r") as f:
        import json
        try:
            history = json.loads(f.read())            
            return history
        except json.JSONDecodeError:
            return []

def sort_history(history, prompt, context_window):
    """
    Sort the history of interactions based 
    on cosine similarity.
    Returns the top context_window segments.
    """
    sorted_history = []
    for segment in history:        
        content  = segment['content']
        preprocessed_content = preprocess_text(content)
        preprocessed_prompt = preprocess_text(prompt)
        embedding_model = "text-embedding-ada-002"
        embedding_content = get_embedding(
            preprocessed_content, 
            embedding_model
        )
        embedding_prompt = get_embedding(
            preprocessed_prompt, 
            embedding_model
        )
        similarity = cosine_similarity(
            embedding_content, 
            embedding_prompt
        )
        sorted_history.append(
            (segment, similarity)
        )
    sorted_history = sorted(
        sorted_history, 
        key=lambda x: x[1], 
        reverse=True
    )
    sorted_history = [
        x[0] for x in sorted_history
    ]
    return sorted_history[:context_window]

while True:    
    # read the user input
    request = input(
      click.style(
        "Input: (type 'exit' to quit): ", 
        fg="green"
      )
    )
    
    if request.lower() in ["exit", "quit"]:
        break    

    # Add the message to the history
    user_prompt = {
            "role": "user",
            "content": request
    }
    
    # Load the history from the file 
    # and append the new messages
    full_history = load_history_from_file()    
    sorted_history = sort_history(
        full_history, 
        request, 
        context_window
    )
    sorted_history.append(user_prompt)
    messages = global_context + sorted_history
    # Send the messages to the API
    response = client.chat.completions.create(
        model=model,
        messages=messages,
        max_tokens=200,
        temperature=1,
    )

    # Debug: print the history    
    click.echo(
        click.style(
            "History: ", 
            fg="blue"
        ) + str(json.dumps(messages, indent=4))
    )
    
    # Get the response
    content = response.choices[0].message.content.strip() 

    # Print the command in a nice way
    click.echo(
        click.style(
            "Output: ", 
            fg="yellow"
        ) + content
    )

    # Add the user prompt to the history
    full_history.append(user_prompt)
    # Add the response to the history
    full_history.append(
        {
            "role": "assistant",
            "content": content
        }
    )

    # Save the history to a file
    save_history_to_file(full_history)
EOF
python src/app.py
I have a black cat.
I have a white shirt.
What's the color of my cat?
What's the color of my shirt?
Input: (type 'exit' to quit): I have a black cat.
History: [
    {
        "role": "system",
        "content": "You are helpful assistant."
    },
    {
        "role": "user",
        "content": "I have a black cat."
    }
]
Output: That's wonderful! Black cats are often associated with mystery and superstition. They are also believed to bring good luck in many cultures. Do you have any specific questions or concerns related to your black cat? I'm here to help!
Input: (type 'exit' to quit): I have a white shirt.
History: [
    {
        "role": "system",
        "content": "You are helpful assistant."
    },
    {
        "role": "user",
        "content": "I have a black cat."
    },
    {
        "role": "assistant",
        "content": "That's wonderful! Black cats are often associated with mystery and superstition. They are also believed to bring good luck in many cultures. Do you have any specific questions or concerns related to your black cat? I'm here to help!"
    },
    {
        "role": "user",
        "content": "I have a white shirt."
    }
]
Output: Great! A white shirt is a versatile and classic clothing item that can be paired with various bottoms and accessories. It's a fashion staple that can be dressed up or down for different occasions. Do you have any particular concerns or questions about caring for or styling your white shirt? Feel free to ask!
Input: (type 'exit' to quit): what's the color of my cat?
History: [
    {
        "role": "system",
        "content": "You are helpful assistant."
    },
    {
        "role": "user",
        "content": "I have a black cat."
    },
    {
        "role": "user",
        "content": "I have a white shirt."
    },
    {
        "role": "user",
        "content": "what's the color of my cat?"
    }
]
Output: Your cat is black.
Input: (type 'exit' to quit): What's the color of my shirt?
History: [
    {
        "role": "system",
        "content": "You are helpful assistant."
    },
    {
        "role": "user",
        "content": "I have a white shirt."
    },
    {
        "role": "user",
        "content": "what's the color of my cat?"
    },
    {
        "role": "user",
        "content": "What's the color of my shirt?"
    }
]
Output: You mentioned earlier that you have a white shirt. Therefore, the color of your shirt is white.