Source: https://lilianweng.github.io/posts/2023-06-23-agent/

Tools can be anything — a calculator, access to a weather API, access to the internet

The agent has access to these tools and its part of their toolbox. The agent can decide when and how to use these tools given the task that it is given from the user. Tools will have a description, which acts as a way for the agent to understand what the tool does and how to use it.

A simple example

What is the stock price of Microsoft?

The Agent will need to create a reasoning plan on how they’re going to achieve this.

Step 1: Get what the stock ticker of Microsoft is.

So how will it do that? Because this is knowledge that isn’t current or hasn’t changed, it actually already knows this. So it returns back that the symbol is MSFT.

Step 2: Find MSFT stock price

Cool, Now that it has this information, the agent will need to find the current stock price. How will it do that? Well, the model does not have that data on hand and it knows that it doesn’t, so it will look in it’s box of tools, which we’ve given a stock API, the ability to search stock prices given its ticker!

So it will return that data and find out that the stock price is $335.05 (as of June 29, 2023) and return that to a user.

Cool, but why is this different than a google search?

This seems pretty simple as most search engines can do that. But let’s put it in the context of a chat bot, which is what makes building agents unique.

Chat bots traditionally have a “hard coded flow” - filled with a bunch of conditional trees, and you need to “predict” how a user will go through a flow.

Now if you use the LLM as the brain or the core controller, it can decide for you, gather information from the tools, and scale a lot faster.

What is an agent?

• While loop - while you’re iterating - use the language model you check to see what step to take and if its done or not

• ReACT type of stuff - the action is a finish action is when you’re done.

• Grey area - using a router in the middle

• Plan an execute agent ->

• Use the language model to determine what actions to take

• agents = planning + memory + tool use

• Autonomy - limited form of intelligence - trust this thing on its own and do things on our behalf.

What is a chain?

• - pre-determined set of steps or sequence of events

Swyx and smol developer:

Link to slides: https://docs.google.com/presentation/d/1L_CHsg26sDxPmKj285Ob5T2xsAUejBlfiGQSnsSHTk0/edit#slide=id.p

The Anatomy of Autonomy: Why Agents are the next AI Killer App after ChatGPT

• Memory at its most superficial level is a tool. These become so specialized that they deserve a special place in the stack

• AutoGPT is very well known but not practical

• recommend: Tech’s two philosophies - ben thompson (Stratechery)

• TRICK: babyAGI - create diagrams with your code base

• 5 different levels of automated driving → Human driver vs AI Driver

• Engineers being able to wield AI more effectively makes them more productive

• Biggest challenge: whole program coherence - shared_dependences.md and name variables

• Markdown is great for prompts

  • code injections

  • structure

• Planner is too primitive but read the voyager edition from nvidia - https://arxiv.org/abs/2305.16291

• 2020 - 2022: Open AI Playground - completions

• End of 2022 - Chat

• 2023 - chat with Bing ← chat + tools/skills

  • March: Chat with plugins

  • April/May: Autogpt/babyAgi

• Chat with plugins/Bing

  • plugin usage that isn’t single iteration

  • There’s some kind of planning (for-loop) with a task queue and task usage

• Short term gains from doing feature engineering

• trade off - capability vs efficiency

  • just pay for open ai

What prompting technique is best for having agents use tools?

• Few shots prompting

I’m an avid reddit user. It’s actually part of my nightly routine before I go to bed where I mindlessly scroll on reddit, browsing some of my favorite subreddits such as r/golf, r/smoking, or r/daddit. So when I saw that a lot of online communities were “going dark” because of the API pricing changes, I figured it’d be interesting to dive into the why .

Data and $

Data is the currency of the future — That’s something that has been said for years now.

Source: https://aclanthology.org/W18-3022.pdf

With everything going on in AI and how we’re seeing a myriad of new LLMs being built every week, these are only possible with the vast amount of data available online. And a lot of LLMs are being trained on the free data available online through sites like Twitter, Reddit, and Stack Overflow (which have either raised their API prices or in stack overflow’s case, turned off their creative commons data dump)

Twitter (March 3, 2023): Twitter announces new API pricing, posing a challenge for small developers

Reddit (April 18, 2023): Reddit will begin charging for access to its API

Stack Overflow (June 9, 2023): June 2023 Data Dump is missing

Interesting pattern right? These companies are essentially seeing their data being used to train these LLMs and not seeing any benefit or payment from it and want a piece of the pie and don’t want to give away their data for free. Also there are talks of a Reddit IPO, so I’m sure there is some financial incentives there as well.

What does this mean for the future of online communities?

Online communities aren’t going away by any means, and in Reddit’s case, third party apps are going to be shut down (:tear: Apollo) and its hurting the 3rd party developer community. I’ve seen people talking about how communities should look to be decentralized or turn to other platforms (like how Twitter users went to mastodon or blue sky).

In the end, users will find a home, but the biggest challenge for communities is gathering enough users to create a community and build a sense of trust like how Reddit did. Maybe we’ll all end up with our VR headsets in the metaverse /s.

This is mostly an experiment, because I’ve been diving into the world of tools and ChatGPT plugins are essentially just tools. GPT4 is really good at deciding what tool to use when, and this feels like a pretty nice way to democratize tools to the general public.

1. First Impressions

2. Browse with Bing

3. Plugins

First Impressions

So I was mostly excited to see how plugins worked, because I already have access to GPT4 through the API. I kind of wish it was a little more discoverable, and having to “turn on” plugins feels like its some sort of secret or something.

Once I had them enabled, I was greeted with this interface. Why do they make these plugins not discoverable at all?! Only 8 plugins per page!

Oh also, I wish these plugins had some sort of documentation associated with it. You’re kinda going into installing these blindly. Anyways lets try some of these out.

Browse with Bing

Ok, this is a pretty interesting experience as it actually browses and clicks on links and can regurgitate information through the web with sources. I can see this being useful for doing research on new topics. I had ChatGPT come up with memes about the Apple Vision Pro headset. Are they funny? Not sure, you tell me 😂.

Plugins

So you are able to have a maximum of 3 plugins enabled, meaning GPT4 will have the ability to decide which three plugins to use. For example, I could probably have Kayak, OpenTable, and wolfram alpha to search for flights and rental cars, book reservations for me, and calculate the costs to split between my group of friends. Thats pretty cool!

Here are a few plugins that I tried out:

Video Insights

Ok, this is pretty sweet. I can give it a youtube link and it will summarize it for me. I normally use YouTube to look for cooking recipes or coffee techniques, and passing this in generates a summary and breaks it down for me really nicely.

Combining Plugins: Link Reader and ShowMe

Link reader takes in links and can grab that information. You can then use ShowMe to diagram information out. Here’s my example:

It did a pretty good job, I can see this being pretty useful!

Wait, there’s a meme generator plugin!

Let’s take the browse with bing output results of the memes from Apple Vision Pro and pass it to the meme generator.

lol.

How to train your (Chat)GPT Assistant

Pretraining

• Data collection - download a large amount of publicly available data. Such as through webscraping, stack overflow/exchange, wikipediea etc.

• Context length = max # of integers that the model will look at to predict the next integer in the sequence

• Dont judge a model by the number of parameters it contains.

• “Base models are not assistants, they just want to complete documents”

  • For example - if you ask it to write a poem about bread and cheese, it will just respond with more questions. But if you write “here’s a poem about bread and cheese”: it will autocomplete your document

• You can trick base models into assistants with few shot prompting.

Mode Collapse

RLHF vs SFT

• RLHF models are ranked higher than SFT

Base models can be better in tasks where you have N example of things and want to generate more N things. Base models = more entropy

Disadvantages: LLM Text generation vs Human

• LLMs spend the same amount of compute on every token

• They don’t reflect, they don’t sanity check, they don’t correct their mistakes along the way.

Cognitive advantages: LLM vs human

• They DO have a very large fact based knowledge across a vast number of areas

• They do have a large and ~perfect “working memory” (Context window)

Prompting is making up for this difference between these two architectures: Human brains vs LLM Brains

Chain of thought

“Models need tokens to think” →

“Let’s think step by step”

By snapping into a mode to show its work it will do less computational work per token more likely to succeed because its doing slower reason over time.

Ask for reflection

• LLMs can often recognize later when their samples didn’t seem to have worked out well

• LLMs can get unlucky in its sampling and a “bad” token but it doesn’t know to go back. LLMs will continue down this bad alley even if there’s no end. You have to ask to see if it met your prompt’s requirement.

Chains / Agents

• Think less “one-turn” Q&A and more chains, pipelines, state machines, agents

Condition on Good Performance

LLMs don’t want to succeed. They want to imitate training sets with a spectrum of performance qualities. you want to succeed and you should ask for it.

• For example: “Lets think step by step:” is okay

  • “Let’s work this out in a step by step way to be sure we have the right answer” is better

Tools Use / Plugins

• Offload tasks that LLMs are not good at. Importantly: They don’t “know” they are not good.

• For example arithmetic → You need to tell it in the prompt that they are not good at arithmetic and ask it to use a tool.

Retrieval-Augmented LLMs

• Load related context/information into “working memory” context window

• Emerging recipe

  • Break up relevant documents/data connectors into chunks

  • Use Embeddings API to index chunks into a vector store

  • Given a test-time query, retrieve related information

  • Organize the information into the prompt.

Constrained Prompting

• “Prompting languages” that interleave generation, prompting, logical control

  

  Default Recommendations

  

  Other recommendations for Use cases

  • Use in low-stakes applications, combine with human oversight

  • Source of inspiration, suggestions

  • Copilots over autonomous agents

So in Airkit we have a GPT “copilot” called Airkit Assistant, which will help you generate web pages and workflows by using just prompts. We also have an action that is connected to GPT4 for you to bring GPT4 into your apps. Check out this app I built in ~10 mins.

Check it Out!

Prompts that I used:

Building the Web Page

/make a form with a header called “Song Recommendation Engine”. Have a single text area input that asks for a users favorite artist and genre. Have a button that says submit. Add a label underneath to display recommended artist and song

Building the Data Flow

/make a data flow to get the access token from spotify and searches Spotify. You will only need one input, which is the query string for the search API. For the access token, I have the client id and client secret. Do not add these as inputs, I will hardcode them in the HTTP Request. Use the client credentials grant_type Then after you get the access token, do a get request on the spotify search API and passes the input in as the q query parameter. Remember to pass in the access token from the previous operation. Then filter the response of the API request to get the URL of the first response and the image URL of the first response. Do this in one transformation so that it outputs an object like {“image” : <image url> “url”: <url>}

Recommending the Song

You are a song recommendation engine assistant. Given the following artists and/or genres: 
text_area


Recommend a new track that the user would like based on the given artist and genre . 


Then output a recommended track The format of this should look like <Recommended Track Name> by <Recommended Artist Name>. Do not wrap the output in quotes.

I spent about 1 hour at 1.5x playback speed going through the course

The course i’m referring to is ChatGPT Prompt Engineering For Developers. Honestly, not sure why its called “ChatGPT Prompt Engineering” as we actually just hit the OpenAI APIs, specifically gpt-turbo-3.5. Does that make it ChatGPT? I don’t think so? Anyways, here are some of my notes and learnings.

Notes

• Use delimeters to indicate distinct parts of the prompt

  • ```, “””, < >, <tag> </tag>, :

    

• Delimeters protect against prompt injection

  • What is prompt injection?

    • If a user is allowed to add input into the prompt - can give conflicting instructions in the model.

    • The model understands that whatever is in the delimiter that needs to be “summarized” or whatever the instruction is

• Ask for structured output - “HTML, or JSON”

• Check whether conditions are satisfied, check assumptions required to do the task.

• Give the model the ability to check if a condition is satisfied. Fore example, “if no steps provided, say “No steps”. It's kind of like an “ELSE” statement

• Few Shot prompting

  • Provide successful examples of what to perform.

  • LLMs are capable of zero-shot prompting but its better to give an example

    

• Give the model time to think

  • provide it a chain of thinking or reasoning so that it doesn’t make up an answer. Such as “do x first, then do y”.

  • Models think kind of like humans and you have to be explicit otherwise it’ll “skim” and hallucinate

  

• Prompts should really never work the first time. Iteration

Consensus

It was definitely a course targeted for beginners but gives you a good overview of the basics in my opinion. Was it worth the time? I think so! Especially at 1.5x. I learned best practices and the correct terminology when thinking about prompts. I feel like you could figure out everything that was taught just by going through different prompt libraries though and finding patterns, if thats your method of learning.

✌️ Ismaen