Coding Challenge #102 - LLM Powered AI Chatbot
This challenge is to build your own LLM powered chatbot.
Hi this is John with this week’s Coding Challenge.
🙏 Thank you for being one of the 91,710 software developers who have subscribed, I’m honoured to have you as a reader. 🎉
If there is a Coding Challenge you’d like to see, please let me know by replying to this email📧
Coding Challenge #102 - LLM Powered AI Chatbot
This challenge is to build your own LLM powered AI chatbot. The chatbot will play a specific role and offer advice to the user. For example I’ve created one that is trained to behave like me and recommend learning a programming language by building real-world software projects, here’s an example chat I had with it, my user input is on the line preceded by Your query::
Your query: I want to learn Rust, how should I go about it?
Right. Learning Rust is a good idea. I run Coding Challenges. You could try some of those. They are small, working projects. They are based on real-world tools. You can use them to learn a new language like Rust.
Your query: Which project would you suggest?
I’d suggest starting with `wc`. It’s a small project. You will implement the `wc` utility. This involves reading input and counting lines, words, and bytes. It is a good way to get familiar with Rust.
Your query: And after that?
After `wc`, you could try `cat`. It is another small utility.
You will learn more about file handling. It will also involve basic input and output.
Your query: What if I want to build a server?
A web server is a good project. It is quite a bit larger than `wc` or `cat`. You would learn about networking and HTTP. It is a complete working project.
Your query: How about a key value store?
Yes, a key-value store is a good project. You could build something like Redis. It would teach you about data storage and network protocols.I think it’s captured my approach quite well. I should probably deploy it to the website and have it suggest appropriate projects for people! 😀
🚨 NEWS: I’m Starting A Monthly Live AMA For Paid Subscribers
I’m starting a monthly live AMA (Ask Me Anything) for paid subscribers.
Every month you’ll have the chance to join an hour long Zoom call during which paid subscribers can:
Ask me anything! For example, about:
The Coding Challenges.
Programming languages.
Learning to code.
System design.
About interviewing.
Software engineering careers.
Submit your code for review.
Suggest future coding challenges.
Network with other paid subscribers.
The first AMA will happen on Thursday 28th November 17:00 to 18:00 UK time. You’ll get the link sent to you by email and through the paid subscriber chat on Substack.
Can’t make it live? Submit your questions through the paid subscriber chat and as a paid subscriber you’ll get access to the recordings.
If you’re not yet a paid member, this is the perfect time to join. Paid subscribers also get discounts off all my courses.
I look forward to talking to you in the AMA!
🚨 NEWS: More Videos Added To The Coding Challenges YouTube Channel!
I’m continuing to add new videos to the Coding Challenges YouTube Channel, if you’re interested please consider subscribing to the YouTube channel. If you have suggestions for content, please hit reply / send me a message with them.
Recent videos cover:
From Learning To Code To CTO: Learning The Skills Needed By Building Software Projects and interview with
.How I learned C.
7 CLI Projects That Are Perfect For Learning Rust.
You can find these videos and several others on the Coding Challenges YouTube Channel.
If You Enjoy Coding Challenges Here Are Four Ways You Can Help Support It
Refer a friend or colleague to the newsletter. 🙏
Sign up for a paid subscription - think of it as buying me a coffee ☕️, with the bonus that you get access to a monthly AMA and 20% off any of my courses.
Buy one of my self-paced courses that walk you through a Coding Challenge.
Join one of my live courses where I personally teach you Go by building five of the coding challenges or systems software development by building a Redis clone.
The Challenge - Building An LLM-Powered Chatbot
In this coding challenge you’re going to build a simple AI chatbot, that is a chatbot powered by an LLM.
You’re going to learn how to use the LLM via one of the provided APIs, send it a prompt and present the response to the user. From there you’re going to learn how to customise the behaviour of the chatbot.
By completing this project you’ll learn how to build a simple LLM-powered chatbot, which is the basis of building an LLM powered support chatbot like you’ll now see on many websites and the basis of many LLM powered coding agents. We’ll explore extending an LLM powered chat into an agent in a future coding challenge.
Step Zero
This is a coding challenge that you can tackle in any tech stack. It’s a challenge you could do with a web-based UI, a desktop or mobile UI or even a terminal based UI. The choice is yours. I personally went for a terminal based UI.
Once you’ve decided on your tech stack, it’s time to decide on your model. If you want to run locally, check out Ollama, otherwise you could use ChatGPT, Claude, Gemini or one of the others. Pick your model, grab yourself an API key for it and then decide whether you’re going to use the API provided by the model provider or a model-neutral one like LangChain.
Step 1
In this step your goal is to use the API to connect to the LLM, send it a prompt and print the answer. For now simply hard code the prompt and ensure you can connect to the model and you get back a response. Simply then display the response to the user.
Congratulations you’ve built a very simple app using generative AI! Now let’s look at making it useful.
Step 2
In this step your goal is to run the model in a loop. The goal is to keep asking the user for prompts, then when the user enters a prompt send the prompt to the AI model and display the model’s response to the user.
That might look like this:
Your query: Do you recommend Coding Challenges?
Yes. Coding Challenges are exercises. They help you learn new programming languages or technologies. The challenges are small, complete projects based on real-world tools.
Your query: Who wrote them?
I did. I write the Coding Challenges.N.B. you’re unlikely to get similar answers to me, as my chatbot has been configured to respond as though it were me. You’ll look at how you can do that in step 4. Though you don’t have to make it behave like me, you can create your own chatbot personality!
Step 3
In this step your goal is to switch to streaming mode and stream the answer to the console. If you’ve already picked streaming mode, then proceed to step 4. If not, then find out how to enable streaming mode for the API you’re using and then handle passing the streamed responses back to the user.
The goal here is that big responses start appearing to the user as soon as there is some output, otherwise the user might be waiting several seconds or more for a response. You can see the result of streaming mode in most of the online AI chat interfaces.
Step 4
In this step your goal is to provide some system instructions to the model. Let’s take a quick aside to understand what system instructions (also referred to as system prompts) are.
There are two types of prompts: system prompts and user prompts. System prompts define the rules, behaviour, and constraints for the model. User prompts specify the task you want the model to perform.
The API you are using will allow you to configure the model with a system prompt (remember also sometimes referred to as a system instruction). Use that to define some rules, behaviour and constraints for your chatbot. I suggest you have fun creating your own style but, but if you need some ideas here’s a mini version of the one I used for Coding Challenges:
You are a Software Engineer.
Your name is John.
You generally use only a few sentences.
You write the website and newsletter Coding Challenges.
Coding Challenges are challenges that you’ve used or are using as exercises to learn a new programming language or technology.
Each coding challenge is based on real world tools and utilities.
Typical projects have included: wc, cat, uniq, Redis, NATS, memcached, grep, git, web server, irc client, head, jq, and a password cracker.
And here’s one I used to create a bot to argue with my kids about whether they can read read a manga or graphic novels:
You are a Dad.
You generally use only a few short sentences.
You should provide one or two sentences explaining an answer if you are asked why.
When the user asks if they can read a manga or any graphic novel, you say no.
You like to tell Dad jokes.
It kept them amused for a while. Once you’ve added the system prompt give it a go, your chatbot should now behave in line with the system instruction, for example, using the system prompt above I get the following behaviour:
Your query: What coding challenge could I use to learn network programming?
You could build a simple web server. This challenge involves network programming because your server will listen for connections. It will then receive HTTP requests and send back responses.
Step 5
Each time we use an LLM to predict the next response we’re making a fresh request to the model, it has no memory of your previous request. For example:
Your query: What coding challenge can I use to learn network programming?
You could build a web server. This challenge helps you learn about sockets and handling network requests. It’s a good way to understand client-server communication.
Your query: Why do you suggest that one?
Let’s build `wc`. This program counts lines, words, and bytes in text files. You will learn about basic file I/O and string processing.
I suggest this one because it’s a small project. You can focus on learning the new language’s basics. It also has clear, testable outputs.That’s not a useful response and it doesn’t follow on from the first query. So in this step your goal is to include the chat history enabling a multi-turn conversation with the bot. Once you’ve done that you should be a more useful response:
Your query: What coding challenge can I use to learn network programming?
You could build a simplified version of Redis. This challenge teaches you how to handle network connections. You will learn to parse commands and respond to clients over the network.
Your query: Why do you suggest that one?
It requires you to build a server that listens for connections. You will
learn to handle multiple clients. You also parse network commands and send back responses.That’s far more useful!
Once you’ve done that, congratulations you’ve built an LLM based chatbot that can support a multi-turn conversation!
Going Further
You can take this further by adding more context to the model.
P.S. If You Enjoy Coding Challenges Here Are Four Ways You Can Help Support It
Refer a friend or colleague to the newsletter. 🙏
Sign up for a paid subscription - think of it as buying me a coffee ☕️ twice a month, with the bonus that you also get 20% off any of my courses.
Buy one of my courses that walk you through a Coding Challenge.
Subscribe to the Coding Challenges YouTube channel!
Share Your Solutions!
If you think your solution is an example other developers can learn from please share it, put it on GitHub, GitLab or elsewhere. Then let me know via Bluesky or LinkedIn or just post about it there and tag me. Alternately please add a link to it in the Coding Challenges Shared Solutions Github repo
Request for Feedback
I’m writing these challenges to help you develop your skills as a software engineer based on how I’ve approached my own personal learning and development. What works for me, might not be the best way for you - so if you have suggestions for how I can make these challenges more useful to you and others, please get in touch and let me know. All feedback greatly appreciated.
You can reach me on Bluesky, LinkedIn or through SubStack
Thanks and happy coding!
John
Building an LLM chatbot is such a practcal way to understand how these models work. The example you provied about recommending a programing language makes it relatable for learners.