From The Challenges - Redis
From The Challenges - Redis
Common Mistakes With The Build Your Own Redis Server Coding Challenge
Hi this is John with this week’s Coding Challenge.
🙏 Thank you for being one of the 69,172 software developers who have subscribed, I’m honoured to have you as a reader. 🎉
If there is a Coding Challenge you’d like to see, please let me know by replying to this email📧
Welcome To Coding Challenges - From The Challenges!
In this Coding Challenges newsletter I’m sharing some of the common mistakes I see software engineers make when tackling the Coding Challenges.
I’m sharing the mistakes people make and some thoughts on how you can you avoid making the same mistakes when taking on the coding challenges or when writing software professionally.
🚨 NEWS! 🚨
📌 I Wrote A Guide To Preparing For DSA Interviews
You can find it over on my other newsletter Developing Skills, it’s the article: Proper Prepping For DSA Interviews. If you’d like more of that content, please give it a like and/or drop a comment letting me know.
📚 There’s A New Coding Challenges Course Now Available: Build Your Own Shell In Python.
Newly released and available in the Coding Challenges Course Shop as Build Your Own Shell (Python Edition). You can get 50% off until Monday 4th November using the code: PYSHLAUNCH !
Recapping The Redis Coding Challenge
In the build your own Redis server coding challenge the goal was to write your own lite version of Redis which supported the same operations as the original version of Redis.
Like all the challenges, you can tackle it in any programming language you like. This coding challenge is particularly close to my heart as building a Redis clone in Rust was what inspired me to start Coding Challenges.
If you'd like a deep dive into tackling this coding challenge I have a course available for purchase that guide you through it: Become a Better Software Developer by Building Your Own Redis Server, it’s available in Python and soon to be available in Go.
If You Enjoy Coding Challenges Here Are Three Ways You Can Help Support It
Refer a friend or colleague to the newsletter. 🙏
Sign up for a paid subscription - think of it as buying me a coffee ☕️ twice a month, with the bonus that you also get 20% off any of my courses.
Buy one of my courses that walk you through a Coding Challenge.
Five Common Mistakes Software Engineers Make Solving the Redis Coding Challenge
I’ve pulled together this list of common mistakes from the hundreds of submissions I’ve been sent privately and the many shared in the Coding Challenges Shared Solutions GitHub Repo.
It’s interesting to note that about 50% of the solutions that have been shared with me are written in Go. Rust and TypeScript being the next most popular languages for this project.
Mistake 1 - Using A Library To Handle The RESP Protocol.
If you want to be a systems software engineer, or to learn how to build reliable network clients and servers then learning how to implement network protocols and how to parse them is incredibly useful. Using an existing library deprives you of the chance to learn.
This is a real missed opportunity as the RESP protocol is a great protocol to learn with, it’s well documented and introduces you to how to handle sending and receiving data over TCP, something most software engineers still don’t know how to do.
Mistake 2 - Mixing Concerns
Several of the solutions mix handling the network client, with the protocol and the command handling. This made the code difficult to follow and mixed the concerns. Part of the code was parsing the protocol whilst also trying to handle the command and determine if a full message has been received.
Another example had the core datastore encoding the data in the RESP protocol. Mixing the concern like this made the code difficult to test, so there were no tests and there are subsequently bugs in the code.
Where possible try to separate concerns. Ideally functions or classes should do, or represent one thing, and one thing only. This can be taken too far, so it does involve some judgement as to the right level of separation.
Mistake 3 - Assuming Network (TCP) Receive Gets Full Messages
TCP is a streaming protocol what you a receiving data from a TCP connection you’re reading from the stream, you’re not guaranteed to be reading a full message.
Consider this simple Python code, you’ll be using the same recv call in most other programming languages as they’re all wrappers around the Berkley sockets implementation for your operating system.
data = client_socket.recv(4096)This call to recv on the socket will get back between zero and 4096 bytes. Assuming it got more than zero bytes then data will container one of the following:
A partial message.
A full message - something we’ll often see when developing and testing with small messages on a local network, meaning inexperienced software engineers often miss subtle bugs.
A full message followed by a partial message.
A partial message followed by either a partial or a full message.
Combinations of the above.
In short the data we get back is a window on a stream of data being sent to the server. It might help to visualise this:
So a key part of writing a TCP based server is to ensure you’re correctly identifying the start and end of a message in the stream, given you’re reading just a window into it.
N.B. You can’t just call recv() passing in the size of a message for several reasons:
The message might not be fixed size. In the case of Redis they’re not.
Even if they are fixed size, they might still be sent as multiple smaller messages ‘over the wire’.
TCP send streams of data, but it’s usually built on top of a lower level protocol, such as Ethernet or IP. As a result only part of a message might have been received by the server’s network stack when you call
recv(), the rest might still be in flight between the server and the client.
Mistake 4 - No Or Little Error Checking
Many of the solutions assumed there were no errors and as such had little or no error handling. That’s never a good assumption to make, particularly when you’re building a server. You have no guarantee that the client that’s connecting to you is always going to send wellformed messages.
Even if it is sending wellformed messages they might be from an earlier or later version of the protocol. For example, this coding challenge used RESP2, a user connecting with a client aimed at an earlier version of Redis might use RESP and a newer client might use RESP3.
Mistake 5 - Not Reading The Specification And Challenge Completely
Many of the solutions skipped implementing the expiry in step 5. Whilst some implemented the passive expiry, most skipped over the probabilistic expiry algorithms used by Redis. Which is:
Every 100ms it tests 20 random keys from the set of keys with an associated expire. Delete all the keys found expired. If more than 25% of keys were expired, start again.
Learning how to do this, running concurrently to handling client connections is a good exercise in writing algorithmic and concurrent code. You get better at writing systems software by writing systems software, particularly the harder bits!
Request for Feedback
I’m writing these coding challenges and this new from the challenges series to help you develop your skills as a software engineer based on how I’ve approached my own personal learning and development.
What works for me, might not be the best way for you - so if you have suggestions for how I can make these challenges more useful to you and others, please get in touch and let me know. All feedback greatly appreciated.
You can reach me on Twitter, LinkedIn or through SubStack
Thanks and happy coding!
John
P.S. If You Enjoy Coding Challenges Here Are Four Ways You Can Help Support It
Refer a friend or colleague to the newsletter. 🙏
Sign up for a paid subscription - think of it as buying me a coffee ☕️ twice a month, with the bonus that you also get 20% off any of my courses.
Buy one of my courses that walk you through a Coding Challenge.
Subscribe to the Coding Challenges YouTube channel!