From The Challenges - Rate Limiter
Exploring the software engineering lessons we can learn from the solutions I've seen.
Hi this is John with this week’s Coding Challenge.
🙏 Thank you for being one of the 88,719 software developers who have subscribed, I’m honoured to have you as a reader. 🎉
If there is a Coding Challenge you’d like to see, please let me know by replying to this email📧
Welcome To Coding Challenges - From The Challenges!
In this Coding Challenges “from the challenges” newsletter I’m sharing some of the common mistakes I see software engineers make when tackling the Coding Challenges.
I’m sharing both the mistakes people make and some thoughts on how you can you avoid making the same mistakes when taking on the coding challenges or when writing software professionally. Sometimes we have to make mistakes to learn from them, somethings we can learn from other people’s mistakes, then make our own new ones! 😀
Recapping The Rate Limiter Coding Challenge
In the build your own rate limiter coding challenge the goal was to write a rate limiter and learn about rate limiting algorithms.
Rate limiters are a key part of building an API or large scale distributed system, they help when we wish to throttle traffic based on the user. They allow you to ensure that one or more bad actors can’t accidentally or deliberately overload the service.
A rate limiting strategy can make your API more reliable, when:
A user is responsible for a spike in traffic, and you need to stay up for everyone else.
A user is accidentally sending you a lot of requests.
A bad actor is trying to overwhelm your servers.
A user is sending you a lot of lower-priority requests, and you want to make sure that it doesn’t affect your high-priority traffic.
Your service is degraded, and as a result you can’t handle your regular traffic load and need to drop low-priority requests.
🚨Would You Like To Learn Go By Building Coding Challenges! 🚨
I’m running the Coding Challenges Learn Go By Building Live Course again in September.
It is a live course that runs for three working weeks from September the 15th to the October the 3rd. During the course I’ll introduce you to ever aspect of Go that you need to build the following five real-world projects (based off five of my coding challenges):
🏗️ cat - By building cat you learn how to build and run command line programs in Go.
🏗️ sort - By building sort you learn how to use Go's data structures and control flow to implement sort.
🏗️ curl - By building curl you learn how to write network clients in Go.
🏗️ wc - By building wc you learn how to process text data and handle locales with Go.
🏗️ Memcached (Capstone Project) - By building a Memcached server clone you learn how to build efficient network servers in Go.
Having built these five real-world applications you will be well equipped to take on new projects in Go!
If you sign up before 25th August you can get $200 off! Use the early bird code: EBLGSEP25
If You Enjoy Coding Challenges Here Are Four Ways You Can Help Support It
Refer a friend or colleague to the newsletter. 🙏
Sign up for a paid subscription - think of it as buying me a coffee ☕️, with the bonus that you also get 20% off any of my courses.
Buy one of my self-paced courses that walk you through a Coding Challenge.
Join one of my live courses where I personally teach you Go by building five of the coding challenges or systems software development by building a Redis clone.
Five Common Mistakes Software Engineers Make Solving The Rate Limier Coding Challenge
I’ve pulled together this list of common mistakes from the hundreds of submissions I’ve been sent privately and the many shared in the Coding Challenges Shared Solutions GitHub Repo.
Mistake 1 - Using A Client API For A Network Service And Assuming Every API Call Succeeds.
It’s great that we have so many client SDKs and language based APIs for network services and servers such as Redis, but don’t forget if they’ve making a network call, which most of them are, then they can (and at scale will) fail occasionally.
Why? Because networks are unreliable, especially if the traffic goes over the public Internet. Servers can also be overloaded or unavailable too. If your code makes network calls and uses other services you should code defensively allowing for intermittent failures and remote services becoming unavailable.
To handle intermittent errors, ensure you check the error codes and where reasonable retry the request. Be mindful that too many retries can overload a backend service so consider adding backoff. To handle a service becoming unavailable, ensure that the code degrades gracefully or fails safely.
In all cases log the failure so it can be investigated later.
Mistake 2 - Not Protecting Shared Data
When working with concurrent code, it's important to handle shared state to avoid race conditions and ensure data integrity. In web-based applications, shared state can include databases, caches, or other external resources.
Here are some best practices for managing shared state:
Use synchronisation mechanisms such as locks or mutexes to protect shared resources from concurrent access.
Avoid global variables or shared mutable state whenever possible.
Consider using connection pooling or connection limits to manage database connections and prevent resource exhaustion.
Use atomic operations or synchronisation primitives to perform thread-safe operations on shared variables. Check what your programming language supports.
Mistake 3 - IO In Performance Critical Sections
Rate limiting will often be applied to services that have a high load. It’s used to manage resources ensuring good responses times and service availability.
In such cases performance is critical, so rate limiting code should not impose an overhead on the service. As such it’s a good idea to avoid any unnecessary IO in the rate limiting code.
If you need access to a backend data service, pick a good in memory data store. If you need to log, either locally over over the network, make it easy to disable the logs and consider offloading the slow IO to a background thread.
Mistake 4 - Doing Unnecessary Work
The server should not need to be doing any continuous work to enforce rate limiting, there is no need for it to have one or more threads/async tasks running timers. Instead record a time stamp in a data structure. If you push new events onto one end of a queue-like data structure the events will naturally be sorted and you can remove “expired” events from the other end easily. Do this house keeping after sending the response to the client.
Remember in a high traffic solution, doing unnecessary work will mean the solution won’t scale.
Mistake 5 - Avoid ‘Utils’
Try to avoid creating catch-all classes, modules, and packages. Utils is a common example of this. It’s better to figure out what the code relates to and place it there. If it genuinely belongs on it’s own find a relevant name.
Also worth avoiding: shared, common, utility; like utils they will become a dumping ground for all sorts of potentially reusable code.
Nice One! Using A Load Tester To Evaluate The Solution
It was great to see several of the solutions leveraged existing open source tools to create a load test. I saw examples using: Locust, k6 and loadtest.
It would have been even better to see someone build their own load testing tool and use that. 😎
Request for Feedback
I’m writing these coding challenges and this new from the challenges series to help you develop your skills as a software engineer based on how I’ve approached my own personal learning and development.
What works for me, might not be the best way for you - so if you have suggestions for how I can make these challenges more useful to you and others, please get in touch and let me know. All feedback greatly appreciated.
You can reach me on Bluesky, LinkedIn or through SubStack
Thanks and happy coding!
John
P.S. If You Enjoy Coding Challenges Here Are Four Ways You Can Help Support It
Refer a friend or colleague to the newsletter. 🙏
Sign up for a paid subscription - think of it as buying me a coffee ☕️ twice a month, with the bonus that you also get 20% off any of my courses.
Buy one of my courses that walk you through a Coding Challenge.
Subscribe to the Coding Challenges YouTube channel!