Coding Challenge #126 - Background Removal Tool
This challenge is to build your own background removal tool for images.
Hi, this is John with this week’s Coding Challenge.
🙏 Thank you for being a subscriber, I’m honoured to have you as a reader. 🎉
If there is a Coding Challenge you’d like to see, please let me know by replying to this email📧
Coding Challenge #126 - Background Removal Tool
This challenge is to build your own image background removal tool that takes a photograph of a person and removes the background, keeping the person and making everything else transparent.
You have probably seen this feature everywhere by now. You give it a photo of yourself, and a moment later you have a clean cut-out that you can drop onto a new background, turn into a profile picture, or use in a design. It feels like magic, but underneath it is a satisfying mix of image decoding, machine learning, and careful pixel work. Getting the body right is the easy part. Handling the fine wisps of hair less so!
In this challenge you are going to build a tool that does all of this on the user’s own device, with no hosted service doing the heavy lifting for you. By the end you will have a tool that can process a single image or a whole directory tree of them.
The Challenge - Building a Background Removal Tool
You are going to build a tool that accepts a photograph of a person, works out which pixels belong to the person and which belong to the background, and writes out a new image where the person is preserved and everything else is fully transparent.
The interesting parts of this challenge are the details. You have to decode common image formats, automatically find the person without any help from the user, produce edges that are smooth enough to look natural against a new background, and do all of the clever work on the user’s own device rather than shipping the image off to someone else’s server/API. Once the single-image case works, you will scale it up to handle whole directories at once while behaving gracefully when one image causes trouble.
This is an intermediate to advanced challenge. The core pipeline is the same whatever you build it with, so pick whatever you are most comfortable with for image processing and, if you go down the machine learning route, running a model without calling out to a hosted service.
Step Zero
In this introductory step you are going to set your environment up ready to begin developing and testing your solution.
Choose your target platform and programming language. I would encourage you to pick a language that has good libraries for decoding and encoding images, because you do not want to be writing a JPG decoder by hand this week. You will also want to think ahead to the person detection step. Many languages have bindings for running pre-trained segmentation models, so it is worth checking what is available before you commit.
It is entirely up to you how the user interacts with your tool. A command-line tool, a native desktop application, and a web app are all perfectly good choices, so pick whichever suits you and the way you like to work.
One constraint to keep in mind from the very start is that the background removal itself must not be handed off to a hosted third-party API. The important thing is that no one else’s server does the work of finding the person and removing the background. Where that computation runs is up to you, as long as it is under your control: running a model you have downloaded on the user’s machine is fine, and so is compiling it to WebAssembly and running it entirely in the browser so the image never leaves the user’s device. What is not in the spirit of things is uploading the image to a hosted background removal service and getting the answer back. Part of the fun here is understanding how the magic actually works, so you want to be the one doing it.
If you go down the machine learning route, be aware that a good segmentation model can be a sizeable download, often in the region of a couple of hundred megabytes. If you are on a slow or metered connection it is worth pulling the model down early so it is ready when you reach the detection step. This is worth bearing in mind for a browser-based build too, where that download becomes something the user’s browser has to fetch.
Gather a small set of test photographs to work with. You will want a few pictures of people, ideally with a mix of easy cases (a person against a plain wall) and harder ones (someone with loose or curly hair, or a busy background). Keep them somewhere handy, because you will run your tool against them again and again.
Step 1
In this step your goal is to load an image and write it back out as a PNG with the same dimensions and an alpha channel.
Before we go any further, it is worth being clear about what the alpha channel is, because it is central to the whole challenge. A colour image is made up of pixels, and each pixel usually has three values: how much red, green, and blue it contains. Each of those values normally runs from 0 to 255.
The alpha channel adds a fourth value to each pixel, on that same 0 to 255 scale, that records how opaque or transparent the pixel is. An alpha of 255 means the pixel is fully opaque and you see its colour completely. An alpha of 0 means it is fully transparent and you see straight through it to whatever is behind, as if that pixel were not there at all. Values in between give partial transparency, which is exactly what you will need later for soft edges.
So when this step asks you to make the whole image fully opaque, you are giving every pixel an alpha of 255. JPG images do not have an alpha channel, which is why they can never be transparent, but PNG images do. That is the reason the output of this challenge is always a PNG: it is the transparency, stored in the alpha channel, that lets the background disappear while the person stays.
Your tool should accept a single input image in either JPG or PNG format. Once it has decoded the image, it should write out a PNG that has an alpha channel and exactly the same pixel dimensions as the input. For now, do not try to remove anything. Just keep every pixel as it is and make the whole image fully opaque, giving every pixel the maximum alpha of 255. This gets your decode and encode pipeline working end to end before you add any of the clever detection logic.
Getting this foundation right matters. The rest of the challenge builds on top of a reliable “read an image, produce a same-sized PNG with alpha” pipeline, so it is worth making sure this is solid.
Testing: Run your tool against a JPG and a PNG from your test set. Open the output files and confirm they look identical to the inputs. To check the dimensions and the alpha channel without having to eyeball them, reach for a command-line tool like exiftool:
exiftool -ImageWidth -ImageHeight -ColorType output.pngYou should see the width and height matching your input exactly, and a colour type of RGB with Alpha, which confirms the alpha channel is present. If you prefer ImageMagick, magick identify -format '%wx%h %[channels]\n' output.png does the same job, printing something like 1200x800 srgba, where the a on the end tells you there is an alpha channel.
Step 2
In this step your goal is to automatically detect the person in the image and produce a mask.
This is the heart of the challenge. Your tool needs to work out, for every pixel, whether it belongs to the person or to the background, and it must do this automatically. The user should not have to click, draw, or mark anything to point out where the person is. Given a photo, your tool figures it out on its own.
The output of this step is a mask, which is simply a second image the same size as the input where each pixel says how much that location belongs to the person. A common approach is to run a pre-trained person segmentation model yourself and use its output as your mask. Remember the no-hosted-service rule from Step Zero applies here.
You do not have to invent the detection method from scratch, and there is a lot of prior art to lean on. Here are some starting points worth a look, and it is up to you which route you take:
U²-Net is a salient object detection network that powers a lot of open-source background removers. See the paper “U^2-Net: Going Deeper with Nested U-Structure for Salient Object Detection” (arxiv.org/abs/2005.09007) and the code at github.com/xuebinqin/U-2-Net. The popular
rembglibrary (github.com/danielgatis/rembg) is built on it and makes a good reference point for what “good enough” looks like.DeepLabv3 and DeepLabv3+ are general semantic segmentation models that include a “person” class and ship ready to use in libraries such as torchvision. The paper is “Rethinking Atrous Convolution for Semantic Image Segmentation” (arxiv.org/abs/1706.05587).
MediaPipe Image Segmenter and Selfie Segmentation from Google (ai.google.dev/edge/mediapipe/solutions/vision/image_segmenter) are designed to run efficiently on-device, including in the browser through WebAssembly, which makes them a natural fit if you are building a web app.
Segment Anything (SAM) from Meta (segment-anything.com, paper at arxiv.org/abs/2304.02643) is a powerful, more recent option that can produce very clean masks.
MODNet (arxiv.org/abs/2011.11961, github.com/ZHKKKe/MODNet) and BiRefNet (github.com/ZhengPeng7/BiRefNet) focus on high-quality matting and fine edges, which will be especially handy when you reach the hair in Step 4.
Whichever you pick, you will typically load and run it with a runtime such as ONNX Runtime, PyTorch, or TensorFlow, all of which can run on the user’s own device and so keep you within the no-hosted-service rule.
If you would rather build the detector yourself from scratch instead of reaching for a trained model, that is a brilliant way to learn, and the trick is to start with the easy case and work up. A good first option is to lean on the plain background in your easy test photo: treat the pixels around the border of the image as background, then grow outwards to every connected pixel of a similar colour, and whatever is left in the middle is your person. That is really just two classic ideas stitched together, colour keying and a flood fill, and it will give you a usable mask on a simple photo with no machine learning at all. From there, a natural step up is GrabCut, a well-known graph-cut foreground extraction algorithm (the paper is “GrabCut: Interactive Foreground Extraction using Iterated Graph Cuts”, dl.acm.org/doi/10.1145/1015706.1015720, and there is a clear walkthrough in the OpenCV docs at docs.opencv.org/4.x/d8/d83/tutorial_py_grabcut.html). GrabCut normally wants a rough box around the subject, which you can supply automatically by assuming the person is roughly central or by running a lightweight face detector such as an OpenCV Haar cascade to locate them. Other classic building blocks worth knowing are Otsu’s thresholding (en.wikipedia.org/wiki/Otsu%27s_method) and the watershed algorithm. A from-scratch approach will not match a modern model on messy backgrounds or fine hair, but it will teach you a great deal about why those models exist.
To make this easy to inspect, have your tool save the mask out as its own greyscale image while you are developing. That way you can look at it directly and see how well the detection is working before you use it for anything.
Testing: Run your detection against your test photos and save the masks. Open them and check that the person shows up as the bright, filled-in region and the background shows up as dark. Try your easy photo first (a person against a plain wall) and then a harder one. The mask does not have to be perfect yet, but the overall shape of the person should be clearly recognisable.
Step 3
In this step your goal is to use the mask to make the background transparent.
Now that you have a mask, combine it with the original image. Every pixel that the mask says belongs to the person should keep its original colour value untouched. Every pixel that belongs to the background should be made fully transparent by setting its alpha to zero. The result is a PNG, still the same dimensions as the input, where the person floats on a transparent background.
Testing: Run the full pipeline on your test photos and open the output PNGs. The quickest visual check is to composite each cut-out onto a bright, solid colour that contrasts with the original photo, for example a strong red or green. With ImageMagick you can build that composite from the command line:
magick output.png -background magenta -flatten check.pngOpen check.png and the person should appear against magenta with none of the original background showing through, and their colours should match the original photo.
You can also automate a few checks with ImageMagick so you do not have to rely on your eyes at all:
# There should now be some transparent pixels, so this prints False
magick identify -format '%[opaque]\n' output.png
# A corner pixel is almost always background, so its alpha should be 0
magick output.png -format '%[pixel:p{0,0}]\n' info:
# The average alpha across the image should sit between 0 and 1,
# roughly the fraction of the picture the person fills
magick output.png -alpha extract -format '%[fx:mean]\n' info:The first command should print False, confirming the image is no longer fully opaque. The corner pixel should come back fully transparent, something like srgba(0,0,0,0). The mean alpha should be greater than 0 and less than 1, which tells you some of the picture has been kept and some made transparent, rather than the whole thing being wiped out or left untouched.
Step 4
In this step your goal is to produce smooth edges around the person, including the tricky areas like hair.
If you composited your Step 3 output onto a new background, you probably noticed a hard, jagged edge, and hair almost certainly looked like it had been cut out with blunt scissors. Real photographs do not have a crisp line where a person ends and the background begins. Strands of hair are semi-transparent, and edges blend gradually. To make a cut-out that composites cleanly onto a new background, the alpha channel around the person needs to transition smoothly rather than jumping straight from fully opaque to fully transparent.
Refine the edges of your cut-out so that boundary pixels take on partial transparency. Pay special attention to hair, which is the classic hard case and the thing that separates a convincing cut-out from an obvious one.
Testing: Composite your refined output onto a new background again, ideally one that is quite different in colour from the original. The edges should look soft and natural, and wisps of hair should blend into the new background rather than showing a harsh outline or a fringe of the old background colour. Compare a before and after against your Step 3 result on a photo of someone with loose hair. The difference should be obvious. It is also worth peeking at the alpha channel on its own, for example with magick output.png -alpha extract edge-check.png, where you should see a soft grey halo around the person rather than a hard black-and-white boundary.
Step 5
In this step your goal is to give the user control over the output filename. We’re half way so this is an easy one to give you a quick rest.
Your tool should let the user specify where the output should be written. When they do not specify a name, your tool should choose a sensible default derived from the input filename. Crucially, the default must never overwrite the input file. Someone processing photo.jpg should get their cut-out saved somewhere else, not have their original silently replaced.
Testing: Run your tool on an input without specifying an output name and confirm a sensibly named output file appears and that the original input is untouched. Then run it again specifying an explicit output name and confirm the file lands exactly where you asked. For good measure, try an input that is already a PNG and confirm the default output name does not collide with it.
Step 6
In this step your goal is to handle invalid input files cleanly.
Not every file with a .jpg or .png extension is actually a valid image, and users will inevitably point your tool at something that is not an image at all. When your tool is given a file that is not a valid JPG or PNG, it should report a clear error message explaining what went wrong and fail with a non-zero exit status (or the equivalent error signal for whatever interface you built). It should not crash with an unhelpful stack trace, and it should certainly not write out a broken or empty output file.
One thing that catches people out here is truncated JPEGs. Many mainstream JPEG decoders are deliberately tolerant and will happily decode a truncated file into a partial image without raising any error at all, so a test you expect to fail passes instead. If you hit this, you may need to validate the file yourself rather than relying on the decoder, for example by checking that a JPEG ends with its end-of-image marker. Truncated PNGs, by contrast, tend to fail cleanly almost everywhere, so they are a gentler test case.
Testing: Start with the easy case. Create a text file, rename it to have a .jpg extension, and run your tool against it. You should see a clear error message and, if your tool is a command-line one, a non-zero exit status.
Next, make a truncated JPEG by keeping only the first slice of a valid one, which chops off the end of the image data and the end-of-image marker:
head -c 2000 valid.jpg > truncated.jpgDo the same for a PNG:
head -c 500 valid.png > truncated.pngRun your tool against truncated.jpg and truncated.png. Both should be rejected with a clear error and a non-zero exit status, and no output file should be produced in any of these cases. The truncated JPEG is the one to watch, since a tolerant decoder may quietly accept it. If that happens, this is exactly the case the note above is warning you about, and it is a sign you need to validate the file yourself.
Step 7
In this step your goal is to process an entire directory of images.
Instead of just a single image, your tool should also accept a directory as input. When given a directory, it should find and process every JPG and PNG image within it, and it should do so recursively, descending into subdirectories to find images nested further down. Each image gets the same background removal treatment you built in the earlier steps.
Testing: Create a directory with a few images in it, plus a subdirectory containing a couple more. Point your tool at the top-level directory and confirm that every image, including the ones in the subdirectory, gets processed. Drop a non-image file into the directory as well and confirm your tool sensibly ignores it rather than trying to process it.
Step 8
In this step your goal is to mirror the input directory structure in the output.
When processing a directory, the output needs to be organised, not dumped into a single flat folder where files from different subdirectories collide. Your tool should mirror the input directory structure under the output directory. In other words, each output image should be written to the same relative path as its source image. An image at people/holiday/beach.jpg in the input should produce a output file at the matching relative location under the output directory.
Testing: Run your tool against the nested directory from Step 7. Inspect the output directory and confirm its structure mirrors the input. The output image for the image in the subdirectory should sit inside a matching subdirectory in the output folder, at the same relative path as its source. There is a subtle collision to watch for as well: because every output is a PNG, an input folder containing both portrait.jpg and portrait.png would map both output images to the same output name and one would clobber the other. Put a matching pair like this in a folder, run your tool, and decide how you want to handle it so that no image is silently lost.
Step 9
In this step your goal is to keep going when one image fails.
When you are batch processing a whole directory, it is frustrating if the entire run stops dead because one image out of a hundred was corrupt. Your tool should continue processing the remaining images when one of them fails, rather than aborting the whole run. Then, once it has finished working through everything, it should report the failures at the end so the user knows exactly which images did not get processed and why. Since a single bad file fails with a non-zero exit status (as you built in Step 6), a batch run that had any failures should also finish with a non-zero exit status once it has processed everything, so that scripts calling your tool can tell the run was not completely clean.
Testing: Build a directory containing several valid images plus one invalid file (reuse the corrupt file from Step 6). Run your tool over the directory and confirm that all of the valid images are processed and written out correctly. At the end of the run, you should see a clear summary listing the file that failed, and, for a command-line tool, a non-zero exit status. Confirm that the one bad file did not prevent any of the good ones from being processed.
Going Further
Once you have the core tool working, here are some ideas to take it further:
Add a preview or “trimap” mode that shows the mask and the refined edges side by side so you can debug tricky images.
Let the user replace the transparent background with a solid colour or another image, turning your tool into a full background replacement tool.
Support additional input formats such as WebP or HEIC.
Add a progress indicator when batch processing large directories, and parallelise the work across multiple CPU cores or a GPU.
Try more than one segmentation model and let the user choose, or compare their results to see which handles hair and fine detail best.
Handle photos with more than one person in them, and give the user a way to keep only a specific person.
Benchmark how long each image takes and see where the time goes, then look for ways to speed up the slowest stage.
If you built a native tool, try a browser-based version compiled to WebAssembly (or the other way round) so people can use it without installing anything.
Share Your Solutions!
If you think your solution is an example other developers can learn from please share it, put it on GitHub, GitLab or elsewhere. Then let me know via Bluesky or LinkedIn or just post about it there and tag me. Alternately please add a link to it in the Coding Challenges Shared Solutions Github repo
Request for Feedback
I’m writing these challenges to help you develop your skills as a software engineer based on how I’ve approached my own personal learning and development. What works for me, might not be the best way for you - so if you have suggestions for how I can make these challenges more useful to you and others, please get in touch and let me know. All feedback is greatly appreciated.
You can reach me on Bluesky, LinkedIn or through SubStack
Thanks and happy coding!
John