Algorithms are at the very core of successful and efficient development. You’ll use them as you learn to code, you’ll be asked about them in technical interviews, and they’ll likely be part of your day-to-day development work.
Learning common algorithms individually is helpful, but what’s even better is getting used to algorithmic thinking. If you can train your brain to understand and follow algorithmic logic, writing your own algorithms will become much more intuitive.
Here to explain algorithmic thinking and how to work through problems with this method is Ethan Urie of Learn to Be a Developer.
Take it away, Ethan!
Disclosure: I’m a proud affiliate for some of the resources mentioned in this article. If you buy a product through my links on this page, I may get a small commission for referring you. Thanks!
Do algorithms make you anxious? Do they seem complicated and too hard for you? Or are you unsure about what they even are?
If you feel any of these things, or feel like you can’t be a real developer unless you know them, you’re not alone.
Algorithms and data structures are bugbears of the software development world. Traditionally educated developers were probably taught about them in one or two classes and self-taught or bootcamp developers often aren’t exposed to them at all.
Even so, for most beginner developers, algorithms and data structures are the sources of a lot of anxiety and imposter syndrome.
Since I wrote about some data structures for Learn to Code With Me a while back, this time I’m going to focus on algorithms. More specifically, I’m going to cover how to start thinking algorithmically.
Using Algorithmic Thinking for Fun and Profit
But mostly profit.
Thinking algorithmically is a mindshift from how we, as people, usually think. It is more of a systematic way of thinking through problems and solutions in a way that’s similar to how a computer would run.
But that’s surprisingly difficult. We all have unconsciously built up shortcuts, assumptions, and rules of thumb that we use to help us solve everyday problems without thinking about them.
For instance, take the simple task of sorting 10 numbers. As it stands, we can take a look at them, tell pretty quickly what the order should be, and arrange the numbers correctly.
However, we’re not used to breaking our thought process down into its individual steps and translating that to what computers can do. For instance, computers can’t jump to general spots in a dictionary to find a word based on its spelling. A computer has to have very specific instructions on where to start (plus they can’t do truly random numbers).
Another example is looking up a name in a phone book or a word in a dictionary. As humans we generally know, given a name or word, how far into the book we need to start our search – if it begins with A we start near the beginning. If the name/word starts with M, we jump to around the middle of the book and so on.
For beginner developers, it’s tricky to break that thought process down and translate that to computable steps, since computers generally can’t make the types of judgement calls about where in the list to start.
So how do you think algorithmically?!
Like all skills, it’s learnable and just it takes practice. It’s like getting a feel for how to organize code into classes in object-oriented design (OOD). You do your best, and iterate on the solution to improve the weaknesses that you find later.
Like OOD, though, there are some guidelines that can help you get there faster.
At its essence, algorithmic thinking is thinking about how to solve a problem in a systematic way. It’s about:
- Defining the problem clearly
- Breaking the problem down into small, simple parts
- Define the solution for each part of the problem
- Implementing the solution
- Making it efficient (eventually)
☝️ Back to top
But, Wait, What Is an Algorithm, Exactly?
Also, why should I care?
Before I started learning about algorithms, I was intimidated by them. I felt that they were complicated, hard to learn, and heavily mathematical. Algorithms felt complex and beyond my ability to understand. I was pretty sure that they were important to know (they sure sounded important), but I didn’t know the best way to learn more about them.
It makes sense, given that we hear about tech companies’ new algorithms constantly, like Google’s new search algorithms and PageRank tweaks, or Uber’s algorithms for finding the best ride.
We hear about and experience algorithm questions in whiteboarding interviews. We’re constantly told about how important and complicated they are…but mostly by people in the media who don’t know much about them.
Maybe my personal anxiety was because the word “algorithm” sounded so much like “logarithm” that I associated algorithms with math and difficult concepts.
I’m not sure, but now that I know what they really are, I want to help you to overcome any fear, uncertainty, and doubt about them too.
“Algorithm” is a general term that has an overblown weight to it in software development, in my opinion.
The simple truth is that algorithms are just ways to do things. They’re processes to solve a type of problem.
- Finding a word in a dictionary
- Sorting a list of numbers
- Generating the Fibonacci sequence
- Finding prime numbers
- Baking a cake
- Doing laundry
- Making a peanut butter and jelly sandwich
Now, to be fair, many algorithms that make the news these days are impressive and complicated and require deep knowledge of computer science theory, machine learning, and mathematics. I certainly don’t understand most of them.
However, just because there are difficult algorithms doesn’t mean that all algorithms are that complex.
To start thinking algorithmically, you can start thinking through challenges one of two ways, breaking the problem down and building the solution up.
Our natural tendency as developers is to build the solution first. However, I would actually advocate starting with breaking the problem down and only then building the solution up.
☝️ Back to top
Breaking a Problem Down to Develop an Algorithm
Don’t worry if you don’t naturally start with breaking a problem down. Breaking a problem down first doesn’t come naturally to most developers and aspiring developers, including me.
However, taking the time to break down a problem helps us better understand the problem and see how solutions naturally spring from that understanding.
This is especially helpful to avoid being overwhelmed when facing a new problem that is outside of your comfort zone.
Take the problem of looking up the word “bumfuzzle” up in a dictionary (bumfuzzle means “to confuse”). In this case, let’s assume that the dictionary is a list of words, in the general, English-language sense of the word “list.”
For search problems, we generally need to know
- Where and how to start the search
- When and how to stop the search
- How to compare two list items to determine which is “before” another
- How to continue the search when you haven’t found the word yet
The better an algorithm is, the shorter the time is between one and two, and the fewer times you need to do number three and four.
For our dictionary search problem, we can expand on the above three bullets and break the problem down into:
- The expected order of the words (e.g., alphabetical English)
- How to compare two different words and determine which should be before the other one — related to point one
- How we know we’ve found the word
- How we know if the word is not in the dictionary
We will assume that points one and two are taken care of by using the English alphabet for the order and that we can easily determine the order of words by the alphabetic order of their letters.
That leaves the last two items.
Naturally, we know we’ve found the word when the word at our search position is exactly the same as our search word. Most programming languages these days can tell if two words are the same or if one is alphabetically “before” another one. So that one’s taken care of.
As for item four, if we get to the end of the search and still haven’t found the word, we know it’s not in the list. This assumes we’ve done the other parts correctly, though.
Start coding now
Stop waiting and start learning! Get my 10 tips on teaching yourself how to code.
Getting Started
Now, how do you get started thinking through these problems in the first place? You have to start somewhere, right?
To start working on a problem, I find it useful to initially think through it using a very small sample set of data. The size should be easy enough for you to think through and physically write or draw out, if necessary.
In math, there’s the process of proof by induction. It’s the idea that if you can prove that a mathematical formula works for a case of 1 item, and you can assume it’s true for n-1 items (where n is some unknown number), then, you try to prove it for a case of for n items. If the formula works for n items, then the formula should work for any number of items.
Using that concept for our algorithm: if we can make it work right with 1 item in the dictionary, and then make it work with 10, we can probably make it work for any number of items. To confirm, you’ll ultimately test it with a lot more.
This slow build-up helps you understand the details and find the subtle traps of the problem.
☝️ Back to top
Building an Algorithm Up
It’s probably very tempting for you to start with trying to build the solution. I’m the same way. Building a solution to a problem is the entire point of coding to begin with, so it makes sense that we’d want to get into it immediately.
I’ve had instances where I jumped into solving the problem too quickly and put in a few days of work before realizing that I can’t solve it the way I thought because I didn’t understand the way the existing code affected the problem. It’s like getting halfway through baking a cake and realizing you didn’t have all the ingredients, or needed to know a certain whisking technique to make it properly. 🎂
I’ve learned the hard way: if we don’t fully understand a problem, we won’t build the best solution — or maybe we will, but it will take us longer to get there.
Now, finally (I can practically hear you sigh with relief), you’re ready to start building the solution since you’ve broken the problem down so you understand its pieces.
At this point it’s down to building a solution in parts. Each part of the solution addresses one or more of the parts of the problems you’ve identified.
How you build the solution now, is somewhat flexible. For me, I like to build the easiest, most straightforward part first so I have some success and a framework to build the rest of the solution on.
In the case of the dictionary search, I might build up the solution like this:
- Write the loop (or recursive function)
- Write the code to exit the loop/recursion if the word is found
- Write the code to exit the loop if the word is not found and you’ve searched the whole dictionary
- Write the code that decides how to search next if the word is not found but the dictionary is not exhausted
- Fix edge cases, boundary condition problems and other details (like what happens if you pass an empty list?)
Searching and sorting algorithms are good places to start getting a handle on algorithmic thinking because they’re related, fairly self-contained, easy to draw and step through, and they build up in complexity.
The Real World ™
This process is often less obvious for most real-world development tasks though.
In most developer jobs, you’re fixing defects in an existing codebase or adding features to it.
You’ll still need to break a problem down to its smallest parts and start building up the solution in pieces to address each of those parts. But it will probably involve designing classes, their methods, and tying them all together.
As you slowly build up from small and simple to big and complex, you keep each step manageable and isolated. You introduce changes slowly and so when things go pear-shaped, the possible causes are small and easily discovered.
Also, it’s important to keep in mind that you may not know how to implement a fix or feature at first. In fact, it is quite likely that you won’t know how to implement it at first. This is especially true if you’re adding features or fixing a defect in an existing codebase, which is very common with most dev jobs.
I’ve had cases where I spent many days or even weeks trying different things to determine how I might implement a feature or a fix. In the end, I’d have something that works, but was ugly and inefficient. But that experimentation gave me the knowledge to go back and rewrite the code in a much cleaner, more efficient way.
Give yourself permission to experiment with solutions and accept the reality that you’ll probably have to try a number of times to get it to work, then even more to get it to be efficient and clean.
☝️ Back to top
How to Get Better at Algorithmic Thinking
Ah, one of the many age-old questions. How do we get better at any skill?
The simple — and annoying — answer is to practice. Like OOD and programming in general, experiencing challenges and learning from them is probably the best way to get better.
But you can speed up the process by learning about existing algorithms and implementing them yourself in different languages or different ways.
Studying Existing Algorithms
It’s inarguably a good thing to understand the algorithms that form the foundation of a lot of fundamental concepts of programming. Here you want to dive into the details of an established algorithm to understand what it’s doing and why – emphasis on the ‘why’. Personally, I need to reimplement it to understand it best.
Most conversations in books and classes revolve around a few well-known search and sort algorithms, which I’ll mention below. These specific algorithms are solid places to start because searching and sorting are two actions that nearly every developer needs to do at some point in their career. Furthermore, searching and sorting algorithms provide a good basis for understanding an algorithm’s efficiency and edge cases.
In fact, in the preface of his third volume of The Art of Computer Programming, Donald Knuth says he believes searching and sorting are great places to start because they get you thinking about:
- How new algorithms are created
- How algorithms can be improved
- How you can determine an algorithm’s efficiency
- How you can choose between different algorithms
Searching and sorting are related activities and the algorithms generally build on each other, refining and improving on prior versions. This gives you the opportunity to see how you can start simply with a straightforward algorithm and slowly improve it as you learn where (and how) it is inefficient.
Here are some of the important search and sort algorithms that every developer should be able to explain.
- Searching
- Sorting
Thankfully, you’ll probably never have to actually implement any of these algorithms as a professional developer. These days, the most efficient search and sort algorithms are provided in standard libraries that come with most languages.
However, just because you don’t have to write a version of bubble sort or binary search, it’s not a good reason to skip understanding and being able to write them.
These algorithms form the foundation for understanding algorithms as a whole. They teach you how algorithmic efficiency has real-world effects, and how newer, more efficient, and more complicated algorithms are developed.
Here are some resources you can use to learn more about the above algorithms and find additional ones to study:
Books
- Introduction to Algorithms (via Amazon) by Thomas H. Cormen and Charles E. Leiserson
- Algorithms (via Amazon) by Robert Sedgewick and Kevin Wayne
- Grokking Algorithms (via Amazon) by Aditya Bhargava
- The Art of Computer Programming (via Amazon) by Donald Knuth (formal and very math-heavy, but you can skip a lot of that)
Online
- Wikipedia (naturally)
- GeeksforGeeks
Using these resources, especially the online ones, you can follow the rabbit hole down past the common, well-known algorithms into the more specialized and esoteric ones. But often, starting with the easier ones and working your way up to more complex ones is a great way to build your understanding.
☝️ Back to top
How to Practice Algorithmic Thinking
As with every skill, practicing it makes you better at it.
However, the way you improve is different from physical skills, where you can perform it over and over again the same way and slowly get better. How many times can you write a heapsort in Python and still improve? I don’t actually know the answer, but I’d guess it’s not many.
In the case of algorithms, simple repetition isn’t going to help as much. You will get some benefit out of it, but the real task isn’t to know one algorithm really well, it’s to come up with new ones and be able to implement and evaluate them as needed.
To this end, like OOD and other coding skills, you need a variety of challenges all revolving around the need to solve a problem.
Furthermore, to get a benefit, you need to know how well you’re doing. How useful is it to take a practice test but not have access to the answers?
With coding, part of the challenge is that there are a lot of ways to solve the same problem. Without knowing how your solution can be better, how can you get better?
To accomplish this, you need to have some way of evaluating your implementation, preferably an objective one. A couple of simple ways are:
- Timing your solution. As you make changes, simply see if the time gets shorter (most languages have built-in methods to help you do this).
- Have a more experienced developer look over your solution.
- Practice with code challenges. Many online code challenge sites will give you the “right” answer to compare to, and some will actually evaluate your solution.
“Okay, okay. Where do you find these challenges?” I hear you ask.
There are a lot of places, a Google search will provide them, but here are a few to start:
These resources should get you started and provide you with a lot of opportunities to hone your skill at thinking algorithmically.
4 Courses on Algorithms and Algorithmic Thinking
In addition to practicing coding challenges, there are online courses you can take to help you understand algorithms better. Here are a few to get started:
1. Data Structures and Algorithms: In-Depth using Python on Udemy: This course teaches you how algorithms are implemented in high-level programming languages, in this case Python.
2. Algorithms for Searching, Sorting, and Indexing on Coursera: This course covers basics of algorithm design and analysis, including algorithms for sorting arrays, data structures such as priority queues, hash functions, and applications such as Bloom filters.
3. Learn Sorting Algorithms with Python on Codecademy: Learn about computational sorting by implementing different sorting algorithms yourself. Covers bubble sort, merge sort, quicksort, and more.
4. Algorithmic Thinking (Part 1) on Coursera: This two-part course goes over algorithmic efficiency. You’ll implement several important graph algorithms in Python and then use these algorithms to analyze two large real-world data sets.
☝️ Back to top
What About Optimization and Big-O?
As you learn about algorithms, you will gain an intuition about how well a given one will perform. You will be able to tell if a given algorithm is better than another as you gain experience. Most times, this feeling is based on how long an algorithm, say a search or sort, will take to run on a list, i.e., its efficiency or performance.
Developers use Big-O notation to measure and compare algorithms’ performance. Almost every interview or book on algorithms will include an explanation of Big-O and how to calculate it.
I’m not going to.
Mostly, I’m going to ignore it because I don’t think it’s important at this point in your learning. But secondly, I can count on zero fingers how many times I’ve had to use Big-O in my daily job. In interviews? Sure, but not in my actual work-work.
So a discussion of Big-O can wait, in my opinion.
Your goal now should be to get used to solving problems first. Get a feel for breaking problems down and building solutions up in a repeatable way.
That’s what this career is about. Solve problems and see how other devs have solved similar ones over the years, and you’ll naturally start to think algorithmically. Leave the Big-O for that time.
Ethan Urie is a professional software developer with over 15 years of experience in a variety of industries. He loves learning new programming languages, frameworks, and technologies. He also loves teaching what he’s learned to others and helping them become professional developers themselves. He writes posts and guides for aspiring developers on his blog learntobeadeveloper.com.
Note: This article contains Amazon affiliate links. As an Amazon Associate, I earn from qualifying purchases.