When facing a new problem, I like to use the Cynefin framework to assess the situation and decide the next steps. Software developers work mostly within the domain of complex problems, and therefore, we must deal with many unknowns in our understanding of both a problem and its solution. Sometimes it’s even difficult to formulate the right questions to ask because of our limited understanding.
There are some particularly challenging problems that can even render technical knowledge useless, as a completely different skill set is needed to navigate through them: problem-solving skills. I’d situate the skills to solve a complex problem closer to an art, with wit and imagination paving the way towards a solution. The more complex problems you face, the easier it becomes to tackle them; just like most activities, practice makes perfect. Keep reading for some of my favourite problem-solving tricks.
Research
Search the problem on the Internet, read the documentation or credible source about the topic, check if someone else faced the same problem and how did they solve it. Choose the keywords for your research carefully, but feel free to add new relevant keywords or remove those that generate too much noise in the search results. Also, don’t be afraid to go beyond the first results page in your favourite search engine.
Depending on the stage of your research, you will have to switch between different reading strategies: scanning for certain words, skimming to get a general idea, or intensive reading to get details.
Fail safely, fail fast
Sometimes we’re granted the wish of being able to quickly check whether a solution works or not. For example, a piece of code that compiles but throws an exception in runtime: we’d be able to make some changes in the code and execute it again to check if it works.
Invest time in trivializing the reproducibility of the problem. Make sure to work in a safe environment where you can make experiments without putting at risk production code, data, or infrastructure. Then aim to minimize the time to reach the point of failure. For that purpose, try disabling or bypassing pieces of code that are irrelevant and could slow you down, being mindful that the disabled code is independent and does not contribute to a false positive solution. Consider also automating manual processes and stubbing responses from external components.
In addition to that, if there is still a long process until you can determine whether a solution works or not, write down the steps so you don’t forget and someone else can reproduce it too.
Experiment
Most times you won’t find the right solution in your first attempt. It’s an iterative process in which your understanding of the problem grows with every attempt to solve it. At the same time, a better understanding of the problem will make you recognize patterns that lead you to a better understanding of how the right solution looks like.
Formulate a hypothesis and develop a way to test it. Iterate until you recognize a valid solution while gaining knowledge to make a more relevant hypothesis. During this process you may come up with new ideas: write them down and don’t discard them without reasoning first, they might hint you towards the right solution. However, don’t get overly attached to a specific idea, you may lose resources trying to make it work when it simply doesn’t.
Build up your resilience
Mentally prepare yourself for failing often, reaching dead ends and having no clue how to continue. It’s OK and happens to all of us, new and experienced professionals alike. It’s important to see failure as valuable feedback, which at the very least will allow you to discard a particular idea and thus reduce your search space.
Arm yourself with patience and keep trying, look for a different approach, but don’t get too obsessed with it. Realize that, for most problems, it’s a matter of time and persistence until you find a solution —be it better or worse— which is acceptable.
Divide and conquer
It’s easy to feel overwhelmed when facing a problem which is not trivial to solve as a whole. For those problems, I suggest to first identify sub-problems, parts of the initial problem that can be isolated and tackled individually. This means that you can tackle one problem at a time if you’re working alone, or split the tasks among your team (depending on the problem). Some sub-problems can be further divided into even smaller sub-problems, you can adjust the granularity until you feel comfortable with the problem size.
Watch your steps
Try one single thing at a time, something simple enough for you to reason about it and understand its behaviour, if it works or not. My rule of thumb is: the more complex the problem is, the smaller the steps to take. Don’t be afraid to go back to the very basics if the complexity of the problem demands it. For instance, try adding some meaningful logs or debugging your code.
Visualize
Complex problems usually involve a number of interactions between classes, components, modules, systems… so it’s hard to have a clear mental picture of what’s going on. I recommend making a diagram of the problem scope and how your solution fits in. Even if it’s just for yourself, it will help to have an instant overview of the problem domain. What’s more, you will surely improve your understanding of the scenario in the process of making the diagram.
Diagrams can be useful for other fellow developers, like your teammates, to align on the problem status, help you to solve it, or understand why a certain solution was chosen. If you intend to share the diagram, make sure to keep it simple and not to mix layers of abstraction. I use the C4 model as inspiration, tweaking it to my needs.
Ask for help
Ask around if anyone has faced a similar problem, you could get some good lead from someone near you. Pair programming is a useful practice for tackling complex problems, as you will have to verbalize your reasoning and reach agreements about which direction to take. If you cannot count on a pair, an alternative technique is the rubber duck debugging (instead of using a rubber duck you could also grab anyone who happens to pass near you).
Another option is to leave a message in some specialized forum, mailing list, or site like StackOverflow and hope that someone will answer. Make sure to formulate a specific question with enough context, provide a minimal, reproducible example, and explain the relevant approaches you already tried but didn’t work.
Breaks
Take a good break and do something else for a while if you have the opportunity. Becoming fixated with a problem can make you experience a “tunnel vision” effect to your mind, and could even bring you close to burnout. Having regular breaks is beneficial for digesting the problem status and, sometimes, unexpectedly come up with bright ideas.
Don’t settle
Congratulations, you found a solution! But don’t settle just yet. Take some time to ponder which implications does it have in the overall picture of the problem. With your renewed understanding of the problem you could soon find another solution, maybe an improvement over the first one, or something completely different. Sometimes you end up with a bunch of solutions and you can even create hybrid solutions with the best bits among all of them.
Write down the pros and cons of each solution, considering your non-functional requirements like performance, readability or extensibility. It’s a good idea to get feedback from your peers as well and discuss why do they prefer one solution or another. Keep also in mind Occam’s razor, solutions with the fewest assumptions tend to be the best.