Developer Progression as a function of navigating complexity
How to become a (more) senior developer
I came across this paper which looks at ways to teach students complex systems courtesy of Jessica Kerr’s blog.
Based on how students reason about complex systems, they were placed in 4 categories in order of increasing sophistication: Completely Clockwork, Somewhat Clockwork, Somewhat Complex and Completely Complex.
“Clockwork responses are those that show deterministic, linear, single-cause, non-networked, centralized, or static system interactions or states, whereas complex responses are those that demonstrate nondeterministic, nonlinear, multiple causes, networked, decentralized, or dynamic system interactions or states”.
In my experience this translates extremely well to the progression of a developer and their understanding of the complexity of the ecosystem they operate in. So, I wanted to attempt to derive a learning progression for developers.
At each level of development, I recommend a few books. Each book hopefully is a thread you can pull on, if you want to go deeper into a particular topic.The book recommendations are subjective, but I believe that one can chart their own curriculum following the basic model. The recommendations that I make are just based on my own experience and aim to serve merely as guidance.
When it comes to progression to the next level, my general advice would be to master the level you are operating on and then aim to understand whats needed to operate at the next level.
At this level most of the work is involved in making changes in a single service or component. This is where most of the ‘coding’ happens. The amount of time that will be spent reading your code will be multiples of the amount you would take to write it. You will work more with existing code rather than writing new code. A corollary of this fact is that you need to know how to make your code easy to change.So your focus should be on writing readable code and making it easy to change.
Your scope of work will probably within your team and its very likely that your circle of influence, impact and visibility would mostly be your team as well. The abstraction at which you operate in is your team.
The advice I would give at this level is to optimise for seeing problems solved through different reference points.
My recommendations for this level would be:
As you begin to build familiarity and also gain experience, you might see that increasingly you begin to look outward from the immediate scope of your team. You begin to try to understand how the surroundings affect the goals of your teams. You start going on organisational side quests to achieve your team’s goals. The abstraction at which you operate is still typically your team but the way you look at things becomes more outward looking.
In modern software development, distributed systems are ubiquitous. Even when you are making a change to a single service, that service most likely lives in an ecosystem of other services and a plethora of other distributed tools. As you grow in seniority you need to understand how best to interact with other services and also the tradeoffs involved with each tool.
Books for this level
As you gain more experience, you would have ideally developed some calluses, had a ring side view of some heart stopping close shaves. This will prepare you to operate at next abstraction: teams of teams. You would be advising on how to organise software and teams best to make teams successful in aggregate. You could be either leading migrations to address technical debt or reclaiming unreasonable software. It will also probably be your call as to when to do what.
A practitioner operating mostly at this level needs to be able to use models to explain the overall system or domain they are operating in and its interactions with its neighbouring domains. These models are not just to explain the software system but also the teams that surround that software. You need to understand the best models for organising teams to drive maximum impact. Additionally, as the operator of complex systems they wear two hats : that of the producer and that of the protector against failure. The practitioner needs to be able to handle this duality well.
Books for this level
The abstraction that you operate in now is the entire organisation or in bigger companies, an entire division or department. Your focus is primarily on what to build rather than the how. You will be doing a lot of writing, since you need to communicate at scale.
According to the paper, a completely complex system is a dynamic process, which is constantly in a state of flux. Any patterns here are emergent. Changes have both short term and long term repercussions. You also realise that the system is self organised for the most part. This sounds a lot like the modern organisation. At this scope you are typically operating as the technical leadership. You need to be able to form technical strategies grounded in the needs of the business.
Books for this level
So to summarise, this is what we end up with as a progression plan for developers over varying degrees of complexity.
There is a pithy saying - “All models are wrong, some are useful.” The model I have described here is not perfect. The skills and responsibilities needed at different levels are at a continuum, not discrete. I merely aim to expose you to the different abstractions that engineers operate in based on where they are in their career. I hope you found this useful.
If you are here, follow me on twitter.