Cognitive load & team composition (Part I)

 

 

If somebody asked me: "What is the primary reason of IT projects failure?" 

My answer will be: "Thats depends, but first tell me what is the cognitive load on roles in that project".

 

Maybe you are an individual contributor in some company on the edge of the burnout, accidentally finding this article. Or may be you are a manager wondering why your team/deprtment(s) fail  to match performance expectation of yours yet another quarter in a row. It's out of my power to promise you a cure to your problems (as the cure is usually within you) -- but I hope to shed some light on that very obvious and very much avoided elephant in the room, called Cognitive Load.

 

What is Cognitive Load?

In cognitive psychology, cognitive load refers to the amount of information human working memory can process at any given time [mcw]. 

For people in IT, an easy way to think about this is to apply the CPU-Memory-Disk model to humans: for any given physical computer, the amount of working memory is limited as well as the sizes of CPU caches; additionally there are physical limitations on bus throughputs for both CPU-Memory and Memory-Disk connections.

Just like running an application on computer means that its working code needs to be loaded into computer memory, solving work tasks by humans means loading "programs" into the working memory of the brain. And just like having too many applications open on a computer makes all of them non-responsive -- loading too many "programs" into human working memory incapacitates. Similarly, just like it takes time to open application on computer -- it takes time for a human to switch between contexts (like a switch between devops and business-problem code specifics); the more context switching is required, the less brain-time left for the actual work.

There is a subtle difference in cognitive overload handling between humans and computers: while operational system of computers would keep track of things, although slow -- human brain tends to cut corners and to forget what it finds in the situation as less essential.

There is a physical limit on how many details an individual can hold in their head. Expecting an individuual to do beyound this limit is the same as expecting a 10 litre bucket to hold 11 litres of water.

So, my dear fellow, you forgot that required course deadline? Or you forgot to update that flaky test failing every third time in the pipeline? Or may be new module is not hexagonal enough? Your presentation on the review haven't spoken to your audince as it changed? -- not your fault, simply your workplace is mismanaged. Nevertheless, regardless of the environment -- there is always something we can do. 

..and by the way, workplace is most probably mismanaged because of the cognitive overload of managers, they are humans too, right?..

We are going to look at team composition with cognitive load in mind, but lets reflect first on some common-place situations where cognitive overload snowballs into a wider project failure.

How did it come to this?

Lets say we completed the first stage -- we realised that we are in some sort of a pit, things are not working and some of our naive attempts to improve things have failed. The next thing we naturally ask ourselves -- how did it come to this? Why things used to work X time ago but they apparently don't work today? Here are some examples:

Just another day in yet another software company. Senior developer was tasked to create yet another kubernetes-hosted micro-service. However there was a recent change -- to synchronise a wider organisation code-base practices, there was a new set of instractions released (involving package naming, class naming, when and when not to create interfaces etc). The micro-service was importnat for the product at the same time and there was a firm deadline. Once the micro-service was released it was found that the very key happy path it supposed to serve was not met. The code however met the organisational criteria and was approved by multiple other team members, unit test coverage was good as well.

Real and classical story. The type of a story where software developer is "punished" in some way in the end, but with no fault on their side if looked objectively. The two branches of power -- the product and the engineering -- had a conflict of interest putting it on the developer team to resolve. For various psychological reasons the direct technical requirement tends to take the priority. Once the conditions of cognitive overload are met (understanding new requiremennts of writing code while having to do it fast), brain has to forget something -- so it forgets integration tests / some details deemed as less important, like some small word or two of the user journey.

Few things could of been done differently: (1) developer being aware of the cognitive overload can choose to push back one of the two things driving to the overload -- the deadline or the new tech requirement [typically leadership is to stubborn for those though, would say nice words but wouldn't get that it is about the physical reality of the brain]; (2) developer can choose to compromise, like solve the problem in the old way and then try to refactor the solution to meet new tech requirements; (3) ideally the leadership is aware of cognitive load impacts and coommunicates/plans changes in such way that people don't get overloaded [leadership agrees to soften deadline without having the team to push back on it].

It was a start up. The team of the dream -- sharp sharks, armed to the teeth with skills and cutting-edge knowledge. We built an initial prototype for a customer A, we set all of the CI/CD processes, architecture, testing infrastructure and principles. Customer A was vague however, and we pivoted to make the offering to customers B and C -- as they promised money if we approach things differently. We had no time to rebuild all of our processes, so reasonable compromises were made. As new year came, customer A returned to us and it was important for our reputation to get them in. Customer B now pays the biggest paycheck and asks about some rawness of the product. Engineering manager introduced DORA metrics and asked everybody to do courses on TDD. We are still behind the schedule..

In this story, the cognitive overload was following from the expansion of the product knowledge: while technologically the team was all up to date, following reasonably the best practicies (no problems with the initial rollout) -- in the scale-up stage there were pragmatic decisions made. The cost of those decisions was a hidden requirement on developers to know the caveats of internal tooling/solution to differentiate between features for A, B and C customers. To avoid cognitive overload and project failure, two obvious options appear: (1) pivot completely, so that old assumptions don't burden developers -- it means that the initial offering to client A is off the table and the pivot takes longer period of time; (2) scale up the team, to isolate the special logic area from the rest of developers and providing them with a tunnel to do their work clearly as they used to. There are of course more options in real situations, but what I hope is clear is that: (1) some decision need to be made consciously or the project/people going to burn unconsciously eventually, (2) there would be trade-offs to make, we need to let go of something less important to continue to have a firm grip on what is essential.

The company decided to add AI to its repertuar. A newly founded team of data science was to build The Model. After half a year, The Prototype was built -- and oh my, it was catching a happy path case! Sales were fast to ring a trumphet of victory, clients got queueing for our Miracle of Technology, contracts signed. With a happy smile CTO came to congratulate the team that The Model is going to be deployed in production. For a moment there was some strange aftertaste from this meeting, like soomething was off -- but there was this next meeting to attend to. A year have passed, but the deployed to production model was exactly the same as the one shown previously. More and more often the clients were giving unhappy signs and examples of the edge cases which are critical for their business. New data scientist was hired, courses on MLops scheduled for the team, line managers notified that performance is unsatisfactory..

Common for the early 2020s, the problem of companies entering new fields with old mindsets.Cognitive load on managers leads to insufficient understanding of a new field, some important red flags go through unnoticed into decisions as a result. Machine learning is an example of a field of technology requiring a very different managment approach and was doomed to fail when managed like a software project (hidden assumption that in software engiineering POC is pretty close to MVP). Cognitive load on managers also deafened the feedback from the team on complexity of the problem, so the work-distance between the POC and MVP for ML was not correctly accessed and clients misinformed on company's ability to deliver on promises. As the result, the team gets burned-out and 'punished' even though there was none of their fault.

In parallel, in this very story case, data scientists who have built the model prototype are expected to setup deployment processes for getting model into production, while the role as it exists today commonly has more than enough on its plate just to tackle the ML/AI space. Again, the hiidden old-mindset assumption of a developer-like requirements put on them by the organisation simply gets them into an overload state and slowing down/stopping the actual ML side improvement.

Things which could be done differently include: (1) don't step into the space without a proper understanding what the production version of it would involve/cost, (2) for new fields, not all of the old experiences are transferrable and its good to be aware of that before making business decisions, (3) understand cognitive load for specific company roles before making assumptions of what work they should be expected to do.

 

Why cognitive overload is so common in IT

It won't be true to say that only IT suffers from cognitive overload -- any driver knows that even a hand-free phone call created dangerous situation at least once. But I think that not many fields exist in such continiously changing environment as it is in IT.

Drivers drive lorrys, but the rules of the road don't change that much or thhat fast, cars share the same way of interacting with them -- although there are disomforting changes. Hammers haven't changed much across the centuries on the construction work -- although there are many tools out there today not existing 30 years ago.

In the IT, the very fabric of work evolves constantly -- the way the code is written that is. Things learned 10 years ago in their direct form are not applicable today -- all of the tools has changed, evolved, work on different assumptions with different requirements.

This requirement to continiously stay up to date with the very technology used to solve problems and the extend of the technology evolution becomes the hidden requirement on any role in the IT, often taken for granted. Now, everybody understands the project/product evolving complexity as it develops, but it is easy to miss on how fast the project/product parts can become technologically outdated, how the sheer volume of the project multiplied by some technology evolution velocity creates amounts of work which needs to be done just to keep everything afloat. This "for granted" requirement gets indirectly applied to developers, but it has impact on the available working memory of developers.

If you followed me so far, you probably can see my lead -- to the detachment between the management and engineering on the ground. The management doesn't need to keep up with the ever evolving tools, their tools stay the same -- they only need to use well known tools to deal with the ever-changing business environment. The IT people on the ground however need to both: continiously adjust the very fabric of what their tools are and to apply the fresh learned-tool to solve business problem (like every the hammer weight is different, screw-head sockets being different every day). 

What follows is that managers has a human assumption "everybody is as cognitive loaded as I am", because the cognitive load of simply surviving in technology is not known to them or gets forgotten over time on the new position. Some decisions push people into the overload state, details gets forgotten. Forgotten dettails lead to mistakes which need to be fixed, anxiety of forgetting -- even more things to keep in working memory, some other details forgotten because of it. Management takes "actions" to fix it by requiring people to do something differently -- new cognitive load, other details get fogotten, even less work done. It becomes a vicious cycle with a hidden blame on developers being not good enough eventually, and things just get toxic behind the polished corporate smiles. Sounds familiar?

I find it especially ironic, provided that in IT everybody works with computers and knows how computing limits on memory influence responsivness of the system. But the same principles are not recognised in the daily work of "human-computers" "compiling" and "testing" the code of the system -- somehow there is this believe that people always can be pushed beyound the limit, that there are those "good" and "highly-performant" and if those are found and hired there will be no problems. Funnily enough, the "highly-performant" people I've met so far are very good in avoiding the micro-management and in pushing back the cognitive overload done by mis-management -- seems like it is a skill worth building.

In the next part of this discussion, I'm going to reflect on how properly adjusted ways of working and team roles help to distribute the cognitive load across the team and to detect which aspectes of the team needs to be scaled. In other words -- how to keep your team happy and it's productivity -- high.

Part II