How to lead software architecture

leading software architecture

Background

“80’s rule man, best time ever! Then that Cobain had to come around and ruin it all.” is a funny quote from the movie Wrestler. Today’s software experts might also yearn for former times. In the waterfall model, the role of architecture was clear – it was designed up front, after requirement analysis and before the implementation phase. In the current Agile era, architecture is often evolving without clear goals and objectives.

Yin and Yang

According to “Clean Architecture”, the purpose of architecture is to support the lifecycle of the system. The ultimate goal is to minimise the lifetime cost of the system and to maximise programmer productivity. Traditionally, dedicated experts have been accountable for this function. In this blog, these experts are called software architects, but technical leads and stack leads are also commonly used titles. Typically, software architects have a strong technical background and good leadership skills.
One of the Agile core values is the development teams’ autonomy. Teams design, implement, test and deploy independently. Traditional Agile methodologies, such as XP (Extreme programming) and Scrum, do not mention how architecture design should be led in practice. XP advises, for example, that architecture design just emerges alongside other design.
The larger the project (i.e. multiple teams), the more complex is the product’s software itself. This means that architectural decisions will have a bigger impact. The question then arises how architecture design is led and what responsibilities software architects versus teams have?

Make your system visible

Systems thinker and the father of the quality evolution, W. Edwards Deming has said that 96% of organisational performance is a function of the organisation’s structure. Only about 4% of an organisation’s performance is attributable to the people. Part of the structure is to understand how collaboration works in the system.
The following table makes collaboration between architects and teams visible. Every row of the table represents a potential point of leverage. The columns represent different options as to how collaboration can be implemented. Left columns represent a more distributed and emerging architecture mindset (i.e. Agile) and the right columns a more centralised and formalised approach. Organisations are different, and therefore leverage points and options vary as well. Rows are also independent, so for example “Architects are part of the teams” and “Architects are dedicated mostly to architecture topics” can be used on the same project.
<– more distributed                more centralised →

Point of leverage
Option 1
Option 2
Option 3
Option 4
Hierarchy levels One level – no special roles, just team members Two levels. Developers and architects Three levels+. Team members, architects, chief architect
Scope of software architects None Architecture principles (e.g. standards, concerns, styles) Architecture principles and infrastructure & operations As option 3 plus enterprise architecture
Alignment No architects. Team member is the only role Architects are part of the teams. Architects work as free agents Architects form a team or department
Decision making No architects. Teams make architectural decisions Teams and architects make architectural decisions together Architects make most of the architectural decisions
Division of Work No architects. Teams share responsibilities dynamically Architects and teams share their responsibilities dynamically Architects have predefined responsibilities
Capacity No reserved capacity. Teams use their time on architectural topics if needed Architects/teams have reserved capacity for architecture topics Architects are dedicated mostly to architecture topics

No “one-fit for all” solution

After the current collaboration model is defined, the next step is to decide the future direction. Organisational systems are determined by their context, so “one-fit for all” solutions aren’t appropriate. Nevertheless, some conclusions can be drawn.
Typically, the long-term goal should be to move to the more distributed and self-organising collaboration model. When people understand the collaboration strategy, it is much easier to take actions toward it. Unfortunately, organisations underestimate the amount of work that the transformation requires. People have different skills and expectation levels and unlearning at both the organisational and individual level is required.
Agile is also a highly disciplined method. Business people and team members must work together daily; the cross-functional team structure is mandatory and software delivery must be frequent, to name just a few principles. If the organisation won’t bend to these demands, a more centralised collaboration model might be the valid option.
Sub-optimising is also a risk in bigger organisations. A team might violate architecture principles thoughtlessly, just to finish their sprint backlog faster. The other team might implement a similar service that has been already created by another. Architects see more easily the whole and are able to make holistic decisions.
Many organisations’ technical debt is huge as well. Products suffer wrong technologies or bad programming practices and an enormous amount of refactoring work is needed. Painful architectural decisions are often made more efficiently by a more centralised collaboration model.

A hero is required

Software architecture is not the hottest topic in the software development field. In addition to this, it is very abstract and hard to lead. However, all software has an architecture regardless of if the architecture is managed or not. Smart organisations understand that the organisation structure should be optimised for the most crucial paths of collaboration – and collaboration between teams and architects is part of it.

Graphic design
Ville Takala 
References
https://www.quotes.net/mquote/132593
The Clean architecture – Robert C. Martin
Technical leadership and the balance with agility – Simon Brown
https://eric.ed.gov/?id=ED577444
https://www.researchgate.net/profile/Varvana_Myllaerniemi/publication/233792525_Comparison_of_Software_Architecture_Design_and_Extreme_Programming/links/0fcfd50b8b54316e7d000000/Comparison-of-Software-Architecture-Design-and-Extreme-Programming.pdf
https://agilemanifesto.org/principles.html
https://sites.google.com/a/scrumplop.org/published-patterns/product-organization-pattern-language/conway-s-law

Avatar

Juhana Huotarinen

Juhana Huotarinen is a lead consultant of software development at Gofore. Juhana’s background is in software engineering and lately, he has taken part in some of the biggest digitalisation endeavours in Finland. His blogs focus on current topics, involving agile transformation, software megatrends, and work culture. Juhana follows the ‘every business is a software business’ motto.

Do you know a perfect match? Sharing is caring

Scrum is Dead

I’ve recently attended Agile events where world-class gurus state that Scrum is broken, the Product Owner is a waste, or LESs is better than SAFe. Tools are built for a purpose. You can use a tool for a certain purpose only in a certain context. When the context evolves, the tools also need to evolve.
What does not change, are the underlying principles. Misuse of a tool is easy if you don’t understand the principles. There are tons of studies regarding ScrumBut, where the Scrum tool is abused. Where the (Agile) principles are not followed, even if the (Scrum) tool is in use.

Maturity of the Organization Defines the Context

According to Reinventing Organizations by Laloux, in today’s predominant organizational model the leader in the hierarchy turns a cog that directs the workers below. Planning and execution are strictly separated. This leads to silos, rigidity, no innovation, no responsibility, no ability to advance meaningfully in the career. Laloux divides cognitive, psychological and moral maturity into five levels*. Higher levels build on top of the lower ones.

 
Name Description Examples
Tribal – “Red” A simple hierarchy to steer work and a wolf-pack mentality Mafia and Startups
Agrarian – “Traditional” Large organizations with numerous roles and steering mechanisms, like long and medium-term planning. Planning and execution are strictly separated. Control what and how. Thinking happens at the top, doing at the bottom. The underlying worldview is that workers are mostly lazy, dishonest, and in need of direction. Church, military and government
Industrial – “Orange” Profit and growth are the most important measurements. Achieved by the means of innovation, accountability and meritocracy. RnD, Marketing and Product Management departments are invented. Control what. The underlying worldview is based on material success. Multinational companies
Information – “Green” Focus on culture and empowerment to achieve high employee motivation. Highly sensitive to people’s feelings. All the perspectives deserve equal respect. Where Industrial level seeks to make decisions top-down, based on objective facts, expert input and simulations, Information level strives for bottom-up processes. Information level brings opposing points of view to eventual consensus. Where Industrial level views organizations as machines, the dominant metaphor of organizations in Information level is the family. It seeks fairness, equality, harmony, community, cooperation, and consensus. Culture-driven organizations
Adaptive – “Teal” “Post-postmodern consciousness”. No bosses, no competition. In this mindset, ego is viewed from a distance. We can see ego’s fears, ambitions, and desires: In the Tribal level, a good decision is the one that gets me what I want. In the Traditional level, decisions are determined by social norms of what is right and wrong. In the Industrial level, effectiveness and success are the yardsticks. In Information, matters are judged by the criteria of belonging and harmony. We use measures of inner rightness: Does this decision seem right? Am I being of service to the world? Network organizations

Summary of Laloux’s levels of organizational development. The names and colour codes come from Lalouxs’ book, however, they were different in the original work of Beck and Wilber.

Both consciousness and adaptability grows with the maturity of the organization

Self-Organizing

According toLaloux, in the highest level of maturity (“Adaptive”) we don’t pursue recognition, success or wealth. We pursue a life well lived. Still, the consequence might be recognition, success, wealth, or love. Adaptive organizations reveal three major breakthroughs: evolutionary purpose, wholeness, and self-management. These major breakthroughs make organizations prosper given the right context. Companies like Buurtzorg, Patagonia or Zappos show strengths of the Adaptive level:

  • Employees thrive
  • Salaries above market rates
  • Grow year in, year out
  • Achieve remarkable profit margins

Adaptive level brings three organizational breakthroughs:

  1. Principles driven decision making. Rather than trying to predict and control, Adaptive organizations try to sense and respond. Similarly, as the Cynefin framework describes working process in the complex environment, that consist of “unknown” factors. The output cannot be planned predicted, but one must proceed with principles driven decision making in an environment where it is safe to fail. Like Stephen Covey says “There are three constants in life: Change, choice and principles”.
  2. Wholeness creates an environment of safe conversations. Safe conversations invite deep listening, authenticity and vulnerability. It is assumed that everyone seeks for the common benefit, even if their opinions seem different. Conflicts are dealt with understanding. This will create smarter networks, unleash creativity and bring higher employee satisfaction. Like Douglas Stone says “The urge to blame is based.. on the fear of being blamed.”
  3. Minimum Viable Management does not mean a free-for-all. It means form-follows-need: Roles are picked up, discarded, and exchanged fluidly. Power is distributed. Decisions are made at the point of origin. Innovations can spring up from all quarters. Meetings are held when they are needed. Temporary task forces are created spontaneously and quickly disbanded again. The leaders of self-managing organizations feel, that it is less risky to deploy self-managed teams. That self-managed teams make better decisions. Like Bruce Lee stated “It’s not the daily increase but daily decrease. Hack away at the unessential.”

These three principles enable the organization to better handle uncertainty, complexity and magnitude of nowadays Agile business. Adaptive organisation work efficiently and fast, while fewer power games and egoism is in the way of creativity. More effort is used on improving individuals and organizations.

Where to Start

Team of Teams by McChrystal underlines the idea of creating nimble, transparent, and horizontal organizations. Organizations which empower their people to execute based on the concept of ‘shared consciousness’. People are empowered to trust their gut and use their best judgement based on their training and knowledge of organizational goals. The book gives good ideas of

  • Break down organizational silos by sending liaisons to other teams
  • Empower everyone to make decisions, don’t act as a decision bottleneck
  • Hold frequent sync up meetings to disseminate information
  • Set up cross-disciplinary open-plan workspaces
  • Enable people to make decisions at lower in the hierarchy and get to know their counterparts in other teams
  • Be a gardener. Plant and provide water and nutrients. No heroic micromanagement
  • Monitor data and real-time information, but stay away from control-freak and micromanagement.


From hierarchy to networks (Team of Teams way of drawing the idea)

Need for Speed

Holacracy by Robertson also seeks for knowledge economy replacement for the command-and-control organization. Robertson quotes Elliott Jaques who identified three types of organization structures:

  • Formal: the official organizational chart and the job descriptions
  • Extant: how things actually work and who really makes decisions
  • Requisite: a setup that would be most natural and best suited to the work and purpose of the organization.

The Idea of The Holacracy is to achieve the requisite organizational structure. Holacracy’s systems and processes are about helping the organization find its own unique identity and structure. In Holacracy, a pyramidal hierarchy is replaced with “circles” dedicated to specific functions. Each of the circles has a “lead” who, like a traditional manager, is responsible for assigning work and seeing that it’s completed. They, however, are not responsible for determining how the work is done. A kind of company-level crowd-sourcing.

From hierarchy to networks (Holacracy way of drawing the idea)

Fit For Purpose

There are no good or bad ways to organize. The question remains whether the structure is a good fit for the task at hand. Mature levels are better suited for the purpose of more sophisticated work, with high complexity and high skill set. And lower levels are better suited for crisis situations. The organizations are far from consistent and are a mix of both. The maturity of an organization is influenced by the context and by the leader of the organization.

Back to Scrum

Scrum is not an absolute value. Just a tool. Originally Scrum was a counterattack to Waterfall. From fixed scope to work decomposition, fast feedback, retrospectives and being close to the customer. The idea of Agile is to create high value to the customer by means of collaboration, demos, improvement and being open to change.
In this context Scrum is a process to take an organization to the next level of maturity. Scrum gives the management a feeling of top-down control by managing the product backlog. Simultaneously Scrum provides the team with a freedom and safety to make bottom-up decisions. At some point, there is no more need for the safety provided by the hierarchical Scrum mechanism. Similarly, you may evolve from component teams to feature teams.
What makes the tools like Scrum, Kanban or SAFe work are the underlying principles like

  • Agile (https://gofore.com/agile-transformation-action-part-1/ )
  • Lean (https://en.wikipedia.org/wiki/Toyota_Production_System )
  • Systems Thinking (https://gofore.com/en/what-is-systems-thinking-and-how-should-i-use-it/)
  • Cynefin (https://en.wikipedia.org/wiki/Cynefin_framework)

Agile is about adapting to an unclear situation by using regular feedback loops and prioritization. Lean is about regular retrospectives and improving the ways of working. Systems Thinking and Cynefin teach about system dynamics and how to operate them.

Summary

Keep the principles in mind and use them as your guiding force. Build on strengths and potential. Understand the context where you operate. Each context presents different dynamics. Keep in mind the ‘fit-for-purpose’ of each tool. Any tool can be abused if it is used following the wrong principles. Consequences are governed by principles, not by the tools. Therefore, value principles.
* The evolutionary model presented comes originally from Don Beck, whose work was borrowed by Ken Wilber.

Jari Hietaniemi

Jari Hietaniemi

Jari Hietaniemi is an enthusiastic digitalization consultant. He specialises in complex and vast software projects. His philosophy is based on thinking that a consultant must know technology, architecture, project management, quality assurance, human resources, coaching and sales. His versatile experience and constant quest for improvement help to finish projects successfully and to bring new drive into client organizations.

Linkedin profile

Do you know a perfect match? Sharing is caring

SAFe vs LeSS Shootout

First of all. Ahoy, all you “Why agile” and “I don’t need scaling, I already have Scrum” people. Please start here.
I’ve followed the growing interest towards methods of scaling agile for the last decade. Personally I faced the idea of scaling Agile, whilst working as a subcontractor for Nokia. Back then two very different kind of frameworks were born. Ari and Ran explains the history of these scaling frameworks, so I won’t go there. I’ll just want to find out, which approach is better. The fastest and most entertaining way is to take our two of the most experienced scaling agile gun men Stefan and Juhana and start shooting. Whilst the gun men are warming up, let’s make a quick generic comparison of the frameworks.

Juhana, Jari and Stefan warming up

We’ve blogged earlier about SAFe (in Finnish) and LeSS in rather positive tone – now it’s time to get some friction between them. Like in all the unforgettable shootouts, the tension must be built gradually from introduction and rising action towards climax.

Guide to drama

Name

I must say, if I created a scaling framework, I wouldn’t name it either “Safe” or “Less”. The “Less HUGE” makes things even more paradoxal.

There is a clear winner on the field of naming the Scaling Agile framework. The Recipes for Agile Governance in the Enterprise aka “RAGE” wins the fight between names.

Portal

Never, ever design usability restricting cr*p on any media in the name of copyrights. It is a lose-lose deal.
Both systems provide similar web interface: A big picture on the front page, where you can click into details.
  • The hippie drawing style of LeSS will make all the wave riding, new age developers feel warm and fuzzy.
  • The engineering style, mind-blowing-in-details drawings of SAFe will make all the managers feel super important. However, the copy-paste restriction of SAFe portal sucks, while it also disables for example text search of browser.

LeSS wins the fight between the portals.

Comparison of web portals. LeSS on left and SAFe on right

Book

We live in a colorful world. Use them.
Both systems are provided as books
  • Both books are the same diameter and around 330 pages.
  • Both books come from Addison-Wesley.
  • SAFe book has more pictures and colors, otherwise they are rather similar.

SAFe wins the fight between books with flying colors.

Comparison of books. SAFe on left and LeSS on right

Introduction of The Gun Men

Question #1: Tell us a little bit about yourself and when you became a fan of SaFE / LeSS?

Stefan:
I’m a business manager and consultant at Gofore. Currently I’m focusing on our customers in health technology, healthcare, social care and wellbeing businesses. Throughout my professional career I’ve been interested in all development methods – from SW development to business & customer development. I was first exposed to SAFe whilst working at Nokia a decade ago, but the real revelation of SAFe to me happened at GE Healthcare in 2014. We had 16 teams, nearly 200 professionals scattered across globe with a goal to create multiple new products together. We applied SAFe to align the goals, sync on delivery and to optimize flows.
Juhana:
I’m a lead consultant of software development at Gofore. I have a background in software engineering and have been in the IT field for more than decade. I’ve taken part for some of the biggest public sector digitalization endeavours in Finland. For some reason, I’ve always been the guy who’s removing obstacles and trying to improve everything. So, the Scrum Master’s duty was a natural choice for me. Lately, I’ve focussed on transforming entire organisations to Agile mode. In the latest project, I was consulting a large software program that required Agile transformation. I needed a powerful and straightforward solution for the transformation. The LeSS approach caught my attention very quickly; the LeSS principles and rules were understandable and simple. The challenges confronted in the target organization were tackled intelligently with the LeSS method. This  would not have been possible if using a one-size-fits-all format such as the SaFE.
Conclusion:
SAFe interest people in management type of roles and LeSS fascinates adventurous problem solvers.

Rising Action

Question #2: Imagine a 40 person project working in waterfall model. How would you deploy SAFe/LeSS?

Educate everyone into Agile mindset

Stefan:
SAFe provides the best practices from industry also for the deployment of SAFe. The things to conside in deployment are:

  • Train everyone: a) train the leaders, b) train the internal change agents, and c) train project personnel.
  • Split project to 2-pizzas-serves-them teams. Select team level development methodology, e.g. Scrum. Make sure that teams have the right set of competences to work as feature teams.
  • Define the way of working for the organization. For a 40 person program the two lowest level of SAFe framework serve the purpose: 1) Team level and 2) Program level.
  • Then it’s about start your Agile Release Train to develop in common cadence according to commonly agreed increment plan. The increment planning and demo sessions are where the magic happens – the whole program planning, resolving and agreeing together in the same room and at the same time.

Juhana:
The LeSS creator Craig Larman’s advice for large, multi-site, and/or offshore programs is just ‘don’t do that’. Instead pick the ten best developers. If you must scale, here are the adaption principles:

  • You should educate everybody. The easiest way is to sign people to the Scrum and LeSS trainings. The formal coaching is followed up by a team, organisational and technical coaching.
  • Define the product as broadly as possible. A broader definition tends to be more customer centric and creates simpler organisations.
  • Define the ‘DoD (Definition of Done)’. A better and stronger DoD eliminates groups, roles, and positions.
  • Have appropriately-structured teams. One team member belongs to only one team, and the teams are stable, long-lived, cross-functional, and co-located. Only the Product Owner provides work for the teams. Keep managers away from the teams.

Conclusion:
Both approaches aim to educate and engage everyone.

Climax

Question #3: Our current organisation has 3 team leaders, 1 architect, 1 project manager, 1 product manager and 1 UX/AD specialist. What would these persons do in SAFe/LeSS?

Stefan:
When using Scrum on team level then there are roles of the Scrum Master, Product Owner and SW developers. On program level you’ll find Product Management and Business Owners. Note, that the role of leaders in SAFe is different from traditional management – the roles are Lean-Agile leadership type of roles.

  • The old team leaders from the legacy organization could become developers, Scrum Masters or Product Owners depending on their capabilities and desires.
  • Architect and UX/AD specialist could continue working across the teams in program level roles.
  • Project manager could be promoted to the role of Release Train Engineer.
  • Product Manager could continue with same title, but in a modified role at product management.

Juhana: 
Specialised roles are forbidden in LeSS.

  • The team leaders can become Scrum Masters. They should follow the servant-leadership-mindset such as putting people first, coaching, removing obstacles, and creating the community. One Scrum Master can serve one to three teams.
  • The Architect and UX Designer will each join a team. In addition, the architect can lead or be part of the architect community, and the UX Designer can lead or be part of the user experience community. The communities ensure the cross-cutting quality and are run by the team members.
  • the product Manager can take the Product Owner’s role and responsibilities.
  • The project Manager could focus on increasing the capability of the organisation’s value delivery with using techniques such as ‘Go and See’ and ‘See the Whole’. The Product Manager role is also temporarily needed if there is a weak Definition of Done.

Conclusion:
Tune-down way of SAFe makes it easy to sell towards legacy management.  SAFe has built-in program management process and roles. Build-up way of LeSS asks “How to simplify the organization, and be agile?”.

Question #4: Okay, now the program is working according to SAFe/LeSS framework. How does the model scale when you add two more teams?

Stefan:
SAFe’s program level guidelines make this rather easy. Assuming the teams are feature teams then it’s rather easy to add new teams – just invite them to the common Release Train Planning event and share the prioritized increment backlog across the teams. SAFe can scale up to multiple release trains and multiple value streams, but at that extreme we would be then talking about thousands of developers, and that’s a bit scary anyway.
Juhana: 
LeSS scales easily – you just add two more teams to the structure. Avoid creating component-based teams such as front-end, integration, or operation teams. If the number of teams exceeds eight, use the LeSS Huge framework. It’s based on the same principles and rules, but adds a few extra practices.
Conclusion:
After reaching Lean Agile organization scaling is easy.

Resolution

Both models require the Enterprise to change. Agile is about mind-set, not about roles or processes.

It seems that the path for agile scaling is rather similar, no matter what framework you are using. After the first hour of shooting, there is yet no clear winner.
Thanks for reading! Please, throw in any additional questions!
We will tackle those in the Shootout Sequel, where we will finally find out, which one of these two frameworks is the best.

Jari Hietaniemi

Jari Hietaniemi

Jari Hietaniemi is an enthusiastic digitalization consultant. He specialises in complex and vast software projects. His philosophy is based on thinking that a consultant must know technology, architecture, project management, quality assurance, human resources, coaching and sales. His versatile experience and constant quest for improvement help to finish projects successfully and to bring new drive into client organizations.

Linkedin profile

Do you know a perfect match? Sharing is caring