“80’s rule man, best time ever! Then that Cobain had to come around and ruin it all.” is a funny quote from the movie Wrestler. Today’s software experts might also yearn for former times. In the waterfall model, the role of architecture was clear – it was designed up front, after requirement analysis and before the implementation phase. In the current Agile era, architecture is often evolving without clear goals and objectives.
Yin and Yang
According to “Clean Architecture”, the purpose of architecture is to support the lifecycle of the system. The ultimate goal is to minimise the lifetime cost of the system and to maximise programmer productivity. Traditionally, dedicated experts have been accountable for this function. In this blog, these experts are called software architects, but technical leads and stack leads are also commonly used titles. Typically, software architects have a strong technical background and good leadership skills.
One of the Agile core values is the development teams’ autonomy. Teams design, implement, test and deploy independently. Traditional Agile methodologies, such as XP (Extreme programming) and Scrum, do not mention how architecture design should be led in practice. XP advises, for example, that architecture design just emerges alongside other design.
The larger the project (i.e. multiple teams), the more complex is the product’s software itself. This means that architectural decisions will have a bigger impact. The question then arises how architecture design is led and what responsibilities software architects versus teams have?
Make your system visible
Systems thinker and the father of the quality evolution, W. Edwards Deming has said that 96% of organisational performance is a function of the organisation’s structure. Only about 4% of an organisation’s performance is attributable to the people. Part of the structure is to understand how collaboration works in the system.
The following table makes collaboration between architects and teams visible. Every row of the table represents a potential point of leverage. The columns represent different options as to how collaboration can be implemented. Left columns represent a more distributed and emerging architecture mindset (i.e. Agile) and the right columns a more centralised and formalised approach. Organisations are different, and therefore leverage points and options vary as well. Rows are also independent, so for example “Architects are part of the teams” and “Architects are dedicated mostly to architecture topics” can be used on the same project.
<– more distributed more centralised →
Point of leverage
|Hierarchy levels||One level – no special roles, just team members||Two levels. Developers and architects||Three levels+. Team members, architects, chief architect|
|Scope of software architects||None||Architecture principles (e.g. standards, concerns, styles)||Architecture principles and infrastructure & operations||As option 3 plus enterprise architecture|
|Alignment||No architects. Team member is the only role||Architects are part of the teams.||Architects work as free agents||Architects form a team or department|
|Decision making||No architects. Teams make architectural decisions||Teams and architects make architectural decisions together||Architects make most of the architectural decisions|
|Division of Work||No architects. Teams share responsibilities dynamically||Architects and teams share their responsibilities dynamically||Architects have predefined responsibilities|
|Capacity||No reserved capacity. Teams use their time on architectural topics if needed||Architects/teams have reserved capacity for architecture topics||Architects are dedicated mostly to architecture topics|
No “one-fit for all” solution
After the current collaboration model is defined, the next step is to decide the future direction. Organisational systems are determined by their context, so “one-fit for all” solutions aren’t appropriate. Nevertheless, some conclusions can be drawn.
Typically, the long-term goal should be to move to the more distributed and self-organising collaboration model. When people understand the collaboration strategy, it is much easier to take actions toward it. Unfortunately, organisations underestimate the amount of work that the transformation requires. People have different skills and expectation levels and unlearning at both the organisational and individual level is required.
Agile is also a highly disciplined method. Business people and team members must work together daily; the cross-functional team structure is mandatory and software delivery must be frequent, to name just a few principles. If the organisation won’t bend to these demands, a more centralised collaboration model might be the valid option.
Sub-optimising is also a risk in bigger organisations. A team might violate architecture principles thoughtlessly, just to finish their sprint backlog faster. The other team might implement a similar service that has been already created by another. Architects see more easily the whole and are able to make holistic decisions.
Many organisations’ technical debt is huge as well. Products suffer wrong technologies or bad programming practices and an enormous amount of refactoring work is needed. Painful architectural decisions are often made more efficiently by a more centralised collaboration model.
A hero is required
Software architecture is not the hottest topic in the software development field. In addition to this, it is very abstract and hard to lead. However, all software has an architecture regardless of if the architecture is managed or not. Smart organisations understand that the organisation structure should be optimised for the most crucial paths of collaboration – and collaboration between teams and architects is part of it.
The Clean architecture – Robert C. Martin
Technical leadership and the balance with agility – Simon Brown
If you only look at megatrends, you can miss opportunities that are hidden beneath the surface. Weak signals are often the first indicators of potential change.
A weak signal can be defined as an emerging issue, not easily identified but one that has potential future impact.
Prior to 2007, there were weak signals around rich graphical user interfaces in consumer devices alongside other signals pointing to the rise in touch interfaces. Apple famously recognised these signals ahead of other more established mobile phone companies. The iPhone launched in 2007 and the rest, as they say, is history. In the example below there are a number of weak signals that The Finnish Innovation Fund, Sitra gathered during 2018 as the basis for future discussion. Some of these are macro issues that are likely to impact society in the long term, however, others such as a potential ban on all plastics could impact many of us both personally and professionally in the medium to long term.
Figure 1: Weak signals, Sitra
In a business context, it is crucial to identify the weak signals related to your organization since they point to the direction in which your industry is moving. Identifying weak signals that are relevant to your organization or business often produces surprises. These weak signals require sensitivity and attention to be able to identify changes required to stay ahead of the competition. Identifying weak signals is therefore critical in developing a competitive business that is prepared for future challenges as we have seen in the iPhone example above.
Characteristics of weak signals
Novelty: a weak signal is an indicator of something new or a new perspective on a known subject.
Surprising: a weak signal is surprising to its interpreter.
Challenging: a weak signal forces one to challenge existing assumptions and is therefore often difficult to detect or easy to overlook.
Significant: a weak signal suggests something that may have an impact on the future.
Delay: a weak signal describes something that is not yet significant but requires time to mature.
Source: Sitra: Mikko Dufva
Identifying weak signals
An open mind is key to identifying weak signals as our personal bias and views will tend to guide our thoughts. Placing yourself in unfamiliar surroundings is a good technique as well as researching futurologists and industry-specific publications and papers. Think as wide as you can as often the most impactful signals are found at the periphery or in areas that lightly touch your organization or industry.
Another classic example of identifying weak signals that produced a market leading product is the design of the original Mazda MX5 sports car (although some of this story may be an ‘urban legend’) In the late 1970’s Mazda launched a design competition between their internal design teams in the US and Japan with the objective of recreating the magic of classic British sports cars. So, the story goes that the US team decamped to a Caribbean island with the task of designing the new car. Their brief was to ‘take as long as it takes’ to design the new car. The combination of unfamiliar surroundings and a relaxed unpressured atmosphere together with talented designers resulted in the MX5 which went on to be highly successful for Mazda winning many top awards.
Once you have mapped the weak signals that have the potential to impact your organization or business, it is important to spend time analysing and interpreting them. Data is nothing without interpretation. Think about how they will change your customers or your employees and how they might combine with other signals or trends to accelerate change.
Take another look at the diagram above and think about your business context. Choose one weak signal and start to ponder and speculate what kind of impact that signal would have on your business or industry. This is often better done in a small group – open your mind and you may well be surprised at the outcome!
At Gofore our consultants work with our clients to help them identify the weak signals that have the potential to impact their future business or organization. Although we have yet to decamp to a Caribbean island, previous projects have resulted in organizational or cultural change programs and new service and business design initiatives. One common denominator in all these projects is that the outcome is always very different from what was expected.
More reading on the theme of weak signals:
I thought it would be a good way to start my first few weeks as a robot expert at Gofore by sharing some very basic info about the robots I am mostly working with here.
RPA (Robotic Process Automation, “ohjelmistorobotiikka” in Finnish) is quite a new phenomenon in business process automation. Basically, RPA is a software robot which uses the computer by itself so it’s not a physical robot. The software robot will follow defined business process logic and usually emulates human interactions to execute the process. RPA often uses the graphical user interfaces of the information systems as humans do especially if there are no APIs available. This means that the RPA sits on top of the existing infrastructure and thus is not an intrusive technology.
RPA robots can be categorized into two groups: attended and unattended robots. Attended robots work in human workstations together with human employees. Attended robots can be started by humans or get triggered by events. They can run in the background or ask for human input if needed during the execution of the process. Attended robots can be useful, for example, in the service desk or other customer-facing operations.
Unattended robots work on their own at either physical or virtual cloud workstations without any human intervention. They usually execute high transaction volume processes by automated schedules. Unattended robots can, for example, process accounting tasks during the night so they are finished by the morning when employees come to work. Even though unattended robots work on their own in the background, they can be monitored, audited and managed by human “robot supervisors”. Skynet is not yet here…
What can RPA do?
Basically, almost anything that humans can do at a workstation. However, robots are still bit dumb so they can’t do something that requires complex human interpretation or thinking. On the other hand, by combining AI with the RPA, robots can do more complex human thinking requiring processes but practical implementations of these are still a bit rare. If the process has inputs in a digital, preferably structured, form and the process can be depicted with e.g. flowchart with unambiguous rules, it is most likely quite easy to automate with RPA.
Typically a companies’ RPA journey starts by automating support function processes:
- Finance (e.g. reconciliation, purchase to pay, reporting)
- HR (e.g. onboarding, payroll)
- IT (e.g. monitoring, user management)
- Sales and operations (e.g updating inventory, creating invoices)
With RPA, it is possible to use your imagination and do all kinds of fun stuff like making a robot play piano while filling in online forms: https://twitter.com/UiPath/status/1025155664421838849
RPA, chatbots and AI
You can automate a lot of processes with only RPA. However, by combining RPA with chatbots and AI you can do a lot more cool things. For example, RPA could gather data from legacy systems for AI to produce a prediction about maintenance needs for industrial machines. After that, RPA could set those machines to get a maintenance visit at an appropriate time. Finally, a chatbot could inform humans about the upcoming maintenance. RPA, chatbots and AI are interconnected like illustrated in the picture below:
Another example of combining RPA and chatbot is a bot which will buy train tickets for you: https://www.linkedin.com/feed/update/urn:li:activity:6430749212501168128
RPA vs traditional software development
Developing human mimicking RPA robots might not seem so sexy for traditional software developers but both methods have their pros and cons in process automation. During the last couple years, there has been a bit of hype about RPA and automating everything with it but it should be seen as just one tool for process automation and compared to other available methods case by case.
- Relatively easy and quick to implement and realize the benefits. With RPA, it’s possible to automate some business processes in just a couple of days or weeks compared to usually a much longer traditional software development time. For example, I have developed an RPA robot in four days which automated a quite tedious financial reconciliation process.
- Quite low-cost thanks to the quick implementation times and potential quick returns on investment
- Basics of developing RPA robots is quite easy to learn also for non-software developers
- Doesn’t disrupt existing infrastructure and systems
- It’s possible to do UI automation if APIs are not available and access basically every needed system and legacy systems
- Changes in existing infrastructure and systems can disrupt RPA robots. If the robot uses the UI of the systems and it changes considerably, the robot may need a reconfiguration
- Robots need to be managed in case of changes in the processes or IT systems. When scaling up to dozens or hundreds of robots all doing different tasks, managing them needs resources
- Traditionally developed e.g. integration between two systems is technically more effective than RPA
- Risk of taking quick wins with RPA and forgetting longer term more traditional system developments
Some RPA technologies:
- UiPath (current market leader)
- Blue Prism (main competitor of UiPath)
- Workfusion (offers a free version for enterprises)
- Robotframework RPA (Finnish open source solution)
At Gofore we are technology agnostic so we can evaluate your specific automation project and recommend the best technology available for your requirements.
Wow robots seem cool, I want to learn more?!
- Leave a comment for me – I am happy to help, for example, I could arrange RPA training.
- Start studying free courses about RPA developer and general business skills online: https://www.uipath.com/rpa/academy
- Download free community edition of UiPath and try e.g. automate some boring manual task you need to do: https://www.uipath.com/developers/community-edition
- If you are more of a software developer person you could also try Robotframework RPA
Get ready for impact
The use of cloud services and web technologies means that it is no longer important where your physical base is located. Technology is changing every aspect of our lives and the ability for people working within both government and private organisations to work remotely is changing the way we do business (for the better). This sociological impact means that designers, developers and business professionals who work remotely can have a direct impact on both the environment around them and the wider world.
This decentralisation of knowledge and work has disrupted many industries and price is often the first thing to change. A coder based in a low-cost region such as Asia can undercut his peers in California or Western Europe. Real time working tools such as Jira boards, Trello and Slack mean that teams can work effectively and efficiently wherever team members are located.
Add culture to this globalised decentralisation and the mix becomes significantly richer. I have first-hand experience working with clients in some of the most multi-cultural cities in the world such as Amsterdam, London and Dubai. Multi-cultural teams look at challenges through different lenses. It is natural to have inbuilt bias even if you don’t realise yourself. Where you were brought up, where you went to college and your circle of friends all influence your outlook on life and how you approach a challenge. Bringing together diverse cultures encourages discussion, it helps to produce a richer all-inclusive solution. Digital products and services are used by humans and so a rich all-inclusive product or service is more likely to be successful.
Culture promotes creativity
It is often said that travel broadens the mind however people who travel but fail to engage with the local culture have less of a creative boost than those who immerse themselves in it.
Lee and Abs from Dubai Future Accelerators
There are many examples of companies who have failed in their attempt at doing business in a new territory because they haven’t recognised the cultural differences. If you want to succeed in developing a product or service internationally you need to consider many things including, language, religious beliefs and different ways of working in order to build a robust and mutually beneficial business strategy.
Understanding your audience
Body language, casual and business etiquette, transparency and respect are some of the first things that I personally consider when approaching new international markets. It is important to recognise the cultural differences and ways of working, but more importantly to respect these differences even if you do not yet understand them. Change impacts all of us, it’s just that some adapt to it easier than others. Whether one shares the same beliefs or not, if you are to achieve success in international business a mutual understanding is imperative. Relationship building blocks are a foundation for growth. It is my belief that strong international collaboration between diverse cultures can and will produce better products and services and allow people to achieve more.
Take every opportunity to learn, understand and appreciate as many different cultures as you can in your lifetime. You never know where the creativity can lead you. Disruptive innovation comes from all corners of the earth. Often a problem in the most challenging environment can create the most life-changing solution.
Dubai Future Accelerators (DFA)
In September 2018 Gofore were selected to participate on the DFA program. This program is designed to bring together companies from across the Globe to co-create products and services with the aim of helping the Dubai Government entities face challenges of making Dubai the City of the Future. Working with the DFA and various government authorities has highlighted the importance of appreciating and embracing cultural differences. I have been working with people from all walks of life from university students to senior government officials all of whom share a passion for improving peoples lives through digital. This phase of the DFA program draws to a close in late November – check out my next post to learn more about some of the exciting solutions that will be developed as a result of the program.
You can read more about the Dubai Future Accelerator program here: The Dubai Future Accelerator Program
The annual X-Road Community Event took place in Tallinn on 12th September 2018. During previous years the event has been organized by Estonia’s RIA but this year the responsibility was taken over by the Nordic Institute for Interoperability Solutions NIIS. NIIS is an association founded jointly by Finland and Estonia in June 2017. Its mission is to develop e-governance solutions, kicking off with the X-Road technology.
Gofore acts as the NIIS’s ‘right hand’ as the development team working on the X-Road core. Currently, we are four developers working closely with NIIS’s CTO Petteri Kivimäki. The whole team was kindly invited to participate in the event and some of us were also given the opportunity to speak in the conference special tracks. So we travelled to Tallinn the previous evening and had some time to have a late dinner in the Old Town which was a very nice way to start the trip.
After an early night at the hotel, the next day the conference started with NIIS’s CEO Ville Sirviö opening the event. There were more than 150 participants from over 10 countries including Japan, the United States and Madagascar.
The event got an electrifying start when the host invited on stage representatives from the member countries Estonia and Finland and surprisingly Iceland too! Iceland joined the interoperability alliance as a partner and signed the partnership agreement on stage. This greatly strengthens the X-Road partnership and co-operative effort.
Next, the enthusiastic host invited Petteri Kivimäki on stage to present X-Road development model big picture and roadmap. NIIS maintains and develops X-Road core and provides support to its members. The advisory group evaluates business enhancement requests and approves new items to the product roadmap. The working group prioritizes product backlog, looks into pull requests and approves new enhancement requests to the backlog. X-Road Product Roadmap and Product Backlog are owned and managed by the NIIS. The current roadmap has Ubuntu 18 and REST support as the highest ranked items. It was also announced that the design of the next generation X-Road 7 will start during 2019.
There was a panel discussion about cross-border services between Estonia’s Siim Sikkut (Government CIO, Ministry of Economic Affairs and Communications) and Finland’s Anna-Maija Karjalainen (Director General, Public Sector ICT, Ministry of Finance).
Jarkko Moilanen (Senior Advisor, Ministry of Education and Culture, Finland) talked about API economy and best practices in API design.
Kristo Vaher (Ministry of Economic Affairs and Communications, Estonia) talked about Fact Driven Architecture. His vision for government services architecture includes a blockchain and the actual services are built on top of that.
Andres Kütt (Interforum OÜ) talked about X-Road design patterns. He introduced a set of recommendations and principles for integrating with X-Road.
There was another panel discussion about X-Road’s international use cases between Estonia’s Kaija Valdmaa (Project Manager of Estfeed, Elering AS), Iceland’s Einar Birkir Eirnarsson (Head of Division, Ministry of Finance and Economic Affairs) and Japan’s Masakazu Ohtsuka (Planetway). Valdmaa’s electricity company Elering is utilizing X-Road in reading meter data across national borders. Ohtsuka’s Planetway is evaluating X-Road technology in Japan’s private sector and hopes to expand its usage to Japan’s public sector in the future. Eirnarsson said they have evaluated multiple data exchange platforms including building their own solution but eventually they ended up with X-Road and the implementation project is currently ongoing.
After lunch, there was a Kahoot quiz about X-Road. People were distributed among specialized tracks based on their interests. The tracks were X-Road users, X-Road operators, X-Road developers and e-governance. Our team’s presentations were in users and developers tracks. Jarkko Hyöty (Gofore) presented the X-Road’s load balancing solution in the X-Road users track.
I participated in the X-Road developer track and gave a presentation about X-Road’s development practices.
The specialized tracks included a workshop encouraging the community to participate and create new ideas. The workshop I participated in was about future visions of X-Road. We were divided into groups where we figured out improvement suggestions and new features for X-Road. Finally, the ideas refined in groups were presented to everybody and collected together as suggestions for future work.
The conference day ended sky high on the 30th floor of the Swissotel overlooking Tallinn. Sparkling wine, people getting to know each other and nice discussions took place.
It seems that there is high demand for an integration platform like X-Road all over the world. It may be because different e-government initiatives require a common integration model to ensure cost effectiveness and ease-of-use of governmental functions. It also helps that X-Road is an open source solution and as such suits the needs of many nations.
Gofore has many years experience developing the X-Road core and related national e-government services. We are more than willing to help in any X-Road evaluation or implementation project.
Overall it was a very nice and well-organized event by NIIS. The X-Road community is already strong and getting stronger each year. Last year there were 70 participants and this year the amount doubled. It is now also a truly international event with participants and speakers coming from all around the world. I can’t wait to see how far we have come next year.
An interesting question was posed during a presentation at a recent Value-Based Based Leadership workshop. We were having a lightning talk about Gofore values together with Daniella Pitaro, this is one of the talks we have with new employees on their first day. In this talk, we use a diagram titled “the circle of success”, which displays five dimensions we want to discover and strengthen in every Goforean.
“How do failures and weaknesses fit in this picture?”
As is the case with many rapidly growing organizations, discussing failures is not a recurring topic at Gofore. I believe that we have a healthy relationship with failures, but the way they relate to success requires an elongated answer. Failures are, after all, inevitable in all lines of work. It’s the way you handle them that defines, whether they break or empower you.
Be transparent about failures
Some of the most expensive mistakes are the ones you sweep under the carpet. In the 1970’s, Ford became aware of a fuel tank problem in their Pinto model, which increased the chance of a crashed vehicle catching fire. Instead of fixing the flaw, Ford concluded that the repairs would be more expensive than settlement lawsuits, and went with the faulty design. The result was a “voluntary” recall of 1.5 million vehicles, over one hundred lawsuits and a damaged reputation. Anecdotal as this example may be, being transparent about failures often yields faster and less expensive ways to remedy them, while refusing to act only prolongs the inevitable.
Be self-determined about your weaknesses
While self-reflection enables you to identify your weaknesses, self-determination is required to power any change. This means being proactive about your training needs, and independently keeping yourself up-to-date. Make a habit of managing a long-term learning plan for yourself, and you will have less risk of losing important opportunities due to competency gaps. As an organization, Gofore facilitates this by providing various coaching services to help you draft your own path. All our employees are expected to allocate 6% of their working time for self-development. On average, that’s over two hours every week our experts invest in themselves!
Nourish your sense of community
Transparent management of failures can be an important factor in building a better sense of community. Even in an organization of exceptional experts, it’s important to understand that nobody’s perfect. The possibility to fail without blame provides a learning opportunity and a crucial reminder for everyone that we are all human. An organization that effectively shares knowledge of mistakes, learns fast. A fast-learning organization still makes mistakes, but always closes the door on recurrence. Letting your community fear failures will only breed other negative phenomena. At worst, employees could contract the so-called impostor syndrome, where they attribute their success to luck, not merit.
Be enthusiastic for your own sake, and disciplined for everyone else’s
Everybody loves stories of exceptional individuals beating the odds. “The doctors told her she would never walk again”. “From selling dried fish to playing in the world cup”. After discounting factors like luck, medical quirks and good timing, the common denominator in this kind of story is enthusiasm. Enthusiasm can help you against crushing odds, but more commonly it powers the resolve you need to deal with everyday problems. While enthusiasm and resolve form a beneficial cycle together, weaknesses coupled with a lack of enthusiasm are a demoralizing combination.
To fulfil our promises to our customers, we need discipline and resolve. Many failures are a result of oversight or negligence, so the importance of doing things The Right Way™ all the time cannot be exaggerated. Saving the time span of four mouse clicks is reckless if you risk having to repair the damage for an hour. Only discipline is guaranteed to ultimately save both your time and the customer’s money. It yields greater customer satisfaction and better end results.
If the time allocated to our original lightning talk would have allowed it, we could have talked about failures in great depth. Now I’m left hoping my short answer was convincing enough: as a Goforean, you’re allowed to fail at things. You are, however, required to keep learning from your mistakes. A weakness is a true weakness only if you do nothing about it.
Why Do Threat Analysis
Well, why not? It can be that due to a lack of familiarity with or experience of doing threat analysis (aka threat modelling), there can be a certain reticence or fear of failing. But such reticence or fear of failing is unfounded. Firstly, you won’t be alone doing threat analysis as you’ll have the application domain experts with you; secondly, a rudimentary threat analysis is more useful than none at all; and thirdly, analysis competence will grow by following up on the rudimentary threat analysis in successive sessions.
The objective is secure-by-design (aka built-in security) and one of the ways to achieve that is to identify and mitigate design-related security vulnerabilities during the system or application design phase. The output from threat analysis will be a set of security requirements that must simply be fulfilled like all of the other requirements.
Note that security is one of the enablers of overall product and service quality, which customers and end users rate trust and reputation upon. Additionally, since the General Data Protection Regulation (GDPR) activated on the 25th of May, and since secure-by-design is an enabler of privacy-by-design, it is prudent to also query and scrutinise privacy risks and mitigations during threat analysis.
When To Do Threat Analysis
There is a timing element to threat analysis. Ideally, threat analysis is performed as soon as the architecture has been established and is sufficiently mature. It is critical to understand that, no matter how late in the development process threat analysis is performed, it is critical to understand weaknesses in the design’s defences.
The fiscal and temporal cost of addressing security issues will generally increase if the design phase misses security weaknesses, so it is much more useful (and more cost-effective) to begin the process of identifying potential attacks and associated mitigations as early as possible.
A threat model should begin when the major structures and major components or functions of the architecture are mature. Note that threat analysis is a useful exercise regardless of how close the system is to deployment or how long the system has been in use; but as said, the earlier, the better. Successive development cycles, as in the norm in Agile development, should include focused threat analyses when design changes are made and to gradually mitigate security risks that the system currently carries. Also note that there is a distinction between end of development and end of support, and even when active support has ceased, a proper threat model will bring clarity about the possible flaws in the system.
Who Does Threat Analysis
Optimally, the product owner, lead architect, lead developer, lead tester, and lead user experience designer should together perform threat analysis with advice and facilitation from a security advisor or consultant. For GDPR aspects there may also be involvement of the Data Protection Officer, who has formal responsibility for data protection compliance within an organisation. As well as having the various insights from each of these competence areas, it is a personal opinion that this cross-competence representation ensures a greater probability of broader security awareness exposure and sharing throughout the development team(s).
The Link to Requirements Engineering
Threat analysis should be planned and applied in conjunction with requirements engineering; in this way, the security requirements can be derived as an integral part of the overall requirements derivation process. The security requirements derived from threat analysis should be linked to related business case epics, use cases and user stories so that they simply become “requirements amongst others”, which helps to dispel some of the unfounded reticence and mystery around security requirements.
How to Do Threat Analysis
How do you want to do it? There’s no single perfect way to do threat analysis; open source and proprietary methods exist. But they all similarly strive to answer the following questions:
- What assets require protecting?
- Who and what are the threats and vulnerabilities?
- What are the controls to mitigate the threats and vulnerabilities?
- What are the implications of not securing assets?
- What is the value to the organisation?
A good threat analysis methodology is one that identifies potential security vulnerabilities and threats, and as an exercise is time-optimised, manageable and repeatable. Project artefacts used for threat analysis include (and are not limited to):
- Architectural diagrams (component and logical models)
- Data-flow diagrams (DFDs)
- User stories and use cases (and related misuse and abuse cases)
- Mind maps
- UI designs
Think like the attacker to understand the relevant threats to both security and to privacy. Use defined security questions to ensure sufficient threat landscape coverage. Use the following categories to understand who might attack the application:
- An ordinary user stumbles across a functional mistake in your application while using a web browser and gains access to privileged information or functionality.
- Programs or scripts that search for known vulnerabilities and then report back to a central collection site.
The Curious Attacker
- A security researcher or ordinary user who notices something wrong with the application and decides to pursue further.
- Rebellious types seeking to compromise or deface applications for notoriety or a political agenda.
The Motivated Attacker
- A disgruntled employee or contractor with inside knowledge or a paid professional attacker.
- Criminals seeking high stake pay-outs by cracking high-value organisations for financial gain.
Apply STRIDE To Classify Identified Threats
STRIDE is a classification scheme for characterising known threats according to the kinds of exploit that are used (or motivation of the attacker). The STRIDE acronym is formed from the first letter of each of the following threat categories:
- Identity spoofing is a risk for applications that have many users but provide a single execution context at the application and database level. Users should not be able to become another user or assume the attributes of another user.
Tampering with Data
- This is about how an application deals with untrusted data – treat all data as untrusted anyway! Examples of tampering are changing GET or POST variable values directly in the URL address bar, failing to check data integrity, and deficient client-side and server-side data validation. In the worst case, you can call it data sabotage. This relates to threats #1 and #7 in the Open Web Application Security Project (OWASP) Top 10 2017.
- Users may dispute transactions if there is insufficient logging and auditing of user and system events, or if you cannot track user activities through the application or from the audit trail. A properly designed application will incorporate non-repudiation controls such as tamper-resistant logs; it should not be possible to alter logged events. A note of caution here: in terms of the GDPR, only the required data to facilitate audits should be logged; do not log personal data if there is no explicit requirement to do so. This relates to threat #10 Insufficient Logging and Monitoring in the OWASP Top 10 2017.
- There are basically two types: a data leak, or a privacy breach. A data leak can be a flaw or be a malicious attack. The most common flaw is simply not using Transport Layer Security (TLS) and not enforcing HTTP Strict Transport Security (HSTS). This flaw provides entry points for attackers to steal keys, execute man-in-the-middle attacks, or steal clear text data from the user’s client (e.g. browser) and data in transit. Concerning privacy breaches, in the context of the GDPR due caution is required concerning the protection of personal data from accidental or unauthorised disclosure.
Denial of Service
- The Denial of Service (DoS) attack is focused on making a resource (site, application, server) unavailable for the purpose it was designed. There are many ways to make a service unavailable for legitimate users by manipulating network packets, programming, logical, or resources handling vulnerabilities, among others.
Elevation of Privilege
- Access control enforces policy such that users cannot act outside of their intended permissions. Flaws typically lead to unauthorised information disclosure, modification or destruction of all data, or performing actions outside of the bounds authorised by the administrator for the user. It is essential that the user cannot manipulate the human-computer interface (HCI) to elevate his/her role to higher privilege roles. This relates to threat #5i Broken Access Control in the OWASP Top 10 2017.
For further types of flaw-exploited attacks, you can refer to the OWASP Attack Categories.
Apply DREAD to Quantify Threats
DREAD is a classification scheme for quantifying, comparing and prioritizing the amount of risk presented by each evaluated threat. The DREAD acronym is formed from the first letter of each category below.
DREAD modelling influences the thinking behind setting the risk rating and is also used directly to sort the risks. The DREAD algorithm, shown below, is used to compute a risk value, which is an average of all five categories.
DREAD_Risk = (DAMAGE + REPRODUCIBILITY + EXPLOITABILITY + AFFECTED USERS + DISCOVERABILITY) / 5
The calculation always produces a number between 0 and 10; the higher the number, the more serious the risk. Here is one way to quantify the DREAD categories:
- If a threat exploit occurs, how much damage will be caused?
- 0 = Nothing
- 5 = Individual user data is compromised or affected
- 10 = Complete system or data destruction
- How easy is it to reproduce the threat exploit?
- 0 = Very hard or impossible, even for administrators of the application
- 5 = One or two steps required, may need to be an authorised user
- 10 = Just a web browser and the address bar is sufficient, without authentication
- What is needed to exploit this threat?
- 0 = Advanced programming and networking knowledge, with custom or advanced attack tools
- 5 = Malware exists on the Internet, or an exploit is easily performed, using available attack tools
- 10 = Just a web browser
- How many users will be affected?
- 0 = None
- 5 = Some users, but not all
- 10 = All users
- How easy is it to discover this threat?
- 0 = Very hard to impossible; requires source code or administrative access.
- 5 = Can figure it out by guessing or by monitoring network traces.
- 9 = Details of faults like this are already in the public domain and can be easily discovered using a search engine.
- 10 = The information is visible in the web browser address bar or in a form.
Implement Threat Mitigations
Fortunately, there already exists a body of industry-scrutinised application security requirements known as the Open Web Application Security Project Application Security Verification Standard (OWASP ASVS) that can be readily applied as security requirements for mitigating identified security weaknesses and threats.
Verify Threat Mitigations
An OWASP Testing Guide will help developers and testers to verify implemented security mitigations to the accepted risk level. By “accepted risk level” is meant that there may be residual risks that will be too expensive to mitigate beyond the value of the business case; project management or owners must be prepared to undersign residual risk acceptance.
Penetration testing should support verification and automated security testing should also be implemented to the Continuous Development/Continuous Integration pipeline using e.g. Checkmarx and Open Source Analysis security verification tools.
Also, verify with the Data Protection Officer that any identified privacy-associated risks have been sufficiently mitigated to the accepted risk level.
I think this is the most important advice; without the curiosity, threat analysis might not happen. As said at the beginning, don’t postpone threat analysis; be curious, jump right in, and learn it by doing it.
OWASP Application Security Verification Standard (ASVS) is an industry-respected open-source framework of security requirements that MUST be incorporated when designing, developing, testing and deploying modern web applications for digitalised environments. It provides the security verification requirements to address your defined security questions.
OWASP ASVS Level 2 ensures that business-level security controls are in place, are effective, and are used by business-critical applications to defend against vulnerabilities and against attacks. Security threats to Level 2 applications will typically be by skilled and motivated attackers who focus on specific targets using tools and techniques that are highly practised and effective at discovering and exploiting weaknesses within applications.
The objective is that applications are secure-by-design and secure-by-default.
Secure Design is King
OWASP ASVS requirements should be applied as a customisable security blueprint for identifying the relevant security requirements according to identified potential threats to business entry points, gateways, critical assets and credentials. The OWASP Top 10 provides a starting-point high-level summary of the very minimum security coverage required.
Bake The Security In
For each application, required OWASP ASVS requirements should be pinpointed during feature design and epics and stories planning in order to ensure that the required security controls are part of development from the outset.
How to start
The development team scrutinise the intended application or service design using the defined security questions as a guideline. This will identify the entry points, boundaries, components and interconnections and that are security-relevant.
The application team can then utilise the OWASP Application Security Verification Standard (ASVS) requirements to produce security epics and stories that can be managed as Jira tickets,
Do not expect perfection from the beginning – getting security right is difficult and it is a learning-by-doing experience – but doing secure design and development in a structured and traceable way using industry-respected methods and materials is already a good start.
I’ve recently attended Agile events where world-class gurus state that Scrum is broken, the Product Owner is a waste, or LESs is better than SAFe. Tools are built for a purpose. You can use a tool for a certain purpose only in a certain context. When the context evolves, the tools also need to evolve.
What does not change, are the underlying principles. Misuse of a tool is easy if you don’t understand the principles. There are tons of studies regarding ScrumBut, where the Scrum tool is abused. Where the (Agile) principles are not followed, even if the (Scrum) tool is in use.
Maturity of the Organization Defines the Context
According to Reinventing Organizations by Laloux, in today’s predominant organizational model the leader in the hierarchy turns a cog that directs the workers below. Planning and execution are strictly separated. This leads to silos, rigidity, no innovation, no responsibility, no ability to advance meaningfully in the career. Laloux divides cognitive, psychological and moral maturity into five levels*. Higher levels build on top of the lower ones.
|Tribal – “Red”||A simple hierarchy to steer work and a wolf-pack mentality||Mafia and Startups|
|Agrarian – “Traditional”||Large organizations with numerous roles and steering mechanisms, like long and medium-term planning. Planning and execution are strictly separated. Control what and how. Thinking happens at the top, doing at the bottom. The underlying worldview is that workers are mostly lazy, dishonest, and in need of direction.||Church, military and government|
|Industrial – “Orange”||Profit and growth are the most important measurements. Achieved by the means of innovation, accountability and meritocracy. RnD, Marketing and Product Management departments are invented. Control what. The underlying worldview is based on material success.||Multinational companies|
|Information – “Green”||Focus on culture and empowerment to achieve high employee motivation. Highly sensitive to people’s feelings. All the perspectives deserve equal respect. Where Industrial level seeks to make decisions top-down, based on objective facts, expert input and simulations, Information level strives for bottom-up processes. Information level brings opposing points of view to eventual consensus. Where Industrial level views organizations as machines, the dominant metaphor of organizations in Information level is the family. It seeks fairness, equality, harmony, community, cooperation, and consensus.||Culture-driven organizations|
|Adaptive – “Teal”||“Post-postmodern consciousness”. No bosses, no competition. In this mindset, ego is viewed from a distance. We can see ego’s fears, ambitions, and desires: In the Tribal level, a good decision is the one that gets me what I want. In the Traditional level, decisions are determined by social norms of what is right and wrong. In the Industrial level, effectiveness and success are the yardsticks. In Information, matters are judged by the criteria of belonging and harmony. We use measures of inner rightness: Does this decision seem right? Am I being of service to the world?||Network organizations|
Summary of Laloux’s levels of organizational development. The names and colour codes come from Lalouxs’ book, however, they were different in the original work of Beck and Wilber.
Both consciousness and adaptability grows with the maturity of the organization
According toLaloux, in the highest level of maturity (“Adaptive”) we don’t pursue recognition, success or wealth. We pursue a life well lived. Still, the consequence might be recognition, success, wealth, or love. Adaptive organizations reveal three major breakthroughs: evolutionary purpose, wholeness, and self-management. These major breakthroughs make organizations prosper given the right context. Companies like Buurtzorg, Patagonia or Zappos show strengths of the Adaptive level:
- Employees thrive
- Salaries above market rates
- Grow year in, year out
- Achieve remarkable profit margins
Adaptive level brings three organizational breakthroughs:
- Principles driven decision making. Rather than trying to predict and control, Adaptive organizations try to sense and respond. Similarly, as the Cynefin framework describes working process in the complex environment, that consist of “unknown” factors. The output cannot be planned predicted, but one must proceed with principles driven decision making in an environment where it is safe to fail. Like Stephen Covey says “There are three constants in life: Change, choice and principles”.
- Wholeness creates an environment of safe conversations. Safe conversations invite deep listening, authenticity and vulnerability. It is assumed that everyone seeks for the common benefit, even if their opinions seem different. Conflicts are dealt with understanding. This will create smarter networks, unleash creativity and bring higher employee satisfaction. Like Douglas Stone says “The urge to blame is based.. on the fear of being blamed.”
- Minimum Viable Management does not mean a free-for-all. It means form-follows-need: Roles are picked up, discarded, and exchanged fluidly. Power is distributed. Decisions are made at the point of origin. Innovations can spring up from all quarters. Meetings are held when they are needed. Temporary task forces are created spontaneously and quickly disbanded again. The leaders of self-managing organizations feel, that it is less risky to deploy self-managed teams. That self-managed teams make better decisions. Like Bruce Lee stated “It’s not the daily increase but daily decrease. Hack away at the unessential.”
These three principles enable the organization to better handle uncertainty, complexity and magnitude of nowadays Agile business. Adaptive organisation work efficiently and fast, while fewer power games and egoism is in the way of creativity. More effort is used on improving individuals and organizations.
Where to Start
Team of Teams by McChrystal underlines the idea of creating nimble, transparent, and horizontal organizations. Organizations which empower their people to execute based on the concept of ‘shared consciousness’. People are empowered to trust their gut and use their best judgement based on their training and knowledge of organizational goals. The book gives good ideas of
- Break down organizational silos by sending liaisons to other teams
- Empower everyone to make decisions, don’t act as a decision bottleneck
- Hold frequent sync up meetings to disseminate information
- Set up cross-disciplinary open-plan workspaces
- Enable people to make decisions at lower in the hierarchy and get to know their counterparts in other teams
- Be a gardener. Plant and provide water and nutrients. No heroic micromanagement
- Monitor data and real-time information, but stay away from control-freak and micromanagement.
From hierarchy to networks (Team of Teams way of drawing the idea)
Need for Speed
- Formal: the official organizational chart and the job descriptions
- Extant: how things actually work and who really makes decisions
- Requisite: a setup that would be most natural and best suited to the work and purpose of the organization.
The Idea of The Holacracy is to achieve the requisite organizational structure. Holacracy’s systems and processes are about helping the organization find its own unique identity and structure. In Holacracy, a pyramidal hierarchy is replaced with “circles” dedicated to specific functions. Each of the circles has a “lead” who, like a traditional manager, is responsible for assigning work and seeing that it’s completed. They, however, are not responsible for determining how the work is done. A kind of company-level crowd-sourcing.
From hierarchy to networks (Holacracy way of drawing the idea)
Fit For Purpose
There are no good or bad ways to organize. The question remains whether the structure is a good fit for the task at hand. Mature levels are better suited for the purpose of more sophisticated work, with high complexity and high skill set. And lower levels are better suited for crisis situations. The organizations are far from consistent and are a mix of both. The maturity of an organization is influenced by the context and by the leader of the organization.
Back to Scrum
Scrum is not an absolute value. Just a tool. Originally Scrum was a counterattack to Waterfall. From fixed scope to work decomposition, fast feedback, retrospectives and being close to the customer. The idea of Agile is to create high value to the customer by means of collaboration, demos, improvement and being open to change.
In this context Scrum is a process to take an organization to the next level of maturity. Scrum gives the management a feeling of top-down control by managing the product backlog. Simultaneously Scrum provides the team with a freedom and safety to make bottom-up decisions. At some point, there is no more need for the safety provided by the hierarchical Scrum mechanism. Similarly, you may evolve from component teams to feature teams.
What makes the tools like Scrum, Kanban or SAFe work are the underlying principles like
- Agile (https://gofore.com/agile-transformation-action-part-1/ )
- Lean (https://en.wikipedia.org/wiki/Toyota_Production_System )
- Systems Thinking (https://gofore.com/en/what-is-systems-thinking-and-how-should-i-use-it/)
- Cynefin (https://en.wikipedia.org/wiki/Cynefin_framework)
Agile is about adapting to an unclear situation by using regular feedback loops and prioritization. Lean is about regular retrospectives and improving the ways of working. Systems Thinking and Cynefin teach about system dynamics and how to operate them.
Keep the principles in mind and use them as your guiding force. Build on strengths and potential. Understand the context where you operate. Each context presents different dynamics. Keep in mind the ‘fit-for-purpose’ of each tool. Any tool can be abused if it is used following the wrong principles. Consequences are governed by principles, not by the tools. Therefore, value principles.
* The evolutionary model presented comes originally from Don Beck, whose work was borrowed by Ken Wilber.
Have you ever had the feeling that the same problems arise constantly, even after numerous discussions? In addition to this, for some reason, it’s hard to make any changes. The systems thinking approach might just be the answer you have been looking for.
Systems thinking characteristics
Systems thinking is a method to analyse the relationships between the system’s parts to understand the potential for better decision-making. The system isn’t just a collection of things, it consists of elements, interconnections and a purpose.
A football team is a system, with elements such as players, a coach, field and a ball. Interconnections are game rules, strategies and players’ communications. The purpose is to win games, have fun, or have exercise. We are all members of numerous systems and subsystems.
Systems thinking has typically some of the following characteristics: the issue is important; the problem faced is not a one-off event; the problem is familiar and has a well-known history and people have unsuccessfully tried to solve the problem before. For these reasons, systems thinking is more a strategic than operational tool.
Make systems visible
The widely used systems thinking tool is a causal loop diagram. The causal loop diagram connects the cause and effect relationships between selected variables. It helps to answer the question “what structure could be causing the system’s behaviour”. Causal loop diagrams contain elements such as variables, links, effects, constraints, delays and feedback loops. See the simple causal loop diagram example below:
This example shows what happens if the product release cycle slows down. An element can have an effect on another; such as if features per release increase, then the number of defects per release increase. A causal link effect also works in the opposite direction; such as if time consumed for the overhead costs increases, then the time consumed for product improvements decrease.
The best way to implement causal loop diagrams is to invite the necessary people to the same room and sketch the diagram on a whiteboard. The output is a shared understanding of how the system works. The next step is to think how it’s possible to change structures or goals to improve results. It is recommended to keep the model as simple as possible.
It’s a learner’s game
According to studies, people are more motivated and have more positive feelings towards activities where they have choice, involvement, and control. By definition, a game is not a game without these things. Games also let people make mistakes without great consequence.
A good resource is the Systems Thinking Playbook which introduces thirty systems thinking exercises. The purpose is to promote a greater awareness of systems under the guise of play. Harvest, paper tear and community maze are just a few examples.
Exercises work well for teams’ workshops and these games last from five minutes to one hour. The necessary equipment is typically available in any office supply store. The causal loop diagram is a good way to pull together the exercise.
Less irrational decisions
Systems thinking helps us to understand why people behave like they do. It is a tool for modern decision making and suits well the Agile mindset. Seeing the whole also reduces suboptimisation and jumping from crisis to crisis. Systems thinking has shown its power when someone asks, “Should we think of other solutions rather than adding more team members to our software project”?
Thinking in systems by Donella H. Meadows
The Systems thinking playbook by By Linda Booth Sweeney and Dennis Meadows