CIO Upfront: Enterprise architecture and the legacy system conundrum
EA is probably one of the most critical areas of the technology domain and it is also likely to be the least understood. More often than not, the purpose of EA is to design replacements for old legacy systems.
At the time, in operations, we were receiving solutions that were well designed on paper, proven in development, proven in test environments, but then implemented differently from the tested design. This resulted in production problems, which required emergency changes in order to make the new solution operational.
A lot of the problems could have been avoided if the existing systems were well documented and well understood. This is an issue almost every organisation faces.
The technology landscape in most businesses is littered with ageing legacy systems doing a respectable job. In one organisation where I worked there were five billing systems all doing very similar tasks and producing similar output. This was the result of five separate businesses being acquired over a short period of time. The company commissioned a project to replace all five billing systems with a new single billing platform.
Some of the problems described above -- stakeholders being left out, lack of good migration planning, lack of documentation, and so on -- made sure the project did not go well. Instead of replacing five systems with one, a new sixth system emerged, sitting alongside the other five. It took a further two years to migrate all five to the new sixth system, at a significant cost to the business.
As time passes by, documentation, knowledge, the critical intellectual property gets lost or leaves the organisation. Additionally, legacy systems live under the spectre of other potential issues. These can range from:
" Vendors not supporting older codebases
" Vendors no longer in business
" Systems where source code has been lost or the original developers no longer in the organisation
" Capacity issues that cannot be resolved by hardware upgrades
" Other age or time related problems.
EA attempts to solve a very complex problem that isn't well understood or properly defined. From a legacy environment perspective the issue is clear:
How do we get from something that was once fit for purpose to something that needs to scale, evolve and be easily upgraded with little or no disruption to the existing systems and processes
With today's focus on scalability, standards, and security, legacy systems don't meet current expectations. The legacy environment could be historically classified as a well-planned town, but now, with the passage of time and more importantly progress, it starts to resemble a slum or shanty town.
The building analogy is appropriate, especially when we see how well some of these shanty/slum structures continue to stand and support the demands placed on them (almost one billion people live in such structures, according to the United Nations Human Settlements Programme (UN-Habitat).
EA today attempts to address how we build modern technology structures (much like high-end skyscrapers) in slum areas but not how we change the slums themselves. Skyscrapers are rarely built in slum areas without first rejuvenating the entire area itself. Some are built on green field sites but most rise out of the rubble of previously demolished buildings in already well developed cities.
The challenge that EA faces is how to turn a slum into a modern town or city without moving the current residents and by causing a minimal amount of disruption. Continuing the analogy to its logical conclusion, urban redevelopment/renewal requires long-term planning, long-term funding and more importantly, a long commitment/vision to see it through.
The evolution of the legacy environment (the slum) is what EA should attempt to address. It may seem shocking to consider a legacy system/environment as a slum or makeshift structure, but the context needs to be made clear: As technology evolves at an ever increasing pace, previous technology solutions seem increasingly basic, primitive and even disorganised. It is a rough analogue to the second law of thermodynamics where entropy increases as time moves forward.
As technology evolves at an ever increasing pace, previous technology solutions seem increasingly basic, primitive and even disorganised.
In layperson's terms this means that systems become or appear to become increasingly disordered as time moves forward. This is especially true as we discover new and better ways to achieve the same results. Advances in technology enable us to do things which we previously inconceivable. Storing entire application systems or databases in memory could not have even been considered a decade ago. Today this is commonplace.
Additionally, as time moves forward, we actually tend to lose knowledge and context of how things used to be. A classic example of this is how technology has changed the way we communicate with each other. Modern messaging, mobile phones, the Internet have all changed and shaped our lives to such an extent that we cannot communicate without them. How did we previously inform people of we were late to a meeting
It's clear that EA needs to focus on a long-term vision and a strategic commitment holistically. Some of our greatest cities and architectural feats didn't happen overnight. Positive and continuous incremental change is required. EA should address these issues by providing guiding principles and a loose enough framework to allow a progressive evolution.
Until we start to embrace the need to for continuous incremental change, we'll be forever stuck in the vicious cycle of big bang projects designed to rid us of legacy systems that cannot be easily replaced without a change in the technology landscape.
Bradley de Souza (bradley@xtanz.com) is an internationally recognised CIO/CTO who has specialised in change and transformation across industries around the world.