Complexity – as I see it and understand..

Introduction

There are simply many ways of viewing and understanding the subject of Complexity. It is overwhelming at times and might confuse any reasonably interested person.

As of now we have different and often differing views on Complexity. And when it comes to understanding and applying complexity in organizational settings it becomes bewildering since there is no dearth of viewpoints, perspectives, techniques, which are often in conflict to each other.

The main reason as it appears to me is that the subject of Complexity is not made up of a single body of knowledge like Physics or Maths or Economics or Philosophy. It is mix of many things. In a way it is good that it makes the discipline extremely rich but flexible at the same time. But nonetheless it can also make things a bit weird at times.

In addition the progress in the field of complexity has tended to take the ‘scientific ‘ slant trying to make things as objective as possible, usually by the help of mathematics. That puts off many who are not mathematically inclined. The reason for taking this mathematical view is not hard to understand. The development of ‘scientific management’ was started by engineers and as we all know engineers love to understand the purpose of a system through measurement, make sense of things through their calculations and then love to predict the future through their created algorithms.

Hence this practice of measure, calculate and predict has become the standard norm in understanding systems. Nothing is wrong with this rigorous approach (myself being an engineer) but at times things are taken a bit too far than necessary – forgetting the basic fact that understanding and responding to complexity is quite unlike any rigorous scientific experiment that usually deals with a few variables or parameters and examines how these variables behave under certain specific conditions (specified by us) strictly in isolation of other factors.

But the study of complexity is completely different. In fact it is just the opposite. It is complex just because there are simply too many variables/parameters playing around in a system – constantly interacting and colliding with each other like atoms and quarks to produce something new. It is an amazingly creative play.

So the study of  complexity is not about observing the movement of individual trajectories of a few variables/parameters operating under specific conditions. It is more to do with observing the trajectory of the ensemble of numerous objects, parameters and variables playing in unison and harmony under their own operating conditions –  not conditions specified by us. It is something like watching a wave rise up at the beach as if to greet us. At that time we are not even trying to see individual strands of water. Well that is impossible. We are seeing the movement of the whole wave as such where the trajectories of individual strands or strings of water combine to produce the entire wave form. In other words we are no longer interested in how a few variables/parameters will behave in different conditions. The conditions are given or as we might say self-created by the system itself – not by us.

The usual way to understand a discipline is to go through most of the theories a discipline offers, master them to a great extent and then go out and see the world with a new pair of lens to make sense of what is happening to decide as to what might then be done about it – i.e improve a situation, maintain it or destroy it.

However, my preferred way to view and understand the discipline of complexity is to turn this usual approach of mastering theories stand on its head. I have seen all systems, even similar systems differ from each other even while operating under quite similar settings. Their behavior differs, which means that every single system or part of the system that we care to observe is unique in its own way. If that be so a few simple rules, laws or theories would not be able to explain such uniqueness expressed by a complex system in variety of ways.

Therefore we would be forced to create an explanation of each system operating in its own unique conditions. This means we would not be able to apply our learned theories or any equation in a straightforward manner. Why is that? That is because of the numerous interactions that keep playing within a system. So there would be many laws, rules, principles interacting with each other to produce what a system produces. So, every time we see a new behavior we are forced to first explain it properly (as close as possible) and then possibly predict its behavior in the short-term since long-term prediction of complex system is impossible.

Hence our earlier approach to viewing the world was ‘Predict first; Explain later; only if the prediction doesn’t match the theory’. In case of complex systems it is just the reverse – ‘Explain first; Predict later, where only prediction in the short-term is allowed.’ This creates a big difference in the way we engage with our world. It means that we don’t have a set of equations to be applied uniformly and normally. The best we can possibly do is to have the data from the present behavior and then create the maths around it. That calls for mathematical thinking not mathematical prowess or great mastery of mathematical techniques.

A Question

However, my question is can we have a balanced view of complexity that might be equally appealing and easily understood by people from both science and humanities? Can it be made simple for every one to understand and apply such understanding not only in organizations but also in our everyday mundane and ordinary lives? If that is possible it would be truly useful to us. Else useless. If it could be made useful then it would not only help us adapt to changes effectively but also help us create something new and better for ourselves and the world at large.

Let us take a live case… 

To illustrate the process let us work through a case in very general terms. Presently I am exploring the complexity of system called Blast Furnace. This system is totally unpredictable. No one is quite sure what exactly is going on inside the system and when and why something would happen.

Let me explain the ‘transforming process’ that happens in a Blast Furnace. Imagine something like a big vertical stove of around 30 to 50 meters high. Hot air is admitted through the bottom of the stove. This hot air moves up the stove due to the pressure difference existing between the bottom and top of the stove (the bottom is at higher pressure than the top).  This provides the energy to the system. From the top iron ore and coke are periodically fed into the stove. This mix, called a burden, takes time (around 6 hours for a mini blast furnace) to come down slowly over a distance of say 30 meters. As it descend through the air the iron ore and coke come together to form layers of rings. Meanwhile the coke starts burning as it interacts with the hot blast air. The heat from the coke melts the iron ore which at a particular stage transforms into a liquid. This molten iron is then tapped regularly from the bottom portion of the furnace ready to be transformed into other products.

Is there something to learn from this?

From this simple description I would like to highlight six fundamental principles of Complexity, which are the following:

1) All complex systems are ‘transforming processes’, which are non-linear and dynamic. (don’t confuse the term ‘process’ with the step by step process we are generally used to in trying to achieve a desired objective). This means unless and until the system is moving it is not complex at all. Complexity arises out of movements initiated by interactions. And as you can see the process that happens within a blast furnace is a ‘transforming process’. Raw materials through various interactions are transformed into molten metal. And different materials through various interactions behave in a particular way. Moreover, material through their interactions produce different shapes.

2) Complex systems need energy to sustain itself. It is simple to understand this. Without energy there can’t be any movement. So as soon as energy flow is stopped complexity disappears. Once energy is admitted into a system the energy spreads all around forcing the different elements within and without the system to interact (even with the environment). Energy can be in different forms. Even communication between different persons within and without in an organization is a form of energy exchange that sustains complexity. Money is also a form of energy exchange sustaining complexity.

3. Interactions between different elements within a system creates complexity. And such interactions are non-linear in nature. The vortex effect arising from the interaction between coke and hot air is an example of non-linearity. Similarly the burden of coke and iron ore taking around 6 hours time to fall through a distance of say 30 meters is another example of non-linearity. This is because in normal case, if I drop a piece of iron ore from a height of 30 meters  it would take a few seconds to reach the ground. So it is fair to say that interactions and non-linearity are the driving forces in complex systems. If any one of these are absent there can’t be any complexity anywhere.

4. The elements in a complex system act in groups or ensembles. For instance the coke forming a spiral cone or the burden forming layers of rings as it descends through the furnace is an ensemble that has its own particular behavior. If it were a simple system every particle of coke would have acted as an individual having its own individual trajectory. Whereas in complex system they come together to behave as a group exhibiting group behavior. Hence we observe behavior of an ensemble not individual behavior of the interacting objects that make up the ensemble.

5. The movement within a complex system is first initiated by valid or authentic constraints. For example, the pressure difference between the top and the bottom forces the hot blast air to climb up. Similarly, the burden climbs down owing to potential difference. Likewise the hot blast air meets with a resistance as it makes its way through the spiral of coke that forms at the bottom of the furnace. Technically this is called the ‘dead man’. So long it is alive the system would work. Remove the ‘dead man’ and the system collapses in a moment. So the movement in a complex system is facilitated by Energy, Constraints and Resistance (resistance in a system is technically known as damping).

6. The shapes these interactions produce is fascinating to see and think about. The interaction of the coke with the hot blast forms a spiral cone of coke through the vortex effect. This spiral cone of coke admits or blocks the passage of hot blast in the furnace. The interaction of the air pressure with the burden not only makes the burden come down slowly over time but also forces it to aggregate as layers of rings like donuts. The spiral shape of the coke cone is also like donuts. Mathematically this shape is known as a ‘torus’.  The reason that it does so is due to the principle of ‘sync’. The materials acquire a uniform collective frequency and amplitude in a complex system.

Let’s go back to the live case… 

Now let me go back to my description of the ‘transforming process’ happening within the blast furnace.

Engineers are very wary of the blast furnace. Why so? Because it operates in peculiar and unpredictable ways. For instance the blast furnace can suddenly ‘hang’ (the burden refuses to descend any further). For for example, the quantity of molten iron that is tapped out from the furnace around every hour varies quite a lot in quantity. Or the furnace can suddenly ‘slip’, meaning that the burden suddenly accelerates its movement and comes down too quickly. It appears that it has a mind of its own. This high degree of unpredictability is simply charming to blast furnace engineers. That is why blast furnace engineers refer to a blast furnace as ‘she’ (no offence meant to women). But you get the idea quickly. No? Engineers aren’t quite sure what causes this. But the discipline of complexity can provide some answers to this vexing issue, which I shall now try to explain with six more fundamental concepts on Complexity.

What do we learn now? 

7. Small difference in the interactions produce such unpredictability. Such small difference in interactions between different elements (useful to visualize the relationship between different elements as vibrating strings) create the uncertainty in the behavior of the ensembles of various elements like the coke, burden etc. The uncertainty that happens over time in a non-linear fashion is exhibited by the formation of new patterns displaying new forms of behavior. If it were linear it would have been highly predictable. Isn’t it?

8. It is interesting to note that such differences are in the form of fluctuations and the group behavior that is displayed is also in the form of fluctuations (hence the image of a vibrating string might be helpful).  Owing to the interactions such fluctuations suddenly amplify (again a display of non-linear behavior) to create new forms and new behaviors both of which are highly unpredictable in long-term.

9. So when a complex system is operating normally, i.e no unusual behavior is observed the system can be said to be operating in equilibrium, i.e. it continues to behave the way it was behaving a moment earlier (very linear). In such a situation the fluctuations either die out or remain in a steady state situation. It implies that the damping factor or the resistance is uniformly applied on the system and there is no non-linear changes in energy content or constraints.

10. However, when the energy content or any (or all) constraint(s) in the system or the damping in the system changes the fluctuations suddenly amplify to produce complex uncertain behavior of the system. Then it is said that the system is suddenly operating away from equilibrium conditions. When this happens the system breaks the symmetry of time (where the past is not the  same as the present) to create a new behavior or a new form on its own (technically know as self-organization). Fair to say that it is the ‘difference that creates the difference’.

11. When such time symmetry is broken then something called ‘bifurcation’ occurs. In plain language it means the system opens up two possible paths to proceed. But what path it cares to take is not known or can be predicted in advance with any degree of accuracy. The system ‘chooses’ the path. Do we have an example of this in our description of the blast furnace? See that the furnace can either ‘hang’ or ‘slip’. These are the two possible paths that are available to the system. What path it would take and why it cares to do so is not know. It is completely up to the system’s ‘collective and contextual intelligence’ to decide the path it prefers to take. Certainly this is out of human control. I love the associated philosophical meaning. While dealing with complex systems human beings have no control. It is much like what keeps happening within our own bodies and minds. We can also see enough examples of that happening in Nature. Enough to say that it is ubiquitous.

12. We may conclude that all non-linear dynamic complex transforming processes are very creative and are highly intelligent.  It possibly helps us to realize that the causes of complexity lie within the dynamics of the system. They are neither outside the system nor separated by time and space.

Is there a simple way to negotiate complexity?

But how do we deal with such complex systems? It is clear that we can’t force our way through complex systems or make it behave the way we like. A possible way that appears to me is to befriend a complex system, listen to what it wants to do and then take creative measures to make it flow in our desired direction or decide to flow with the system (adaptation) or create new directions (creation).

The possible path to effortlessly flow along with any complex system might be stated in a simple way, which is as follows.

a) Pay close attention to the changes in behavior of the system.

b) Pay close attention to the difference in the fluctuations arising from the numerous interactions that go on in a complex system to note when these die out or when they amplify.

c) Change/Re-design, Maintain, Destroy the Energy content of the system, Constraints of the System or Damping of the system. These are the only ways to play with any creative, intelligent complex systems.

It hardly matters whether we approach a complex system through qualitative understanding or through quantitative mathematical techniques. It is all the same. But while trying to tackle through quantitative techniques it is useful to remember that we don’t start with an equation and then apply it to predict as to what might happen. We start by observing the changes in behavior and difference in fluctuations and then form necessary equations and algorithms to explain and predict in the short-term.

These 12 principles can be applied to any complex system with ease.

Conclusion

To conclude Complex Creative Systems is philosophically very appealing to me. Human beings are always in search of meaning, purpose and prediction. I believe that we do so since it is our innate nature to transcend our small self in order to reach the infinite to be joyful and happy.

For this, we have tended to look at the past to find causes. Or we have projected our mind to see the future, however dimly. Then we have tried breaking down parts of the system (reductionist method) to create meaning. Or we have tried hard looking at the whole (holistic) to find the purpose.  Both approaches are frustrating and stressful to say the least when we use them to deal with complex systems.

Understanding of Complex Creative Processes offers us a new way — to be in the present moment and be there. It is the place where the past, the present and the future converge and meld into one. It transcends both reductionist and holistic approach simply by paying attention to the given moment. That is happiness in the true sense of the word. The ability to do something worthwhile from that sets us free to change ourselves and possibly the world. The future is unknowable but at the same time can be created. Isn’t that powerful?

I agree with the Upanishads, ‘You are That!”

References:

1. End of Certainty by Ilya Prigogine

2. Web of Life by Fritjof Capra

3. Sadhana by Rabindranath Tagore (Tagore Omnibus)

4. Complexity and Management by Ralph D. Stacey and others..

Note:

If you are interested in understanding the approach to flow with complex systems you might like to go through another post of mine, which is ‘How to Embrace Complex Systems with a Smile’ http://wp.me/p2CS2f-A

To a Teacher on Teachers’ Day…

5th of September is special in India. It is celebrated as ‘Teachers‘ Day’. This is a day to remember with gratitude the most precious asset in our lives – teachers. It is a day to reflect on the contributions they left behind in our lives and how close we are to the paths they showed us to lead our lives.

Every year this day is special to me. I remember with fond gratitude my teacher Prof Tim Henry. He has long retired from the University of Manchester, UK. But he continues to live life creatively and with lot of zest. This is the letter I wrote to him today – (certain portions are explained in italics to make sense of some words I used)

Dear Sir, (In India we refer to teachers as Sir).
My pronam (it is a deep bow touching the feet of a teacher) to you on this day of remembrance. It is not that I wait for this day to remember. It happens every morning as I am up to be delighted by another day. And if there were something more than gratitude, I would love to offer that to you. Hope you and Madam are happy and healthy.
As life moves on for me it becomes simpler by the day. Increasingly I am getting to love my “aloneness”, which I feel is a necessary component of life at this stage.
There is no need to overdo anything. Or over convince anyone. Or be too much active than needed. When I look back at the years gone by I am amazed as to how much activity I did and the energy I put into those, which I believed as hard work – much of which was plain stupid. The constant pressure to prove myself and a maniacal attitude to share what I enjoyed so as to inspire others was stressful to say the least. Now I do what I like or love doing not bothering to inspire or impress others. It is a joy. I no longer harbor any illusion or even the faintest intention to teach others anything. I realize that the way through which I saw and felt life and my work is something that can’t be taught to others. You were right when you warned me not to attempt teaching the ways. I was a fool then not to have listened. Fair to say all my attempts to teach others have failed. I could not develop any standardized format for my non-standardized ways to effectively teach others however enjoyable that might have been.
Having left that ‘outward journey’ I feel peaceful and breathe with ease. What a relief that is with a simple realization that I can’t change anything, less the world except change myself. The world has its own crazy unpredictable ways of moving. Watching that is of course a pleasure.
Rick is now in his fifth and final year of his architectural course. In less than an year’s time he would be well on his way with his own ideas repeating the same cycle – bubbling with lots of ideas and concepts waiting for expression all of which one day he would also have to shed like me being defeated by our so-called society that fanatically believes in an ordered ‘linear world’ – a world created by our imagination only.
Rit is in his second year of his electronics course and struggling with Laplace and Fourier Transforms. He is yet to appreciate the beauty and the limitation of maths. But seeing me do that for years he feels he can somehow handle that. He often strikes up conversations on these and it is a joy to engage with a willing and experimental mind. He understands when he can visualize maths. That is quite a challenge for me to help him visualize the nuances of maths.
Madhumita continues to struggle with her health, which has remained the same in predictable ways. She is now in a steady state equilibrium position — perhaps a very difficult position to maintain in life. I always thought life oscillates in non-linear ways operating away from equilibrium. Like many other concepts I held in my mind she proved me wrong. So I had to let go of that concept too.
Perhaps I am now confused about many things in life. Nothing is certain, sure and predictable in the long run. That is possibly a lesson I would have been forced to learn in the hard way had it not been for CBM (Condition Based Maintenance of systems based on non-linear dynamics) principles you taught me in the formative period of my career.
I still continue with some interesting work, one of which is the Blast Furnace. It appears that no one has been able to predict as to when a blast furnace would ‘hang’ or ‘slip’. When I started exploring the phenomenon it gave me nightmares and left me clueless for the first three months of the study. Then suddenly an idea struck me. What if I trend the difference of the frequencies of the multiple phenomena going on in the furnace? And as luck would have it I am now finding a pattern to go with and predict the situation 3 hours to 45 mins in advance, which is a big breakthrough as I understand from my client. Luckily the client did not expect any outcome from my play. But now they are sitting up and expecting more from the study. To me the blast furnace is the most mysterious of all systems I have seen and engaged with so far. It produces beautiful patterns on its own in a spontaneous manner. It simply leaves me fascinated and wonder-struck – Nature at her best. Incidentally I started off by using the same trend sheets you designed but the only difference is that I am plotting differences not the actual value of the parameters. This is because I think the differences feed back into the system dynamics to produce new order from non-linearity. It is fun to see stability and instability appear at the same time.
The other interesting work I did was on quality improvement in a Japanese factory that makes ductile iron pipes. Since the Japs failed to make any significant breakthrough in bringing down quality rejections to an acceptable limit through their time-tested universal methods it became a challenge to see quality improvement through the new perspective of non-linear dynamics. And soon I found that non-linear dynamics gives instant clues to quality problems which I did not suspect any earlier. It was a year of fun. For example, slippage of v belts of the main pulley that powers the mold of the casting machine causes as much as 100% rejections. It was hard to believe this. But then I discovered how true it was when small changes in fluctuations amplify over time to cause such problems. The causes of problems lie in the dynamics. This to me was a new understanding in quality improvement though it took time for the client to swallow all that.
Work on downtime reduction also continues. But that has become a piece of cake for me. It is as easy as A, B, C. The fun part is as I approach a machine or a system and pay attention, the machine starts off an extremely interesting and lucid conversation with me and keeps talking to me all the time like a girl friend would have done so in my younger days. All I do is to hear her intently with all the pure attention I can shore up and then simply write out the prescription for the client. It is lovely to hear the stories machines tell. For example, maintenance engineers take ‘leakage’ in a very straightforward way – just change seals. But ‘leakages’ tell me very amazing stories. It tells me where dimensions are not right, where synchronicity is a problem, where pipes contribute to the leaks and it goes on never stops surprising me… the stories are fascinating.
During the last week of August we had our usual Vibration Level 2 certification event at NTPC (National Thermal Power Corportion), PMI (Power Management Institute) at Noida This time I changed the nature of the interactions between participants and myself. No more of those dreadful Powerpoint slides. I asked the participants to bring their own cases where they failed to make any breakthrough or weren’t sure of what they did or simply wanted to know more about what was going on in their system. 28 participants came to challenge the test. It was a fun-filled event and I think they got inspired by the way vibrations tell stories. From their signatures and wave-forms when I said that a CW pump (Circulating Water pump without which a power plant can’t run) was too near the walls of the water tank and the distances between the 3 pumps were less than usual they gasped. They were equally surprised when informed that a particular pump that failed had its delivery pipe bent at the wrong place. Or they wondered when I said that the catenary of a turbine shaft changed since the oil pump wasn’t working properly.  They thought that I was gifted with the gift of clairvoyance. They found it hard to believe that I just listen to the stories vibration waves try to tell me and the stories are just available to anyone who cares to pay attention and listen.
Such new learning and understanding has made my life more simple and enjoyable. Never a boring moment for me.
Would love to hear about your life since that has been the source of my inspiration for years. Do you and Madame still visit Wales? Is she still busy with her paintings? I feel that I must again take up sketching and painting a bit more seriously. That is another thing that gives me a lot of pleasure. What about Jill and Kay and their children? Has Natalie, if I am not wrong, joined the architectural course? And what about the little boy? He seems to have taken after you. Is he interested in Engineering?
I think  in more ways than one I am simply copying you or rather cloning you without having any of your genius to create a fascinating discipline. I have stuck to your advice of staying alone and not expanding my practice into an organization. It has helped me tremendously – giving me enough time and space to go down the numerous rabbit holes of puzzles and problems and stay in a perpetual state of wonder and curiosity. It seems that I would never grow up though much of my hair has disappeared while the rest are quickly turning grey.
In the later years of your career I saw how you were engaged but detached at the same time without failing to enjoy what you did. To me it appeared to be the most sensible way to go forward in life and as I continue to work in a detached manner life becomes more enjoyable. It is hard though!
Pronam to You and Madame
Dibyendu