As we know now, the world is full of complex systems. However much we wish, we can’t avoid them. We see them in our organizations. They are there in our societies. They can be experienced in ‘cloud bursts’ and in flash floods. We even find them in our families.
They are nagging and they are wicked at times. Wicked in the sense that it often leaves us baffled preventing us from acting skillfully.
The question is how do we deal with them?
It is easy to deal with any complex system if we are able to predict their behavior over time.
However, the idea of prediction is a bit different from our usual idea of prediction. Our usual idea about prediction is, if we know sufficiently about the behavior at any point of time, we would be able to predict the behavior of a system any time in the future.
But that is not what bothers us about understanding and dealing with complex systems. And it is fair to say that such predictions are absolutely impossible with complex systems. Therefore, with any complex system all we are interested about is to detect the appearance of a ‘black swan‘. A black swan is some sudden unexpected change in the behavior of a system that disrupts the system and brings it crashing to the floor. The crash of 2008 provides an extreme example of a black swan. But such examples are rather common in our lives. Our careers crash. Organizations crash. Our health suddenly crashes. A ‘cloud bursts’. Or for that matter any system is likely to be disrupted any time by the sudden appearance of black swans.
So what might we do about it? Does it help if we notice the change in behavior of each agent or individuals that form the part of the system? For example, does it help if we watch individual behavior of employees in an organization? Or for instance, does it help if we monitor individual performance of school or college students? We know that such methods hardly help improve the system though we are enslaved by such methods by ‘blind faith’. A complex system would keep doing what it does. That is its role or purpose.
The good news is that the behavior of any complex system can be monitored and predicted for black swans, a little in advance, before it strikes us with full force to bring the system to its knees.
In order to predict black swans we need to know of one very peculiar phenomenon of any complex system. That is a complex system behaves linearly for most of its time when it is free from a black swan or an outlier. Then as a black swan slowly creeps into the system the system suddenly behaves non-linearly. When it behaves linearly it gives us a false sense of security. We feel everything is fine and hunky dory and would stay like that forever. We take pride in our design.
Lulled by our false sense of security, we then forget that non-linearity is just waiting to strike us. And when it strikes we are so much confounded that we rush like headless chickens to ‘fix’ the ‘problem’. And believing in our superior intelligence, we keep ourselves busy ‘fixing’ problem after problem till we drop dead from such heroic efforts.
Mathematically speaking, while linear behavior of any complex system follows Gaussian distribution; the nonlinear behavior follows some sort of power law. So, it is the mix of the two, never one thing or the other.
By understanding this phenomenon clearly, we can ‘predict’ a black swan or an outlier very easily. It is deceptively simple. Simpler than what we perceive it to be.
It is liberating too. Once the presence of a black swan is detected, much before it actually happens, we are left with enough time on our hands to deal with it effectively and skillfully. It also leaves us with the possibility to dramatically improve the system big time.
The truth is there is no ‘randomness’ anywhere. The concept of ‘randomness’ is a big illusion at best.
So, then, what are we waiting for to improve our lives?
Reliability Management Consultant Pvt. Ltd (RMCPL) 
“The truth is there is no ‘randomness’ anywhere. The concept of ‘randomness’ is a big illusion at best.
So, then, what are we waiting for to improve our lives?”
Hi Dibyendu… I’m going to have to disagree with you on this. I’m assuming you’ve read Nassim Taleb on Black Swans. I tend to agree with his reasoning that Black Swans are random absolutely, and that it is the manner in which we address the uncertaintly they present that matters in terms of being resilient and productive despite what we absolutely cannot predict… we control our action, reaction and proaction despite our total lack of control for the antecedent event… a Black Swan.
To, nemetics is essentially a language connoting a form of art. Perhaps through a combination of inductive and deductive reasoning about unpredictable events, our skill in dealing with them (nemtical analysis?) grows and we become more “artistic” in our ability to cope with uncertainty, what I believe actually to be a much more frequent and prominent element of human interaction and communication. I believe that randomness is present everywhere, and that nemetics is evolving as s vessel to cope with it and understand it better. It’s a vetting out process analyzing what could be removed from the classification of randomness for more logical analysis, and then more positive energy can be applied to accepting the unpredictable event and deriving a purpose for it moving forward… asking the question, “what can we learn from this?” perhaps so the random act becomes less random if it was to reoccur. For example, if we were to accept that our limited control of knowledge and awareness frames our inductive understanding that swans are white, then also accept deductively (once discovering one that is black) that animals within the same species come in different colors, therefore swans could also come in different colors, we would be less shocked to find a green swan if in fact we were to discover one in the future, and quicker to learn how to accept and effectively interact with our new green swan friends:) Nemetics at its best?
LikeLike
Thanks Sean for the deep thought provoking comment.
If Taleb felt that ‘random is absolutely random’ then I think he partially revised his views in his follow up book ‘Fooled by Randomness’. But I think his POV needs to be revised further to include randomness of different types of systems.
We can see our world in many ways. But the ways in which I see it are two, which are a) Linear b) Circular.
Now if I see it in a linear fashion then random events really look absolutely random. Random in what? These are random in the temporal sense. That means the periodicity of appearance of any random event or emergence on a given and accepted timeline is not uniform. If something happens in x time from a reference point (in time) then the same thing might happen again at say 1.7 x from the previous point and in its third appearance it may be 0.5 x from the second event. It means we have no control over the timing. That is we can’t say for sure when the next similar or dissimilar event would occur. When looked at linearly it is perfectly logical to say ‘random is absolutely random’ and is perhaps the only view of the phenomenon from that angle.
But when I look at it in a circular fashion the view changes from a purely temporal view to that of a spatio-temporal view that includes both space and time. There we see the ‘thing’ taking shape in a given space over time through interactions. But we must be careful not to consider the cybernetic view of ‘feedback’ only, since such a view excludes the possibility of an open system operating away from equilibrium. Emergence (a random event) is only born in an open system operating away from equilibrium conditions.
Then what causes a system to operate ‘away from equilibrium conditions’? It is the energy that is either lost by the system or taken up by a system from another system losing energy. Both cases give rise to ‘randomness’. Conceptually that is entropy at play.
If that is so, then we can always monitor this rise or loss of entropy of a system. So long it is under steady state nothing much is expected to happen. That is no ’emergence’ is expected to take place. This period of activity neatly fits the linear range with its usual variations, of course. Therefore, it might easily be expressed in Gaussian distribution. But as soon as there is an excess build up of energy or a sudden (Power law) loss of energy (entropy) an emergence is surely expected. Such a change can be spotted in time before any emergence actually manifests on the scene. Once we spot it we can ‘predict’ the coming of a particular ’emergence’. There appears to be no harm if we are able to do so. However, it also means ‘randomness is not absolutely random’.
This signal is vital since it triggers a meaningful inquiry of the emergent phenomenon, which might lead to a redesign to contain the randomness to a great extent. This is because the circular view may then help us to understand the significant interactions that helped the emergence to manifest. We might always redesign the nature of the interactions.Can’t we?
It is an easy and effective method, which we might hardly ignore.
LikeLike
I think what might be tripping up Sean is the matter of ‘both’ — which is a fundamental operating principle. The universe operates on a principle of choice — even the elements make choices, but that choice operates within a realm in which there are some fundamental ‘constraints’ (elements of guiding design principles) at play.
I love watching how these play out — to look for the dance in it all.
LikeLike
I agree. The idea of ‘both’, a fundamental principle of any design or any complex system, appears to be counter-intuitive to many. In fact a complex system operates on the principle of least action. As you rightly point out it makes choices. At any instant of time it selects an action from the multiple choices a system has. And as soon as a choice is selected the system is reset to new initial conditions or a new state. When it is linear, with variations, it has selected an action. When it is non-linear it has also selected an action based on the ‘least principle’ (minimal intervention). So linearity and non-linearity are both choices before a complex system. It depends what choice it makes at any instant of time.
LikeLike
I’m not sure to be tripped up… the statement I firstly disagreed with was “the truth is there is no ‘randomness’ anywhere. The concept of ‘randomness’ is a big illusion at best.”
This assertion connotes an absolute void of randomness. The idea of both appears to be counter-intuitive to this statement.
The overtly mathematical and physical basis of nemetics will be lost on 99% of people. I am the furthest from an expert (read I am a neophyte) pertaining to math and physics, and for nemetics to have on ground positive influence for people, it will have to be understood in common, simple language.
I perceive that nemetics as explained in a complex system derives from the quantum side, and perhaps explains the view that randomness is never really random owing to things never actually being at rest… motion is observable, and perhaps predictable owing to laws of quantum physics?
Anyway, if not for practical purpose to improve human interaction with each other, and with other elements, I wonder what is the purpose of creating this language/process. Sometimes I feel like nemetics just has to hit the ground running:)
LikeLike
Sean,
I think everyone in this thread believes Nemetics jargon is the exclusive domain of us lunatics 🙂 When talking to the real world whatever insights are derived must be in the language the other speaks. The opportunity with our little band of lunatics is the words seem to capture reality as each of us has experienced it. I think I “know” more than most about political economy, sociology of knowledge, demography and printing. I know from exerience the frame works to clarify.
But I would never say to my printing buddies Print is a neme that triggers cascades of emotion, cognition and thus increases the possibility of new behavior. They would think I was a lunatic.
When I see that Dibyendu sees how powerful a simple framework with only four or five fundamental terms can be in his work. And when I see that Brenda said to me today on twitter that she sees Noitce or not, Engage ornot, Mull ornot. Exchange ornot all over her work in education, it’s an important data point to say we are on to something very real.
The way I see it is the only point of helping more people to see what we see is to make it easier for them to do what they do. That’s the test.
Meanwhile, I would not worry about “hitting the ground running.” What I see is that you, Brenda, DD are already running very fast. It’s all I can do to keep up with you.
What do you think I’m missing?
LikeLike
Ok, cool… the inner circle is alive and well:) I agree with your view; I don’t think you’re missing anything at all. My only anxiety is that I would be at a spot someday when I thought I could explain everything through the “science” of nemetics. I am much less of a scientist than I am a sociologist (of course not officially.) From my phenomenological post-modern perspective I believe there is always a better way through analysis of the phenomena that is the human spirit as affected by human experience and perspective. I have always thought of nemetics as this process of analysis. Withmore time on my hands now that summer vacation is here, I am trying to wrap my head around what nemetics looks like in the real world… why I am so intrigued by the publishing side of things. Ed and nemetics. Social media and nemetics. Teacher training and nemetics… and whatever else our band of conspirators dreams up. I don’t think you’re missing anything… I just want to make sure I’m not missing anything:)
LikeLike
I’m sure we are going the same way. ( Lots has happened while you were busy at school 🙂
“Phenomenology : The science of phenomena as distinct from that of the nature of being.
An approach that concentrates on the study of consciousness and the objects of direct experience.
Any object or system of interest in nemetics is a phenomenon. – a pattern of neme exchanges. A neme is sent a neme is sent in response. The first player sends a neme in response to the others response and on and on we go..
I like to take the stance of humility. At the end of the day we don’t really know much except what we can see in the responses of the real world. Nemetics is not yet something to be learned. It is merely a code that is useful or not useful in any particular situation.
While I think it’s fair to say the I, DD and Daniel have been playing with the words persistently over the last year or so, it’s a completely open frame. As soon as we think it is now longer useful, or if we find some words that are more useful, off it goes into the dustbin of history. Most likely to emerge again in another form.
“he human spirit as affected by human experience and perspective.” yes. The spirit is made partly visible by the patterns and properties of the nemetic exchange. Human experience is the nTube that has emerged from persistent nEx in the past. Usually crystallized in the moments of response to a traumatic event. Perspective of course is a Notice focus created by an interaction of the nTube and the nShape in which it finds itself.
Btw, I much prefer to think of the Art of Nemetics. Science only in the sense of common sense rigorously applied to the phenomena we confront every day.
So what say you?
LikeLike
Michael: Both of you are right. There is necessitated that it be both science and art. Both are necessary for the functioning ‘both’ I mentioned earlier. In my mind’s eye, when both are embraced we achieve ‘optimal’ design — design is the embodiment of ‘both’. ‘Optimal’ design is the embodiment of all things at their ‘best’ — their highest potential (so economics fits in there somewhere too). We haven’t had too many nemetic conversations testing it for the fundamentals of economics — except when we were recently talking about choice (economics being the ‘science’ of choice).
LikeLike
Paula – Very nice point about “economics being about choice.”
Economics and poltical economy is one root of nemetics for me. I spend some time watching the markets. ( My “day job” is trying to keep my IRA safe.) Consider that money is a neme that triggers cascades of cognition, emotion and thus behavior. Watching the movements of prices on the various markets is a clear example of trying to intuite the patterns of neme exchange to inform decisions assessing forward going risk.
Not surprisingly the maths for risk assessment are most developed in the world of finance and markets. The conjecture is Dibyendu will lead us to some day being able to use those maths in the service of articulating pattern formation in any complex system. There is a group, whose name I forget right now, that is already pretty advanced on that path.
If you would like to engage on Twitter or any other context, I will be happy to share what I think I see.
LikeLike
+Dibyendu De
I recall hearing about animals that avoided earthquakes because they could feel the slight vibrations that indicated the coming event. When there are enough sensors monitoring the interconnected parts of various systems then I’m guessing black swans we’ll have extended senses that can protect us. While black swans may prove worrisome, there are catastrophic risks such as rising sea levels that are to be expected from global warming which require people to adapt. So much human infrastructure is unstable and engineered without much thought for tomorrow. In any case, I’m glad you’ve written about this subject and I hope to learn more from your insights as @PredictSwan.
LikeLike
Thanks a lot Dan for the encouragement to write more on this subject. The point really is very simple. If we know something is happening, just like the ants or the hens understand the coming of an earthquake or a tsunami, it is no longer a random event. However, if we don’t know anything about its existence and it catches us unawares it becomes ‘random’. This, however, must be something away from the usual pattern of things that we get used to. The ‘signal’ that informs creates all the difference. Fortunately, such ‘signal’ or information comes ahead of an event, which helps us to notice, engage, mull and exchange (NEME) in time. Without this, it is simply not possible to spot and understand any ongoing phenomenon of interest.
LikeLike
Thank you for “If we know ( know as in high probability of occurence) something is happening it is no longer a random event.” Best description of your comment that randomness is a myth. More like the perception of random – aside from random number generators in math – is a product of the history of ourselves and the appropriateness of our Notice.
LikeLike
Pingback: Predicting Complex Systems - rgbwaves | Philoso...
Pingback: Predicting Black Swans – Part II | rgbwaves