Tag: Prediction

Trend charts: Inflation & Unemployment predictions

On By Author of #PowerofSeeIn Education, Management, Nemetics1 Comment

Trend charts: Inflation & Unemployment predictions

Here is a nice set of charts that try to correlate inflation and unemployment in the states and see what best can be done now.

“The Federal Reserve is said to have a “dual mandate”: keeping inflation in check and the unemployment rate low. These measures, which tend to change cyclically and in concert with each other, are charted for every year since the Great Depression.

In speeches and in meetings, Ms. Yellen, the nominee for the next Fed leader, has commented on the Fed’s actions during significant periods, providing a window into her views and priorities.”

The question is can anything be predicted from these charts?

Is it that by keeping inflation low we can boost employment?

It is difficult to say since there seems to be no linear relationship between the two factors.

What is seen instead is the non-linear dynamic nature of both the parameters.

If so, what needs to be done is anybody’s guess.

Any clue?

Exchanges: The Global Economy — Taleb

On By Author of #PowerofSeeIn Design Kata, Management, NemeticsLeave a comment

This is a dialog between the celebrated Nassim Nicholas Taleb and Justin Rowlatt about Exchanges: The Global Economy.

About:

“Do you underestimate the risk you are under? Nassim Nicholas Taleb’s ideas on probability and risk challenge assumptions made by markets and mathematicians all of over the world. His ideas on our blindness to the impact of improbable events have lead him to be described as a ‘super hero of the mind’ and ‘the hottest thinker in the world’. He is the author of the mega-selling books on randomness and the impact of the improbable on life and on economics – Black Swan and Anti-Fragile. His official title is Distinguished Professor of Risk Engineering at New York University and he continually ranks as one of the most influential people on the planet. He joins Justin Rowlatt and an audience at the Grand Amphitheatre of the Sorbonne in Paris for a special event, staged in partnership with Paris Dauphine University.”

http://www.bbc.co.uk/programmes/p01p174q

So what is prediction?

It is indeed unwise to predict the future. It just isn’t possible. Our traditional ideas of probabilities to ascertain risk is flawed. So the question ‘When something is going to happen?’ is pretty useless.

It does not serve any purpose predicting the past. It is only trying to answer the question ‘Why?’. When done, it is only a story with no certainty of being right.

Then is there any use of prediction?

Possibly yes, when we ask, ‘What is happening now?’ The future is unfolding now. The incipient seeds of the future have already taken roots in the now. Those give the clues. So it is useful to check what is happening right now.

But how do we look at it? Do we do it by thinking? There seems to be very little scope of doing so.

So do we frame a story? That could turn out to be a myth.

The only plausible way is to ‘observe’ or see or notice the present. Then only we have the chance to ‘see’ the future. In other words ‘predict’ it, even if it is a potential ‘black swan’.

Hence the most important question to predict anything is to ‘see’ what is happening now and not ask why something has happened or when something is going to happen.

That is the best way we can predict anything and remain useful.

Predicting BlacK SwaNs

On By Author of #PowerofSeeIn Management, Nemetics15 Comments

As we know now, the world is full of complex systems. However much we wish, we can’t avoid them.  We see them in our organizations. They are there in our societies.  They can be experienced in ‘cloud bursts’ and in flash floods. We even find them in our families.

They are nagging and they are wicked at times. Wicked in the sense that it often leaves us baffled preventing us from acting skillfully.

The question is how do we deal with them?

It is easy to deal with any complex system if we are able to predict their behavior over time.

However, the idea of prediction is a bit different from our usual idea of prediction. Our usual idea about prediction is, if we know sufficiently about the behavior at any point of time, we would be able to predict the behavior of a system any time in the future.

But that is not what bothers us about understanding and dealing with complex systems. And it is fair to say that such predictions are absolutely impossible with complex systems. Therefore, with any complex system all we are interested about is to detect the appearance of a ‘black swan‘.  A black swan is some sudden unexpected change in the behavior of a system that disrupts the system and brings it crashing to the floor. The crash of 2008 provides an extreme example of a black swan. But such examples are rather common in our lives. Our careers crash. Organizations crash. Our health suddenly crashes. A ‘cloud bursts’. Or for that matter any system is likely to be disrupted any time by the sudden appearance of black swans.

So what might we do about it? Does it help if we notice the change in behavior of each agent or individuals that form the part of the system? For example, does it help if we watch individual behavior of employees in an organization? Or for instance, does it help if we monitor individual performance of school or college students? We know that such methods hardly help improve the system though we are enslaved by such methods by ‘blind faith’. A complex system would keep doing  what it does. That is its role or purpose.

The good news is that the behavior of any complex system can be monitored and predicted for black swans, a little in advance, before it strikes us with full force to bring the system to its knees.

In order to predict black swans we need to know of one very peculiar phenomenon of any complex system. That is a complex system behaves linearly for most of its time when it is free from a black swan or an outlier. Then as a black swan slowly creeps into the system the system suddenly behaves non-linearly. When it behaves linearly it gives us a false sense of security. We feel everything is fine and hunky dory and would stay like that forever. We take pride in our design.

Lulled by our false sense of security, we then forget that non-linearity is just waiting to strike us. And when it strikes we are so much confounded that we rush like headless chickens to ‘fix’ the ‘problem’. And believing in our superior intelligence, we keep ourselves busy ‘fixing’ problem after problem till we drop dead from such heroic efforts.

Mathematically speaking, while linear behavior of any complex system follows Gaussian distribution; the nonlinear behavior follows some sort of power law. So, it is the mix of the two, never one thing or the other.

By understanding this phenomenon clearly, we can ‘predict’ a black swan or an outlier very easily. It is deceptively simple. Simpler than what we perceive it to be.

It is liberating too. Once the presence of a black swan is detected, much before it actually happens, we are left with enough time on our hands to deal with it effectively and skillfully. It also leaves us with the possibility to dramatically improve the system big time.

The truth is there is no ‘randomness’ anywhere. The concept of  ‘randomness’ is a big illusion at best.

So, then, what are we waiting for to improve our lives?

Learning Complexity — Leadership Series … 3

On By Author of #PowerofSeeIn Education, Management, Nemetics4 Comments

In one of the many tanks in the fish farm lived three big fishes who were great friends. They were Silvy, Goldy and Platy. All the while they were practically moving together and gossiping for hours on end.  They would discuss for hours on the problems of life and how such complex problems might be resolved. They were very intelligent too. They devised an underwater Twitter like system, called ‘Fishter’, to continue their long conversations through the day.

However, they did not seem to have any problem of their own. Food was not a problem since they were well fed by their owners. So there was no need to hunt for food. In the process they forget the art of hunting for food. Life was otherwise comfortable, without a care in the world. Their joy of living was greatly enhanced by the presence of many beautiful female fishes to flirt with. The only regret they ever had was their inability to take a monsoon holiday in the nearby paddy fields as owners quickly drained off excess water fearing loss of fishes from their tanks.

Days passed by merrily with nothing much ado. Only in the evenings they swam up to the surface to pounce upon the tiny tasty morsels gleefully thrown at them by children living in nearby localities.

Then one day, terror struck. Silvy, Goldy and Platy overheard their owners discussing on the banks of their tank, ‘Tomorrow morning we are going to farm this tank for our order from Japan‘. It meant the end of the world for our three friends. They would be caught and kept alive to be sold in the Japanese market. The Japanese are terribly fond of buying fishes live. They abhor buying dead fishes.

On hearing this, Silvy was anxious and nervous. His blood pressure shot up. He began to fret and think of his fate – death at such a young age? He thought it to be unfair of the owners. He thought it unfair of God to have sentenced him to death for no fault of his. Then he thought of all the losses he would incur. He thought of the comfortable life he would lose forever. He thought as to why he was born in this tank. And he blamed his fate. After a few hours his body was slowly turning blue. His breath was labored. And he was light in the head, unable to think any further. His throat ran dry though he gulped gallons of water. And his body shivered. He felt his brains bursting out.

Meanwhile Goldy was adamant. He thought, “All these years we have mastered solving complex problems through our innumerable discussions and dialogs. So we can definitely think about this and solve it. Why not?”

Goldy sat in a corner like Rodin’s ‘Thinker’ and thought of ways and means to save himself and others from the impending danger. He asked himself questions and tried hard to find answers to those. He predicted possible scenarios and how to tackle them if they were to come up. He ‘tweeted’ about this to his world. But he received no response. Everyone seems to be worried about their impending fate of doom. Then Goldy thought of ways to hide from the fishermen or escape from the net. He thought about the tools and techniques that might come in handy during the process. He carefully worked out and rehearsed his designed processes. He also thought of taking up the matter with the ‘Fishes Right Commission’….

But Platy seemed to be himself looking cool and composed without a fish-scale being ruffled. He was a bit philosophical too, ‘Death is inevitable  None can escape the jaws of death. But didn’t Shakespeare say, ‘Cowards die many times before their death’. I shall meet it when it really comes.”

With this nonchalant view he went about with his flirting with young females, drinking, love making, perhaps for the last time and merry making in his usual spontaneous carefree attitude.

Night passed with its uncanny shivering. At the break of dawn fishermen were all round the tank. They cast their net. Finding themselves caught in the net, poor Silvy died of a heart attack. Goldy with his eyes wide open, was still trying to figure out ways from this dreadful situation. His plans were not working. But Platy kept observing what was going on in the most relaxed mood ever, possibly conserving vital energy for the very last moment.

After a few minutes the net was hauled up. The fishermen were delighted by the big catch — all big fishes were neatly rounded up. That meant more money for them. Then they started doing something queer. They started sifting the ones in poor health or the ones that were long dead from the healthier ones.

Platy saw that they put Goldy into a smaller tank which had water in it so that he could be kept alive for the Japanese market. Then he saw them throw out Silvy to the far end of the narrow strip of land that separated two fish tanks.

Immediately, Platy feigned death and lay as still and inflated as possible. Sure the fishermen took him to be long dead and threw him to the far corner of the narrow strip of land. That brought him very close to the other tank. Sensing opportunity he leaped for his dear life and jumped into the adjacent tank as soon as fishermen looked the other way.

There he did not waste a moment further. He gleefully glided towards the beautiful young female fishes.

(An ancient Indian story retold in the modern context)

Frontiers & Challenges of Complexity Discipline

On By Author of #PowerofSeeIn NemeticsLeave a comment

I think that the frontiers and challenges of complexity as a discipline has been very well highlighted by Steven Strogatz in his book Sync on page 287 (Ref 1). Strogatz is a mathematician whom I admire for his intuitive approach to maths, which I believe might make maths popular amongst the masses.

I quote from his book the two relevant paragraphs, which I feel are important.

“…. I don’t want to leave you with a false impression. Sync is just a small part of a much larger body of thought. It is by no means the only approach to the study of complex systems. The chemist Ilya Prigogine and his colleagues feel that the key to unlocking the mysteries of self organization lies in a deeper understanding of thermodynamics. They see the emergence of order as a victories uphill battle against entropy, as a complex system feeds itself on energy flowing in from the environment. The community of physicists interested in pattern formation see fluid mechanics as its paradigm, where the rolling of a turbulent fluid gives rise intermittently gives birth to coherent structures like helices and plumes, rather than degenerating into a bland, uniform smear. The physicist Hermann Haken and his colleagues view the world as a laser, with randomness and positive feedback conspiring to produce the organized forms that occur all around us. Researchers at Santa Fe Institute are struck by the ubiquity of evolution through natural selection, not only in biological population, but in immune systems, economics, and stock markets. Others conceive the universe to be a giant computer, running a cryptic program whose discovery would constitute the end of science.

But for now, these are mostly pipe dreams. We’re still waiting for a major breakthrough in understanding, and it could be a long time in coming. I think we may be missing the conceptual equivalent of calculus, a way of seeing the consequences of the myriad interactions that define a complex system. It could even be that this ultracalculus, if it were handed to us, would be forever beyond human comprehension. We just don’t know.”

That is quite a grim reminder that not only reveals a quick glimpse of the unchallenged frontiers of the discipline of Complexity but also throws at us a challenge at the same time.

But I think a very likely discipline that has been missed out or the practitioners of the discipline hasn’t yet explicitly joined in is Engineering, especially the wing that practically deals with non-linear dynamics — the discipline called Condition Monitoring. Having my roots in that discipline I think that the new maths of complexity is mostly likely to be worked out or generated from this field. It is not that the maths doesn’t exist. One thing that is quite mature in the field of Condition Monitoring is  “prediction of emergence in complex systems”. That is how the field got its name. The prediction, as it should be, is always done in the short term taking into account the ensemble as a whole. Admittedly, most of the maths is graphical. But I don’t see any issue with the graphical maths since it does the job so well indeed, which is a) short term prediction b) understanding and interpreting the interactions at play at any given moment along with interpretation of how new orders are created.

The most interesting thing is that it does not stick to one world-view as most streams, outlined above, have done. It sees the world from multiple perspectives – both Newtonian and Non-Newtonian — waves & vibrations, dynamics and non-linearity, sync & resonance, thermodynamics & fields, flow & fluid mechanics, randomness and feedback, wear and electrons, chaos and evolution, determinism and probabilistic. Undoubtedly it is tall order. A true master in this field (though such masters are difficult to spot since they mostly live like recluses) can easily flow from one perspective to another or hold multiple perspectives together at the same time while observing a phenomenon (I am referring to only one rather secretive school; with a handful of practitioners having not more than 10 masters). There are no inhibitions or ideological hold ups.  That is where the masters draw their strength from.

But the maths that might be created wouldn’t look the same as we are used to. And it must not be so. It must not be ‘calculus’ that can predict the past, present and future for all times to come from a single observation. But at the same time it must not overlook the role of ‘differences’ and ‘integration’. It also must not be so universal that it can be applied to all or many frames of references. It must not look like ‘laws’. However, at the same time it must be simple enough for people to make sense to gain insights that would help them to model, adapt, innovate, re-design and predict. What more is needed?

The good news is that Nemetics (a branch-out from that secretive school of practice) research is rather close to creating that practical maths. At least, as of now, it can predict an emergence in the short term for the most complex of all cases close to 85% accuracy. For relatively simpler cases of complex systems the success of prediction is now close to 100%. And that is quite an achievement.

It is based on interactions of three vital components — Energy, Damping and Constraints, without which no real life complex system or transforming process can exist.

Reference:

1. Sync, Steven Strogatz, Penguin Books, 2003