Abstract: The young science of complexity, which studies systems as diverse as the human body, the earth and the universe, offers novel insights on the question raised in the title. The science of complexity explains large-scale collective behavior, such as well-functioning capitalistic markets, and also predicts that financial crashes and depressions are intrinsic properties resulting from the repeated nonlinear interactions between investors. Applying concepts and methods from complex theory and statistical physics, we have developed mathematical measures to successfully predict the emergence and development of speculative bubbles as well as depressions. This essay attempts to capture and extend the essence of the book with the same title published in January 2003 by Princeton University Press. Recent novelties and live predictions are available at http://www.ess.ucla.edu/faculty/sornette/

Complexity theory: what, why and how?

The history of complexity goes back to antiquity, Greece, China and beyond. Complexity was mostly thought of as being characterized by somehow going beyond what human minds can handle. The idea of complexity as a coherent scientific concept is quite new and dates back to the early 1960s with efforts to define computational complexity in the development of modern computers. The issue of computational complexity arose naturally with the need to measure the resources, time and memory as a function of the size and nature of the input problem to solve. The concept of complexity is also attached to the impossibility theorems of Godel and other mathematicians developed in the 1930s and later dealing with the sizes of axioms for logical theories, and with numbers of ways to satisfy such axioms. The idea that complexity might also be related to information content was also developed with the notion of algorithmic information content as the length of a shortest program to represent a given system. A new wave of interest spurred with the involvement of physicists who discovered in the early 1980s that complexity might offer a general guideline to understand the physical, biological as well as social worlds [Wolfram, 2002].

The study of out-of-equilibrium dynamics (e.g. dynamical phase transitions) and of heterogeneous systems (e.g. glasses, rocks) has progressively made popular in physics and then in its sisters branches (geology, biology, etc.) the concept of complex systems and the importance of systemic approaches: systems with a large number of mutually interacting parts, often open to their environment, self-organize their internal structure and their dynamics with novel and sometimes surprising macroscopic "emergent" properties.

The complex system approach, which involves seeing inter-connections and relationships, i.e., the whole picture as well as the component parts, is nowadays pervasive in modern control of engineering devices and business management. It is also plays an increasing role in most of the scientific disciplines, including biology (biological networks, ecology, evolution, origin of life, immunology, neurobiology, molecular biology, etc), geology (plate-tectonics, earthquakes and volcanoes, erosion and landscapes, climate and weather, environment, etc.), economy and social sciences (including cognition, distributed learning, interacting agents, etc.). There is a growing recognition that progress in most of these disciplines, in many of the pressing issues for our future welfare as well as for the management of our everyday life will need such a systemic complex system and multidisciplinary approach.

A central property of a complex system is the possible occurrence of coherent large-scale collective behaviors with a very rich structure, resulting from the repeated non-linear interactions among its constituents: the whole turns out to be much more than the sum of its parts. It is widely believed that most of these systems are not amenable to mathematical, analytic descriptions and can only be explored by means of "numerical experiments." In the context of the mathematics of algorithmic complexity [Chaitin, 1987], most complex systems are said to be computationally irreducible, i.e., the only way to decide about their evolution is to actually let them evolve in time. Accordingly, the future time evolution of complex systems would be inherently unpredictable. This unpredictability does not prevent however the application of the scientific method for the prediction of novel. In contrast, it refers to the frustration to satisfy the curiosity, strengthened by the anguish and hope that humans have always projected on their future. Is modern science really putting out of reach the grail of predicting (some of) the future evolution of complex systems?

This view has recently been defended persuasively in concrete prediction applications, such as the socially important issue of earthquake prediction (see the contributions in [Nature debate, 1999]. In addition to the persistent failures at reaching a reliable earthquake predictive scheme, this view is rooted theoretically in the analogy between earthquakes and self-organized criticality [Bak, 1996]. In this "fractal" framework, there is no characteristic scale and the power law distribution of sizes reflects the fact that the large earthquakes are nothing but small earthquakes that did not stop. They are thus unpredictable because their nucleation is not different from that of the multitude of small earthquakes which obviously cannot be all predicted.

Does this really hold for all features of complex systems? Take our personal life. We are not really interested in knowing in advance at what time we will go to a given store or drive in a highway. We are much more interested in forecasting the major bifurcations ahead of us, involving the few important things, like health, love and work that count for our happiness. Similarly, predicting the detailed evolution of complex systems has no real value and the fact that we are taught that it is out of reach from a fundamental point of view does not exclude the more interesting possibility to predict phases of evolutions of complex systems that really count.

It turns out that most complex systems around us do exhibit rare and sudden transitions, that occur over time intervals that are short compared to the characteristic time scales of their posterior evolution. Such extreme events express more than anything else the underlying "forces" usually hidden by almost perfect balance and thus provide the potential for a better scientific understanding of complex systems. These crises have fundamental societal impacts and range from large natural catastrophes, catastrophic events of environmental degradation, to the failure of engineering structures, crashes in the stock market, social unrest leading to large-scale strikes and upheaval, economic drawdowns on national and global scales, regional power blackouts, traffic gridlock, diseases and epidemics, etc. It is essential to realize that the long-term behavior of these complex systems is often controlled in large part by these rare catastrophic events: the universe was probably born during an extreme explosion (the "big-bang"); the nucleosynthesis of most important atomic elements constituting our matter results from the colossal explosion of supernovae; the largest earthquake in California repeating about once every two centuries accounts for a significant fraction of the total tectonic deformation; landscapes are more shaped by the "millennium" flood that moves large boulders rather than the action of all other eroding agents; the largest volcanic eruptions lead to major topographic changes as well as severe climatic disruptions; evolution is characterized by phases of quasi-statis interrupted by episodic bursts of activity and destruction; financial crashes can destroy in an instant trillions of dollars; political crises and revolutions shape the long-term geopolitical landscape; even our personal life is shaped on the long run by a few key decisions and happenings.

The outstanding scientific question is thus how such large-scale patterns of catastrophic nature might evolve from a series of interactions on the smallest and increasingly larger scales. In complex systems, it has been found that the organization of spatial and temporal correlations do not stem, in general, from a nucleation phase diffusing across the system. It results rather from a progressive and more global cooperative process occurring over the whole system by repetitive interactions. An instance would be the many occurrences of simultaneous scientific and technical discoveries signaling the global nature of the maturing process.

Standard models and simulations of scenarios of extreme events are subject to numerous sources of error, each of which may have a negative impact on the validity of the predictions [Karplus, 1992]. Some of the uncertainties are under control in the modeling process; they usually involve trade-offs between a more faithful description and manageable calculations. Other sources of errors are beyond control as they are inherent in the modeling methodology of the specific disciplines. The two known strategies for modeling are both limited in this respect: analytical theoretical predictions are out of reach for most complex problems. Brute force numerical resolution of the equations (when they are known) or of scenarios is reliable in the "center of the distribution," i.e., in the regime far from the extremes where good statistics can be accumulated. Crises are extreme events that occur rarely, albeit with extraordinary impact, and are thus completely under-sampled and thus poorly constrained. Even the introduction of teraflop (or even pentaflops in the futur) supercomputers does not change qualitatively this fundamental limitation.

Recent developments suggest that non-traditional approaches, based on the concepts and methods of statistical and nonlinear physics coupled with ideas and tools from computation intelligence could provide novel methods in complexity to direct the numerical resolution of more realistic models and the identification of relevant signatures of impending catastrophes. Enriching the concept of self-organizing criticality, the predictability of crises would then rely on the fact that they are fundamentally outliers, e.g. large earthquakes are not scaled-up versions of small earthquakes but the result of specific collective amplifying mechanisms. Similarly, financial crashes do not belong to the same distribution as smaller market moves [Johansen and Sornette, 2002]. To address this challenge posed by the identification and modeling of such outliers, the available theoretical tools comprise in particular bifurcation and catastrophe theories, dynamical critical phenomena and the renormalization group, nonlinear dynamical systems, and the theory of partially (spontaneously or not) broken symmetries. Some encouraging results have been gathered on concrete problems, such as the prediction of the failure of complex engineering structures, the detection of precursors to stock market crashes and of human parturition, with exciting potential for earthquakes [Sornette, 2002].

We now proceed to present how these ideas have been explored and exploited in the financial and social spheres.

Questions and lessons from the stock Market Crach of October 1987

From the opening on October 14, 1987 through the market close on October 19, major indexes of market valuation in the United States declined by 30 percent or more. Furthermore, all major world markets declined substantially in the month, which is itself an exceptional fact that contrasts with the usual modest correlations of returns across countries and the fact that stock markets around the world are amazingly diverse in their organization (Barro et al., 1989, White, 1996).

The crash of October, 1987 and its black monday on October, 19 remains one of the most striking drops ever seen on stock markets, both by its overwhelming amplitude and its encompassing sweep over most markets worldwide. It was preceded by a remarkably strong "bull" regime epitomized by the following quote from Wall Street Journal, August 26, 1987, the day after the 1987 market peak: "In a market like this, every story is a positive one. Any news is good news. It's pretty much taken for granted now that the market is going to go up." This and other indicators, such as the implied volatility, indicate that investors were thus mostly unaware of the forthcoming risk happenings (Grant, 1990).

A lot of work has been carried out to unravel the origin(s) of the crash, notably in the properties of trading and the structure of markets; however, no clear cause has been singled out. It is noteworthy that the strong market decline during October 1987 followed what for many countries had been an unprecedented market increase during the first nine months of the year and even before. In the US market, for instance, stock prices advanced 31.4 % over those nine months. Some commentators have suggested that the real cause of October's decline was that over-inflated prices generated a speculative bubble during the earlier period. The main explanations that have been invoked include computer trading, derivative securities, lack of liquidity, trade and budget deficits, overvaluation, etc. Other cited potential causes involve the auction system itself, the presence or absence of limits on price movements, regulated margin requirements, off-market and off-hours trading (continuous auction and automated quotations), the presence or absence of floor brokers who conduct trades but are not permitted to invest on their own account, the extent of trading in the cash market versus the forward market, the identity of traders (i.e. institutions such as banks or specialized trading firms), the significance of transaction taxes, etc.

More rigorous and systematic analyses on univariate associations and multiple regressions of these various factors conclude that it is not at all clear what caused the crash (Barro et al., 1989). The most precise statement, albeit somewhat self-referential, is that the most statistically significant explanatory variable in the October crash can be ascribed to the normal response of each country's stock market to a worldwide market motion. A world market index was thus constructed by equally weighting the local currency indexes of the 23 major industrial countries mentioned above and normalized to 100 on september 30, 1987 (Barro et al., 1989). It fell to 73.6 by October 30. The important result is that it was found to be statistically related to monthly returns in every country during the period from the beginning of 1981 until the month before the crash, albeit with a wildly varying magnitude of the responses across countries. This correlation was found to swamp the influence of the institutional market characteristics. This signals the possible existence of a subtle but nonetheless influential world-wide cooperativity (contagion) at times preceding crashes.

According to this view, the crash of a given country's market is "explained" as driven by or responding to a world-wide market crash. But what is the origin of the world market crash itself? Is it the result of the superposition of all individual national crashes? This line of argument is reminiscent of the infamous chicken-and-egg problem and we believe that it does not reply adequately to the question posed in the title.

Questions and lessons from the stock Market Crach of March-April 2002 on the Nasdaq index

With the low of 3227 on April, 17, 2000, the Nasdaq Composite Index lost over 37% of its all-time high of 5133 reached on the 10th of March 2000. The Nasdaq Composite consists mainly of stock related to the so-called "New Economy," i.e., the Internet, software, computer hardware, telecommunication and so on. A main characteristic of these companies is that their price-earning-ratios (P/E's), and even more so their price-dividend-ratios, often came in three digits . Opposed to this, so-called "Old Economy" companies, such as Ford, General Motors and Daimler-Chrysler, had P/E's of the order of 10. The difference between Old Economy and New Economy stocks was thus the expectation of future earnings: investors expected an enormous increase in for example the sale of Internet and computer related products rather than in car sales and were hence more willing to invest in Cisco rather than in Ford notwithstanding the fact that the earning-per-share of the former is much smaller than for the later.

In the standard fundamental valuation formula, in which the expected return of a company is the sum of the dividend return and of the growth rate, New Economy companies are supposed to compensate for their lack of present earnings by a fantastic potential growth. In essence, this means that the bull market observed in the Nasdaq until the end of the first quarter of 2000 was fueled by expectations of increasing future earnings rather than economic fundamentals: the price-to-dividend ratio for a company such as Lucent Technologies (LU) with a capitalization of over 300 billions prior to its crash on the 5 January 2000 was over 900 which means that you got a higher return on your checking account(!) unless the price of the stock increases. Opposed to this, an Old Economy company such as DaimlerChrysler gives a return which is more than thirty times higher. Nevertheless, the shares of Lucent Technologies rose by more than 40% during 1999 whereas the share of DaimlerChrysler declined by more than 40% in the same period.

The generic scenario

This makes clear that it is the expectation of future earnings rather than present economic reality that motivates the average investor, with the potential for the creation of a speculative bubble. History provides many examples of bubbles driven by unrealistic expectations of future earnings followed by crashes. The same basic ingredients are found repeatedly. Markets go through a series of stages, beginning with a market or sector that is successful, with strong fundamentals. Credit expands, and money flows more easily. (Near the peak of Japan's bubble in 1990, Japan's banks were lending money for real estate purchases at more than the value of the property, expecting the value to rise quickly.) As more money is available, prices rise. More investors are drawn in, and expectations for quick profits rise. The bubble expands, and then bursts. In other words, fueled by initially well-founded economic fundamentals, investors develop a self-fulfilling enthusiasm by an imitative process or crowd behavior that leads to the building of castles in the air, to paraphrase Malkiel (1990).

Furthermore, the causes of the crashes on the US markets in 1929, 1987, 1998 and in 2000 belongs to the same category, the difference being mainly in which sector the bubble was created: in 1929, it was utilities; in 1987, the bubble was supported by a general deregulation of the market with many new private investors entering the market with very high expectations with respect to the profit they would make; in 1998, it was an enormous expectation to the investment opportunities in Russia that collapsed; before 2000, it was the extremely high expectations to the Internet, telecommunication etc. that fueled the bubble.

Positive feedback, collective behaviors and herding

In a culmination of more than ten years of research on the science of complex system, we have thus challenged the standard economic view that stock markets are both efficient and unpredictable. The main concepts that are needed to understand stock markets are imitation, herding, self-organized cooperativity and positive feedbacks, leading to the development of endogenous instabilities. According to this theory, local effects such as interest raises, new tax laws, new regulations and so on, invoked as the cause of the burst of a given bubble leading to a crash, are only one of the triggering factors but not the fundamental cause of the bubble collapse. We propose that the true origin of a bubble and of its collapse lies in the unsustainable pace of stock market price growth. As a speculative bubble develops, it becomes more and more unstable and very susceptible to any disturbance.

Large stock market crashes are thus the social analogs of so-called critical points studied in the statistical physics community in relation to magnetism, melting, and other phase transformation of solids, liquids and gas (Sornette, 2000). This theory is based on the existence of an (unwilling) cooperative behavior of traders imitating each other which leads to progressively increasing build-up of market cooperativity, or effective interactions between investors, often translated into accelerating ascent of the market price over months and years before the crash. According to this theory, the specific manner by which prices collapsed is not the most important problem: a crash occurs because the market has entered an unstable phase and any small disturbance or process may have triggered the instability.

Think of a ruler held up vertically on your finger: this very unstable position will lead eventually to its collapse, as a result of a small (or absence of adequate) motion of your hand or due to any tiny whiff of air. The collapse is fundamentally due to the unstable position; the instantaneous cause of the collapse is secondary. In the same vein, the growth of the sensitivity and the growing instability of the market close to such a critical point might explain why attempts to unravel the local origin of the crash have been so diverse. Essentially, anything would work once the system is ripe. In this view, a crash has fundamentally an endogenous, or internal origin and exogenous, or external shocks only serve as triggering factors.

Recent academic research in the field of complex systems suggest that the economy as well as stock markets self-organize under the competing influences of positive and negative feedback mechanisms. Positive feedbacks, i.e., self-reinforcement, refer for instance to the fact that, conditioned on the observation that the market has recently moved up (respectively down), this makes it more probable to keep it moving up (respectively down), so that a large cumulative move may ensue. "Positive feedback" is the opposite of "negative feedback," the latter being a concept well-known for instance in population dynamics in the presence of scarce resources. Rational markets and stable economic equilibrium derive from the forces of negative feedback. When positive feedback forces dominate, deviations from equilibrium lead to crises. Such instabilities can be seen as intrinsic endogenous progenies of the dynamical organization of the system.

Positive feedbacks lead to collective behavior, such as herding in sells during a financial crash. This collective behavior does not require the coordination of people to take the same action but results from the convergence of selfish interests together with the impact of interactions between people through the complex network of their acquaintances. Complex system theory tells us that such collective behaviors may be very robust against external intervention, as along as the selfish individualistic nature of individual so-called utility function dominates. The collective is robust because it derives from a bottom-up mechanism. Similar resilience is observed for instance in the Internet for instance due to its delocalized structure and self-organization.

As a consequence, the origin of crashes is much more subtle than often thought, as it is constructed progressively by the market as a whole, as a self-organizing process. In this sense, the true cause of a crash could be termed a systemic instability. This leads to the possibility that the market anticipates the crash in a subtle self-organized and cooperative fashion, hence releasing precursory "fingerprints" observable in the stock market prices (Sornette and Johansen, 2001; Sornette, 2003). These "fingerprints" have been modeled by "log-periodically decorated power laws" (LPPL), which are beautiful mathematical patterns associated with the mathematical generalization of the notion of fractals to complex imaginary dimensions (Sornette, 1998). We refer to the book (Sornette, 2003) for a detailed description and the review of many empirical tests and of several forward predictions. In particular, we predicted in January 1999 that Japan's Nikkei index would rise 50 percent by the end of that year, at a time when other economic forecasters expected the Nikkei to continue to fall, and when Japan's economic indicators were declining. The Nikkei rose more than 49 percent during that time. We also successfully predicted several short-term changes of trends in the US market and in the Nikkei. Or course, we are not able to predict stock markets with anything close to 100 percent accuracy - just as weather forecasting cannot say with absolute certainty what the weekend weather will be - but our predictions will become more accurate as we refine our methods.

Our theory of collective behavior predicts robust signatures of speculative phases of financial markets, both in accelerating bubbles and decreasing prices (see below). These precursory patterns have been documented for essentially all crashes on developed as well as emergent stock markets. Accordingly, the crash of October, 1987 is not unique but a representative of an important class of market behavior, underlying also the crash of October 1929 (Galbraith, 1997) and many others (Kindleberger, 2000; Sornette, 2003).

Stock market crashes are often unforeseen by most people, especially economists. One reason why predicting complex systems is difficult is that we have to look at the forest rather than the trees, and almost nobody does that. Our approach tries to avoid that trap. From the tulip mania, where tulips worth tens of thousands of dollars in present U.S. dollars became worthless a few months later, to the U.S. bubble in 2000, the same patterns occur over the centuries. Today we have electronic commerce, but fear and greed remain the same. Humans remained endowed with basically the same qualities today as they were in the 17th century.

It is often thought that the efficiency of the market acquired by the diligence of greedy investors makes impossible the existence of predictability: as soon as a pattern is detected, if profitable, it should disappear by the action of arbitragers. This common wisdom is correct for most patterns. However, there are some structures, such as the LPPL, which result from positive feedbacks. Hence, the more people believe in such patterns, the more their action will be in line and will reinforce them. This idea is incorporated in our theory of rational expectation bubbles [Johansen et al., 1999; 2000].

"Antibubbles" in Japan and the US and World markets

Imitation between investors and their herding behavior not only lead to speculative bubbles with accelerating overvaluations of financial markets possibly followed by crashes, but also to "antibubbles" with decelerating market devaluations following market peaks (Johansen and Sornette, 1999), that can be modeled by the same type of "log-periodically decorated power law" decay found for accelerating bubbles. There is thus a certain degree of symmetry between the speculative behavior of the "bull" and "bear" market regimes. The concept of an "anti-bubble", that we coined to stress the fact that positive feedbacks are also at work in decreasing markets, is an adaptation of the concept of anti-particle in particle physics, such as the positron which is the positively charged particle exactly symmetric to the negatively charged electron. This concept stresses the symmetry between bubbles and anti-bubbles.

We have studied in details several anti-bubbles, the prominent example being the Japanese Nikkei stock index since January 1, 1990. Other examples include gold after1980. Both after their all-time highs (Johansen and Sornette, 1999). The Japanese antibubble from 1990 to present is all the more interesting because we published a prediction in January 1999 of the behavior of the Japanese stock market in the following two years that have been remarkably successful (Johansen and Sornette, 2000; Sornette, 2003). The fulfillment of this prediction is quite remarkable because it included a change of trend: at the time when the prediction was issued, the market was declining and showed no tendency to increase. Many economists were at that time very pessimistic and could not envision when Japan and its market would rebound. Not only did we correctly predicted a rebound of 50% for the end of 1999 but they also foresaw another change of trend at the beginning of 2000. The approval in Oct., 1998 by the Japanese parliament of legislation to allow the government to nationalize failing banks and to commit more than $US 500 billion to rescue the nation's banking system led to a short revival of Japan's economy however bought at the expense of more than $1US trillion in government spending in a series of economic stimulus packages that included numerous public works projects. Sornette (2003) develops further this question and reports all cases, successes and failures of past predictions in several different markets.

The situation of Japan in 1992 is no more very different from that of the US after the burst of the "new economy" bubble in March-April 2000 and the cascade of discoveries (which will probably never be fully unveiled in their full extent) of creative accounting of companies striving to look good in the eyes of analysts rather than to build strong fundamentals. The growing appreciation in 2002 of the crisis in the American financial system is reminiscent of the starting point of Japan's massive financial bubble burst more than 10 years before and of the inter-twinning of the bad debts and bad performance of banks whose capital is invested in the shares of other banks, thus creating the potential for a catastrophic cascade of bankruptcies. Japan has rediscovered before the US the faults of the 19th century financial system in the US in which stock markets were so much intertwined with their overall banking financial system, that busts and bursts occurred more than once every decade, with firms losing their credit lines and workers and consumers their savings and often their employment. It is often said that the 1930s depression was the last of the stock market and bank-induced economic collapses. The growing fuzziness between financial banking systems and stock markets, in part due to the innovations in information technology, has re-created the climate for stronger bubbles.

The big problem is that, in the collapse following them, policy interventions such as lowering interest rates, reducing taxes, and government spending packages may be much less effective, due to several mechanism such as the so-called liquidity trap, a process in which government and the central bank policy becomes essentially useless due to an effective vanishing short-term interest rate, or due to lack of consumer confidence who reduce their consumption and spending. Often forgotten within macroeconomics policies, the human aspect of the problem has to be fully appreciated: for instance, how to restore the confidence of Japanese households into a brighter future so that they resume spending and innovating even more rather than saving too much. Saving is a natural reaction to losses but may accentuate the problem by the process of "positive feedback." We argue that standard macro-economic reasoning will not be sufficient as long as one forgets the possible stable and unstable regime shifts resulting from the emergence of collective behaviors of imitation and herding, which themselves emphasize a strongly non-linear dynamical view point in order to understand economies and stock markets.

Recently, we have applied our theory and its derived methodology to the US since the burst of the last 'new economy' bubbles in 2000 (Sornette and Zhou, 2002) as well as to many other western and emergent markets in the world (Zhou and Sornette, 2002). We find the same characteristic signature of an anti-bubble regime that started almost synchronously in most westerm markets in August 2000. As of September 2002, we also issued a prediction for the next two years for the US stock market: the S&P500 US index should continue its up-trend for no more than a few months and then resume a descent extending well in the first semester of 2004 with an amplitude of more than 20%. This prediction can be considered to be refinement of a longer term analysis combining three pieces of empirical evidence (Johansen and Sornette, 2001), namely human population, gross national product worldwide and stock market indices, which suggest all together a fundamental turning point in the growth of the economic impact of mankind in the decades ahead of us. In addition, a prediction is made that starting around 1999, a 5 to 10 years consolidation of international stock markets will occur, allowing a purge after the over-aggressive appetite of the preceding decade (Sornette, 2003). Since 2000, this prediction has been born out.

Generalization to other markets: Is a Real Estate Bubble Ready to Burst?

Following the collapse of the "new economy" bubble of 2000, the Federal Reserve aggressively lowered its discount rate from 6.5 to 1.25 percent in less than two years in an attempt to coax a stronger recovery of the U.S. economy. But, there is growing apprehension that this rate reduction is creating a new bubble in real estate, as historically low mortgage rates fuel strong housing demand. Are we going from Charybdis to Scylla? This question is all the more excruciating at a time when many other indicators suggest a significant deflationary risk.

The young science of complexity, which studies systems as diverse as the human body , the earth and the universe, offers novel insights on this troubling question. As we already mentioned, this approach led us to predict the recovery of the Japanese Nikkei in 1999 by 50 percent, to detect a speculative "anti-bubble" in the U.S. stock market and worldwide since the summer of 2000 with a degree of synchronicity never observed before and, recently, to predict that the U.S. stock market will continue to weaken until the summer of 2004.

Stock market losses have destroyed as much as $5 trillion in investor wealth since the market's peak in 2000. Fortunately, this has led to relatively minor effects on the economy: the gross domestic product (GDP) exhibited a descent of about one-half percent, a drop that would have been far worse without a strong real estate sector.

While the economy has generally been contracting in the last two years, real estate has been growing: house prices have been rising at a rate of about 2 per cent a year faster than income gains. Real consumer outlays and spending on residential construction each rose about 3 percent during 2001. One of the reasons for the relatively minor impact of the stock market losses may be found in the offsetting effect in the real estate market. Home equity has gained about $1.7 trillion in the same period, according to the chief economist for the biggest U.S. mortgage firm, Fannie Mae. Since, according to the Federal Reserve, home values have twice the impact on consumer spending that stock values have via the "richness effect,"' the housing boom has offset almost two-thirds of the stock market losses on the economy.

What is the risk for a real estate crash? Federal Reserve Chairman A. Greenspan and Governor D.L. Kohn dismissed recently the possibility of a crash and do not see any problem with the current real estate boom. Many others believe they have detected a real estate bubble in the U.S. Statistics released every month confirm that "the housing sector continues to defy all odds," in the words of the chief economist for the National Association of Realtors, David Lereah. American mortgages are on the path of becoming the single largest class of fixed income securities on the planet. Total mortgage debt outstanding has risen sharply during the last decade. While the total was about $2.7 trillion in the first quarter of 1990, by the fourth quarter of 1999, it had almost doubled, to $5.2 trillion. As a comparison, the total amount of cumulative borrowing by the Federal Treasury, the national debt, was about $5.7 trillion in August 2000.

Add to these elements that the demand for mortgage borrowing outstrips aggregate domestic saving which is currently negative and has reached the lowest level since record keeping began in 1959. This negative saving rate combined with the continuing rapid growth of mortgage borrowing implies that there must be a reduction in non-mortgage lending or an increase in fund flows from abroad or both. This may lead to increased instability through globalization, resulting from the behavior of international investors.

To make matters even worse, the real estate bubble is part of a general huge credit "bubble" that has developed steadily over recent decades, which includes the various U.S. federal money supply, and the personal, municipal, corporate and federal debts (estimated by some to add up to as much as several tens of trillions of dollars), which may not only drag down the recovery of the economy but also lead to vulnerability to exogenous crises.

What is the risk of a real estate crash according to the science of complexity? As we have already mentioned, recent research in the field of complex systems suggest that the economy, as well as stock markets, self-organize under the competing influences of positive and negative feedback mechanisms, such as momentum investing in stock markets. Positive feedbacks lead to such collective behavior as herding in buys during the growth of bubbles and sells during a crash. Using this theory and its specification in the mathematics of fractals, we have been searching for specific mathematical signatures of bubbles (Zhou and Sornette, 2003). Speculative bubbles are observed in all assets at all times and locations in history, from the tulip mania in Holland culminating in 1636, to stocks, commodities, currency and real estate markets, in the past and present. Our experience suggests that speculative bubbles have a rather long characteristic gestation time, typically years.

Our analysis finds that the US real estate market is still far from an instability and that there are no significant risks for a crash this year. The situation is the opposite for the U.K. housing market as two unambiguous signatures show that an unsustainable bubble started years even before the end of the stock market bubble in 2000. These signatures have been found to be reliable predictors of past crashes in financial markets.

The analysis points to the end of the bubble for the U.K. housing market no later than the end of the year, with either a crash or a strong change of direction in the UK housing market. While there are very strong correlations between stock markets in developed countries at present, no such correlation has yet materialized in real estate markets. In the longer term, however, investors should remain watchful for indications of a possible spread to the U.S. real estate market. Such signs would include an increase of correlation between real estate markets and the growth of patterns similar to those found for the UK real estate market.

Limitations of the log-periodic power law (LPPL) model

There are several important limitations to the predictability of markets, when using the LPPL theory of herding.

(1) In order to be consistent with the self-correcting nature of markets, crashes cannot be deterministically predictable but must contain stochastic components. In our models, this is taken into account by realizing that we are detecting only the growth or decay of a bubble, its climax, but this culmination is not necessarily the time of the crash. The end of a bubble is the time when the crash is the most probable. But a bubble may end in other ways than by crashing, for instance by a smooth change of regime. There is always a non-zero probability that the crash will not occur at all, a possibility that rationalizes why investors may remain invested at times when markets grow in unsustainable ways.

(2) A recent study combining ideas from critical phenomena, the impact of agents' expectation, multi-scale analysis and the mathematical method of pattern recognition of sparse data shows that the LPPL model detects more specifically large changes of regimes rather than crashes per se [Sornette and Zhou, 2003]. This is a problem for practical applications dealing with hedging.

(3) Using the LPPL model, Johansen and Sornette [2003] have performed a systematic classification of drawdowns in the two leading exchange markets (US dollar against the Deutsmark and against the Yen), in the major world stock markets, in the U.S. and Japanese bond market and in the gold market. They find that, out of 49 significant outliers, 25 can be classified as endogenous (that is, predictable with the LPPL theory), 22 as exogenous and 2 as associated with the Japanese anti-bubble. Restricting to the world market indices, they find 31 outliers, of which 19 are endogenous, 10 are exogenous and 2 are associated with the Japanese anti-bubble. The existence of exogenous crashes, that is, genuine surprises that can move the market significantly, leads to an intrinsic limitation of the predictability of crashes. This seems to be the unavoidable lot of complex systems that are open to the outside, i.e., that are subjected to a complicated flux of "news." The search for general and systematic differences in the response of the stock market, for instance in the volatility of prices, to endogenous versus exogenous shocks is described in [Sornette and Helmstetter, 2003; Sornette et al., 2003].

Deflationary risk and a top-down extension of the collective behavior in markets

A deflationary risk is looming over the US. Deflation could take huge proportions, last years and cost a lot in terms of quality of life for many. With the unprecedented debt levels of the US [Godley, 2003], it is probable that the familiar monetary and fiscal remedies would fail. Indeed, a big problem is that policy interventions such as lowering interest rates, reducing taxes, government spending packages (including lavish war expenditures) and any measure to restore investors' confidence may be much less effective than expected, as discovered with the Japanese so-called liquidity trap, which is as we said a process in which government and the central bank policy becomes essentially useless. In addition, loss of confidence by investors may lead to a non-negligible cost to the overall economy, providing a positive feedback reinforcing the bearish climate.

We suggest that what is needed to avoid or get out of deflation is for individual countries to abandon the selfish policy of "Everyone for himself" that is bound to blossom even further as the hardships unfold (such as trade wars, protectionist backlash, dollar depreciation) in favor of a new approach where the well-being of countries is considered collectively: the US economic problems are the problems of the rest of the world (this is the implicit or explicit standard US view point) but the economic problems of the rest of the world are also US problems. Beyond lip service, this is a "new" view point in the following sense: the US economic problems have no solution outside a process in which the benefits (and drawbacks) attributed to policies followed by country A incorporate also the benefits (and drawbacks) to the economy of country B, and vice versa. Incorporating such an (apparently) unselfish component in the assessment of policies seems like an hopeless chimera, in view for instance of the difficulties of the European countries trying to achieve just that. We contend that there are no other ways.

The argument is based again on the science of complexity, which studies the emergence of organization in complex systems as diverse as the human body (biology), the earth (geology) or the cosmos (astrophysics). This bottom-up mechanism explains the robustness and strength of modern developed economies as well as their vulnerability to endogenous instabilities. The theory of complex systems thus explains the origin of Adam Smith's invisible hand in society according to which a collection of selfish self-centered individuals (or countries) coldly maximizing their individual "utility functions" achieve an optimal aggregate social welfare. This theory explains capitalism and free trade. However, it also explains and predicts the occurrence of instabilities and of far-from-optimal equilibrium situations, which are inherent in the bottom-up self-organization (Sornette, 2003).

This problem is also linked to Arrow's impossibility theorem for aggregating individual preferences [Gaertner, 2001]. What Kenneth Arrow was able to prove mathematically is that there is no method for constructing social preferences from arbitrary individual preferences. In other words, there is no rule, majority voting or otherwise, for establishing social preferences from arbitrary individual preferences. In simple words, a consistent policy for the world, which is agreeable to all parties, would not be possible. There is however one way out of this impasse for making social decisions through the political process. If the individual preferences have some commonality, then social preferences can be constructed. This understanding suggests its remedy: a group approach, or in other words a kind of top-down approach, recognizing the necessity of extending the standard economic goal of individuals' utility maximization to the reality of a "social capital" both within and across countries. Rather than focusing on individual utilities, this could be term the group utility of society. Recent works studying the behavior of groups, in particular the existence of altruistic behaviors, suggest the existence of such group utility [Fehr and Gachter, 2002].

Famous economists such as P. Krugman and J.E. Stiglitz and financier G. Soros have also emphasized the limits of free markets and the need for well-thought regulations and interventions. This chorus the spirit of J.M. Keynes who stressed that, at times of crises, there is a need for governments to develop a world-conscious solidarity extending beyond the selfish interests of each individual country. The science of complex systems provides a novel twist to these analyses by suggesting a deeper justification that extends the bottom-up approach to a more fundamental understanding of our societies incorporating the utility of groups at many levels of interactions.

The transition from selfish to group utility will not be done easily and gleefully but the torments ahead may force us to realize this is our only long-term solution. This may actually be an opportunity to transition towards a society with a better balance between freedom and group welfare.

References:

Bak, P. (1996) How nature works: the science of self-organized criticality (York, NY, USA: Copernicus).

Barro, R.J., E.F. Fama, D.R. Fischel, A.H. Meltzer, R. Roll and L.G. Telser (1989) Black monday and the future of financial markets, edited by R.W. Kamphuis, Jr., R.C. Kormendi and J.W.H. Watson (Mid American Institute for Public Policy Research, Inc. and Dow Jones-Irwin, Inc.).

Chaitin, G.J. (1987) Algorithmic information theory (Cambridge, New York: Cambridge University Press).

Fehr, E. and S. Gachter (2002) Altruistic punishment in humans, Nature,415, 137-140.

Gaertner, W (2001) Domain Conditions in Social Choice Theory (Cambridge University Press, Cambridge, UK)

Galbraith, J.K. (1997) The great crash, 1929 (Boston : Houghton Mifflin Co.).

Godley, W. (2003) The US economy; a changing strategic predicament, working paper of the Levy Economics Institute, February 15, http://www.levy.org/docs/stratan/stratpred.html

Grant, J.L. (1990) Stock Return Volatility During the Crash of 1987, Journal of Portfolio Management 16, 69-71.

Johansen, A., O. Ledoit and D. Sornette (2000) Crashes as critical points, International Journal of Theoretical and Applied Finance 3 (2), 219-255.

Johansen A. and Sornette D. (1999) Financial "anti-bubbles": log-periodicity in Gold and Nikkei collapses, Int. J. Mod. Phys. C 10, 563-575.

Johansen, A. and Sornette, D. (2000) Evaluation of the quantitative prediction of a trend reversal on the Japanese stock market in 1999, Int. J. Mod. Phys. C 11, 359-364.

Johansen, A. and D. Sornette (2001) Finite-time singularity in the dynamics of the world population and economic indices, Physica A 294, 465-502.

Johansen, A. and D. Sornette (2002) Large Stock Market Price Drawdowns Are Outliers, Journal of Risk 4(2), 69-110.

Johansen, A. and D. Sornette (2003) Endogenous versus Exogenous Crashes in Financial Markets, submitted to Quantitative Finance (http://arXiv.org/abs/cond-mat/0210509)

Johansen, A., D. Sornette and O. Ledoit (1999) Predicting Financial Crashes using discrete scale invariance, Journal of Risk, 1 (4), 5-32.

Karplus, W.J. (1992) The Heavens are falling: the scientific prediction of catastrophes in our time, (Perseus Publishing).

Kindleberger, C.P. (2000) Manias, panics, and crashes: a history of financial crises (4th ed. New York: Wiley).

Malkiel B.G. (1990) A random walk down Wall Street (WW Norton & Company: New York).

Nature debate (1999) Is the reliable prediction of individual earthquakes a realistic scientific goal? http://helix.nature.com/debates/earthquake/

Sornette, D. (1998) Discrete scale invariance and complex dimensions, Physics Reports, 297, 239-270.

Sornette, D. (2000) Critical Phenomena in Natural Sciences, (Springer Series in Synergetics, Heidelberg).

Sornette, D. (2002) Predictability of catastrophic events: material rupture, earthquakes, turbulence, financial crashes and human birth, Proceedings of the National Academy of Sciences USA, V99, 2522-2529 (2002).

Sornette, D. (2003) Why Stock Markets Crash? Critical Events in Complex Financial Systems, (Princeton University Press, Princeton, NJ)

Sornette, D. and A. Helmstetter (2003) Endogeneous versus exogeneous shocks in systems with memory, Physica A 318 (3-4), 577-591.

Sornette, D. and A. Johansen (2001) Significance of log-periodic precursors to financial crashes, Quantitative Finance 1, (4), 452-471.

Sornette, D., Y. Malevergne and J.F. Muzy (2003) What causes crashes? Risk, 16 (2), 67-71.

Sornette, D. and W.-X. Zhou (2002) The US 2000-2002 Market Descent: How Much Longer and Deeper? Quantitative Finance 2 (6), 468-481.

Sornette, D. and W.-X. Zhou (2003) Predictability of Large Future Changes in Complex Systems, Submitted to Phys. Rev. Lett. (http://arXiv.org/abs/cond-mat/0304601)

White E.N. (1996) Stock market crashes and speculative manias. In The international library of macroeconomic and financial history 13. An Elgar Reference Collection, Cheltenham, UK; Brookfield, US.

Wolfram, S. (2002) A New Kind of Science (Wolfram Media, 2002).

Zhou, W.-X. and D. Sornette (2002) Evidence of a Worldwide Stock Market Log-Periodic Anti-Bubble Since mid-2000, submitted to Physica A (e-print at http://arXiv.org/abs/cond-mat/0212010).

Zhou, W.-X. and D. Sornette (2003) 2000-2003 Real Estate Bubble in the UK but not in the USA, in press in Physica A (e-print at http://arXiv.org/abs/physics/0303028)