How the Western banking system came close to total collapse is not hard to explain. Tight credit conditions turned into full-blown financial panic when Lehman Brothers, the US investment bank, failed in September 2008. Banks suddenly ceased to have access to funding because the interbank lending market froze up. It required governments to rescue the banking system with huge sums of taxpayers’ money. Why this happened is less clear. My thesis is that there were incentives throughout the financial system for various actors to behave rationally by their own lights but with terrible consequences for financial stability.

It is worth first recounting the mechanics of the crisis, because there is a danger of focusing too much on Lehman’s. When Lehman’s collapsed, the reasoning behind the decision to let it fail was far from obviously misguided. It was a calculated risk. The notion that every bank was so inextricably linked with every other bank that it needed to be rescued committed the taxpayer to open-ended support. Lehman’s collapse was entirely attributable to the hubris of one man: Dick Fuld, the bank’s chief executive since 1994. He could have arranged a deal with Korea Development Bank; but he insisted on an impossible price, and his negotiating partner walked away.

There was no reason that Lehman’s should have suffered that fate. It had been a good business, succeeding consistently in three different areas: investment banking, fixed income and asset management. That model was essentially abandoned by Lehman’s in the great credit explosion of the first decade of the 21st century. Fuld expanded into structured products and committed Lehman’s (unusually for an investment bank) to a massive exposure to commercial property. He did the damage; it was reasonable to let the bank take the consequences.

In practice, Lehman’s collapse precipitated financial panic. But if it had not been Lehman’s, it would have been something else that sparked a transition from credit crunch to chaos. Northern Rock in the UK was a terrible augury: what had previously been a mortgage lender with its roots in regional friendly societies essentially became a financial engineering shop, borrowing money short-term in the wholesale market and lending it long-term as mortgage finance. When the interbank lending market froze, the Rock could not meet its commitments.

This was a scenario that no one had foreseen. There had been doomsayers, whose warning were ignored, but they had seen the risk of financial instability in hedge funds rather than in banks. Yet in retrospect the risk ought to have been obvious. There is always a theoretical risk of runs on the financial system, because banks hold only a small proportion of their assets in cash and liquid securities. Even a solvent institution might suddenly suffer a run because of the illiquidity of its assets. The banking crisis of 2007-09 was partly about liquidity and partly about solvency – the banks had no idea what their assets were worth, because there was no market for them.

Bank assets are always illiquid to some degree, because by definition banks know less about the purposes to which a loan will be put than the borrower does. A bank that extends a loan, conversely, knows more about the creditworthiness of a borrower than a third party does. In the jargon, there are information asymmetries throughout the financial system, and these contributed in the crisis to the collapse of interbank lending. It is those asymmetries that make the financial system incipiently unstable. That risk was realised in the banking crisis of 2007-09. Banks were worried that they would not get their money back. This happened because institutions had incentives to behave in ways that were individually rational but collectively disastrous. While governments were at fault in various ways – not least in lax regulation – this was essentially a failure born in the private sector, and it required government action to stem the panic and stabilise the financial system.

These disastrous actions were that banks ramped up their risk-taking regardless of the consequences for financial stability. Because of a stress on maximising returns on equity, banks’ capital reserves were depleted and leverage increased. Banks that did these things were handsomely rewarded by the stock market, and bank executives whose compensation was linked to the share price did well. They were not being stupid or “greedy”: they were operating rationally, in a system where the individually rational course carried moral hazard and great risks to the wider economy.

The principal mistake was to suppose that new financial instruments, which transformed bank lending into marketable securities that could then be widely distributed to investors, had reduced the risks of lending. In reality these ensured that financial contagion was spread across the system to investors least able to bear the risks. Many other factors compounded the crisis but the fundamental failure was of the market economy itself. As Keynes anticipated, the aggregate decisions of numerous private agents produced cyclical instability rather than stable equilibrium.

That is the fundamental explanation for the crisis. The reason for its huge extent was that a number of other factors combined to produce a near-perfect storm. These were: a property bubble; an explosion of credit; excessively easy monetary policy; errors of risk assessment; errors of regulation; conflicts of interest within the ratings agencies; and a failure of bank boards to keep aggrandising chief executives in check. Let us go through each of these.

First, it is a mainstay of financial theory that you cannot identify asset price bubbles ex ante: a sharp run-up in asset prices may be attributable to rational pricing decisions, such as a downward revision in investors’ required risk premia or upward revisions in expectations for corporate earnings. But there can be little doubt that across several countries property prices rose for more than decade with little relevance to any economic rationale. The bubble exhibited classic signs of speculation: people bought houses not for residential purposes or even investment, but in the expectation that they could sell them again rapidly for a profit. In the UK, property investment has generally not shown high returns since 1945, other than in specific periods – notably the early 1970s and the mid-1980s. But from 1995 to 2007 house prices rose from four and half times average earnings to nine times, in real terms.

Secondly, when the bubble burst in several countries, the effect on the wider financial system might have been limited. The stock market crash of 1987, for example, had little direct economic impact. But property is most people’s only leveraged investment, and the expansion of credit over the early years of this century ensured that many householders were financially stretched. When the property cycle turned, they could not keep up with payments, and they thereby contributed to a huge impairment of banks’ assets.

Thirdly, the explosion of credit was itself sparked to a large degree by errors in monetary policy. At the start of the decade, there was a conscious policy in the advanced industrial economies of seeking to displace recessionary pressures by stimulating particular segments of the economy. The Federal Reserve, concerned about the wider potential effects of the bursting of the dot-com bubble in 2000, cut interest rates and kept them low, in the sure knowledge that it would stimulate the housing market. It was a policy that was bound to result in imprudent borrowing and lending. This effect combined with flaws in the approach – not adopted in the US, but widely observed elsewhere – of inflation targeting.

In principle, inflation-targeting was a vast improvement on the less flexible approaches tried by governments in the 1980s of targeting the monetary aggregates or targeting exchange rates. Targeting inflation was a framework rather than a mechanical rule. The UK adopted it in the wake of the ignominious expulsion of sterling from the European Exchange-Rate Mechanism in 1992. And the system appeared to work. Central banks targeted consumer price inflation, which remained low and thereby contributed to economic stability. In retrospect, it is clear that this was deceptive.

Inflation was subdued to a large extent because of the effect of cheap imports from China, which grew rapidly from 1997. At the same time, there was huge inflation in asset prices, which the central banks noticed but did not regard as any part of their remit to control. That was a terrible mistake. Real interest rates fell to exceptionally low – and even negative – levels for long periods. Monetary policy had the effect of stimulating the asset price bubble and thereby destabilising the wider economy.

Fourthly, the banks themselves were working with risk models that were deeply flawed. There are two main ways of modelling the riskiness of assets – mean-variance optimisation and value at risk. The technicalities of these are beyond the scope of this paper, but the crucial point is that that they both measure the risk of investment positions with reference to historic asset price returns. The basis of financial theory is that over time asset returns form a lognormal distribution (that is, the logarithms of the returns form a normal distribution, or a bell-shaped curve).

No one in the financial sector believes this accurately describes asset returns, but many believe it is a useful prescriptive model, because it assumes the market is informationally efficient. These assumptions were disastrously wrong in the financial crisis, because they could not take account of either historically exceptional events or the way that investor expectations reinforce each other. When one part of the financial system lost confidence in the creditworthiness of counterparties, the contagion rapidly spread.

Fifthly, regulation was seriously lax. This came to light in the UK only when Northern Rock collapsed. It transpired that the bank had been inspected by regulators (from the Financial Services Authority) who were not even banking specialists. On a wider scale (see below), regulation tended to underestimate the need for bank liquidity – because no one expected the interbank lending market to freeze up.

Sixthly, the private-sector agencies that examined institutions’ creditworthiness failed to do an effective job because they were shot through with conflicts of interest. These were the ratings agencies. Typically, the consultancy side of these businesses would advise banks on how to structure financial products so as to gain a high credit rating. The ratings side of the same business would then come in and award that rating. The conflict of interest badly needs to remedied by breaking apart these institutions, just as the link between consultancy and auditing had to be broken after the Enron scandal earlier in the decade.

Seventhly, the quality of banks’ boards was truly lamentable. It is a shame in many respects that the most public face of commercial banking in the UK, and possibly in the world, is Sir Fred Goodwin, the former chief executive of Royal Bank of Scotland. Sir Fred won the journalistic sobriquet “the world’s worst banker” for his business decisions, which included building up a US banking network with punishing exposure to a collapsing housing market. But the most destructive of all his decisions was RBS’s acquisition of ABN-Amro, the Dutch bank, for a massively inflated price of Euros 70 billion at the top of the market. It wiped out shareholder value and any competent board should have prevented it. But RBS’s board proved itself useless. This was an extreme case, but throughout the banking sector, business decisions were taken by people who lacked the financial experience and education to understand what was happening on proprietary trading desks and other product lines involved in complex derivatives.

These do not exhaust the contributory factors to the financial crisis, but they are prominent in the allocation of blame. It must be stressed that the main fault lay with private-sector agents rather than with government. This does not mean, however, that the principles of economic policy were overturned. While the crisis was a private-sector phenomenon rather than a government failure, my explanation for the crisis places the onus on the financial system rather than some inherent weakness of the market economy.

The distinction is important. Crises within the capitalist economy are not necessarily crises of capitalism. The Great Depression will be engrained in folk memory long after it has ceased to be part of the recollection of anyone alive. Yet it was not, strictly speaking, caused by a deficiency of aggregate demand. It was driven rather by bad policy – namely, adherence to the Gold Standard. Monetary policy in the 1930s was geared to defending convertibility of paper money into gold (there was till 1933 a fixed price of one ounce of gold for $20.67).

The Gold Standard was premised on the notion that money had to be backed by some asset, lest confidence in the currency and the ability of the government to meet its liabilities be undermined. The consequence of this near-superstitious policy was that the Federal Reserve could not serve effectively as lender of last resort to the banking system, because it had first to take account of the demand for gold. (If it lent money to banks, then there would be more money in circulation, thereby reducing the credibility of the Fed’s guarantee to exchange paper currency for gold.) Consequently monetary policy was exceptionally tight, even as the economy plunged into slump.

The recession of 2008-09 was comparable to the Great Depression in its severity. Data on the collapse of industrial production, the fall in asset prices and the contraction of trade volumes are quite similar in their sheer scale. Another crucial similarity is that the crisis was born not of a flaw in the economy but of a specific failure in one component of it. In the more recent crisis, it was the failure of a dysfunctional financial system – not of liberalism more generally. The crisis demonstrates that financial capitalism does contain an inherent systemic risk: the banks are tied to each other, in the wholesale lending market, and if there is a problem of bad debts then it can contaminate the entire system. But the policy response worked: there was not a general crisis of capitalism. One segment of the economy created massive instability; and governments and central banks acted, in the Keynesian mould, to stabilise the economy.

Without doubt, financial deregulation was advanced as part of the wider liberalising impulse of the 1980s. But this was not some doctrinaire belief born of irrationalism. There were sound reasons for abolishing fixed commissions in the stock market (to reduce the costs of trading) and encouraging financial innovation (to allow companies to manage their risks better and investors to structure their portfolios more efficiently by diversifying at low cost). The danger was rather that, as the securities industry devised ever more complex products, the risks to wider financial stability were disregarded.

That the failure was private rather than public does not mean that policymaking was benign. On the contrary, governments and central banks also conformed to the model of behaviour that I have suggested: they responded to incentives to behave in ways that appeared rational at one level but were destabilising at the level of the aggregate economy. Thus, as I have argued, central banks kept interest rates too low, even to the point of prolonged negative real interest rates, and regulators were also at fault.

The stress on capital requirements, while paying inadequate attention to the need for liquidity, had particularly dangerous consequences. Banks could be solvent but still not be able to meet their liabilities. There was historical precedents for this (such as Credit Anstalt, the Austrian bank whose failure in 1931 caused financial panic and helped the rise of political extremism across the continent). But regulation did not take account of it. Thus, when a shock hit the banking sector, lending froze. An entire class of asset-backed securities turned out to be impossible to value accurately, so banks hoarded cash in preference to lending it out.

The crisis in the banking sector led inevitably to a collapse in economic activity. There are many historical cases of collapses in the prices of particular assets or asset classes, but the crisis of 2007-09 was different. This was is a collapse of a credit bubble. As credit is the lifeblood of the capitalist economy, the costs in lost output and employment were huge. While the remedial measures – easing policy dramatically and rescuing the banking system – were authentically Keynesian, in seeking to stabilise a cyclically unstable economy, the flaws that they corrected were within the financial system itself.

The crisis demonstrated that finance is different from the rest of the market economy. It is not like an industry that produces goods or provides services in a competitive market, but is more like an essential utility whose smooth running is essential to the wider economy. Were it left to fail, then it would bring down the rest of the economy with it. Where the financial regime failed in recent years was a misalignment of incentives and poor regulation. Bankers had scant conception of the risks they were taking on; and given that finance is above all a discipline of the efficient management of risk, this was a huge systemic failing.

The significant but limited sense in which the neo-liberal consensus of the late 20th century failed was in its indulgence of perverse incentives in the financial sector. Financial markets rewarded companies that reorganised themselves – whose executives saw the constituent businesses as a portfolio of assets to be traded. Banks rewarded traders without regard to the riskiness of their investment positions. Because bank bonuses were paid out annually, a trader could afford to take on a position that risked blowing up further down the line.

Traders would not be the ones paying he price of this failure, because there was no way that their bonuses could be clawed back once they had been paid out. What mattered was simply their investment return, rather than some measure of risk-adjusted returns. When the banks’ positions did eventually blow up – with their massive exposure on and off the balance sheet – they were bailed out by the taxpayers. The profits of banking were enjoyed privately; the losses were borne socially.

That outcome has caused great public hostility and incredulity towards the banks, for good reason. The central lesson of this crisis is that banks are different from other companies. They have wider responsibilities than to their shareholders alone. They need to look to the stability of the wider financial system. Regulation must be designed so far as possible to ensure it, and government needs to be prepared to stabilise the banking system and the wider economy.