Open Source Hits The Big Time With Red Hat

The proposed acquisition of Red Hat by IBM, the largest-ever software deal in history, is obviously important from the signalling: it means that the open-source movement has finally arrived. If a firm that offers paid services around freely available source code (much of it written by volunteers) is valued at the astronomical sum of $34 billion, it surely suggests open source is now viable. It is the ultimate triumph of the ‘bazaar’ over the ‘cathedral’ as suggested by Eric Raymond in his seminal, eponymous paper some time ago: barely-organised chaos has defeated the hierarchical, top-down organisation.

It is surely ironic that IBM, the ultimate in buttoned-down, rule-bound, navy-blue-suited firms (although I hear the tie is now optional even if the white shirt remains de rigeur) should be the one buying the biggest player in the Linux-Apache-mySQL world of be-sandaled, bearded, shorts-wearing geeks infused with the libertarian cry that “software yearns to be free” or words to that effect from the arch-priest of that cult, Richard Stallman of the Free Software Foundation.

For, IBM was for long the epitome of the closed, hierarchical ‘cathedral’ as opposed to the ‘bazaar’ (though Microsoft took on the cathedral mantle later). It was the object of hatred by legions of computer science students, who decried its monolithic, impenetrable operating systems such as for the System/370 mainframes, the corporate staples in the bad old days of the 1970s. (Operating systems are the most core software of computers, the innards of the system that give them their character, such as Windows, Apple iOS, or Android).

In fact, it was as an act of rebellion against the work of thousands of programmers and their inscrutable code at IBM that the legendary Ken Thompson and Dennis Ritchie at AT&T’s Bell Labs wrote UNIX as a small, transparent operating system. It was a disruptive innovation, and it created a paradigm shift: here was the guts of the machine (reminiscent of Tracy Kidder’s The Soul of a New Machine) done by just two creative people, not an entire army of drone-like, faceless souls (as suggested in Apple’s famous ‘1984’ commercial where IBM is the Big Brother analog).

Now UNIX wasn’t open source, but it was almost, because Bell Labs licensed it freely to universities, which then made useful modifications, especially at Berkeley. Later, Linus Torvalds did make it open source, by rewriting the core (or kernel) to be free of Bell Labs’ code. In association with the Free Software Foundation’s GNU, he was able to build a complete OS known as Linux (although technically it should be called Linux+GNU), which was crowd-sourced and fully open-source. Linux and its successors have become extremely successful (Android and Apple iOS are both Linux derivatives), and thus UNIX’s compelling idea of simple building blocks has lived on.

I was fortunate to work at Bell Labs, and later at Sun Microsystems, and I was a member of the team that attempted to make UNIX commercially successful in the 1990s. We failed then because of fratricidal warfare between Sun, AT&T, Toshiba, NCR et al on one side, and HP, IBM, DEC, et al on the other. It was a good case study in how invention isn’t enough: you need strong partners to turn it into an innovation.

But there’s another reason why the success of the open-source movement appeals to us: because it shows that the old Indic idea of creative freedom is alive and kicking. In its heyday, the innovation engine in India was second to none, producing brilliant idea after brilliant idea; in fact one could make a case that India was by far the most creative of civilisations in both the breadth and depth of its inventions: everything from ‘sewn’ ocean-going vessels to abstract metaphysical theories to algebra to nano-carbon steel.

The remarkable thing is that this ecosystem had a completely different perspective compared to the Intellectual Property Rights (IPR)-focused regimen of innovation that prevails today. Instead of reasoning that the prospect of personal riches would motivate individuals to invent and create, Indic systems assumed that the very act of creation was reward enough. Thus, in many cases, the inventor or discoverer of some great idea would simply put the idea in the public domain, unsigned.

So much so that it often becomes difficult to even identify the individual who actually arrived at the breakthrough. Often it is only through others’ commentaries that we come to know that a certain individual or group was responsible. In some cases, we don’t even know that. For instance, there is the widespread belief that the decimal number system and the notion of zero were invented by Aryabhata. However, it is likely that it was the later Brahmagupta who did so.

If so, what is the motivation? Why would someone go to the trouble of thinking great thoughts if he or she wasn’t going to become rich and famous? A pertinent example is Jagdish Chandra Bose, the Bengali scientist and polymath, famous for discovering that plants may have some amounts of emotion: a plant that has ‘observed’ a person destroying another plant is thereafter ‘afraid’ of that ‘assassin’.

Bose discovered the fundamental properties of wireless transmission of information, which is the basis of communication technologies today. However, on principle, he refused to take out patents, because he didn’t think it was appropriate in his Indic value system. Guglielmo Marconi, a marketer, did; he became rich and famous, and there’s still a firm named after him. (There’s the audio speciality company named Bose, but that is a different Bose).

In the Indic system, it appears that there was a sort of contract between inventors and creators on the one hand and patrons on the other hand. The patrons, often kings or temples, would give the creative people a basic income to live on, and they would then dedicate their creations to the kingdom or deity, and thus it would be appropriate to put the invention in the public domain, for anyone to use.

Is such a system necessarily inferior to the IPR-heavy system that prevails today? In fact, does the IPR system as it stands really engender the creation of new knowledge? The evidence is mixed. On the one hand, entities like pharmaceutical industry stalwarts claim that without the monopoly profits that the patent system gives them, they could not afford the immense cost of pharmaceutical research and development. That sounds plausible, but it’s not clear if it’s actually true.

Because, on the other hand, there’s research from Petra Moser of NYU Stern: she suggests in Patents and Innovation: The Evidence from Economic History that too many rights to inventors deters invention, and that a more open system that encourages the diffusion of ideas and facilitates more competition may lead to more innovation. Perhaps, that’s what India had in its olden days, and perhaps we might try to revive that spirit of healthy competition. How can we move to capitalise on this trend?

The open-source movement has not quite taken off in domains other than technology, and in particular software. There is the Berkeley RISC-V open-source instruction-set architecture for designing computer chips. Using this, Indian Institute of Technology Madras engineers recently announced the first indigenously-developed and manufactured chip, Shakti. The first version of this is a microcontroller for Internet-of-Things (IoT) devices, but others down the line will be chips optimised for artificial intelligence (AI), machine learning (ML) and so forth.

The security and commercial benefits, letting Indian engineers leverage the open-source architecture to produce customised chips, are quite high.

On the one hand, for military and government applications and secure communications, it is dangerous to depend on imported chips (for instance there were concerns about the chips in the electronic voting machines, and whether they were vulnerable to Trojan Horse software attacks). Indigenous chips can be validated and verified locally.

Besides, in the emerging worlds of AI and ML, not having a semiconductor base is a big handicap, as the Chinese are now discovering, thanks to their trade war with the Americans. In effect, India can be held hostage by a foreign country that simply stops supplies of chips. As we saw in the case of China’s ZTE, wherein the company essentially halted operations in the wake of an American embargo (later lifted), being dependent on any one supplier can have grave consequences.

But going back to IBM, why on earth did they buy Red Hat? Turns out it’s IBM’s new play in the cloud computing market. As is well known, the move from company-owned data centres to cloud data storage is well under way. In the former, firms bought, installed and maintained their own machines, and they were often large IBM systems. But in the latter, firms use data-as-a-service, and pay a vendor to store their data securely somewhere on the vendor’s large server farms. In this race, IBM is nowhere to be seen, as it’s between Amazon and Microsoft, with Google some distance behind them.

This is fine in theory, but may lead to a crushing monopoly as we have seen in other areas of technology, where one firm is the proverbial 800-pound gorilla, throwing its weight around and extracting monopoly rents. It may get to be exceedingly difficult to extricate oneself from a relationship with one’s vendor because one is so tied into their ecosystem. It happens, and Amazon is very good at customer lock-in. And so are Microsoft, Google, Facebook, and all the other tech titans.

With Red Hat’s technology, IBM can offer a compelling story: a ‘hybrid cloud’. This is a variant of the “open-ness” meme, because it means you can be cloud-neutral. IBM-Red Hat will package up your data in a ‘container’ of sorts that can then be moved smoothly from cloud-vendor to cloud-vendor. You can avoid the lock-in.

There are other examples of such intermediary systems: TIBCO (originally The Information Bus Company) comes to mind, where you are able to use any vendor’s network standards, and TIBCO can translate among them. There is value to vendor independence, and people will pay for it.

The irony of the ultimate cathedral sort of succumbing to the ultimate bazaar is sweet. In future, we can expect to see many more open-source projects, I think: there was a project to do open-source pharmaceuticals, but it seems to have not gained enough momentum. There is, along with the boom in genome work, a whole new subculture of bio-hacking, where people do non-academic work in small labs to explore the limits of biology, often experimenting on themselves.

There are significant crowd-sourcing activities, for example Hyperloop Transportation Technologies has attempted to build an entire company using a distributed, crowd-sourced mechanism where there are very few employees but a large number of contributors.

In some sense, many of these trends point to the possible end of the corporation as the standard model of organisation. The gig economy, open source and free agents all fit together nicely.