Amara's law

Illustrating a hype cycle, used to show how Amara's law works

A hype cycle perfectly visually illustrates Amara's law.

Amara's Law is a computer saying which states:

We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.

While this law, coined by the then president of the US-based Institute for the Future, Roy Amara, has been associated with both cyber attacks and nanotechnology, another area where it has shows its truthfulness is in the Internet itself.

Amara's law applied to the Internet

As the World Wide Web began to take off in the 1990s, this generated a tremendous amount of hype or belief in its power and ability to make money for investors and hard-working entrepreneurs. Venture capitalists initially overestimated the power and importance of the Internet, making massive and often unrealistic investments in online start-ups, which resulted in the over-inflating of their value and their subsequent economic crash. This is now known as the 2001 dot-com bubble, of itself an example of an economic bubble, which, when it gets pricked, loses money for many investors and businesses.

A BBC TV documentary at the time illustrated this very well. A farmer's wife was selling women's clothing online from her kitchen table, and without any help, had a higher turnover than a company which had attracted investment through its business model of selling products for cats online, and which had an office large enough to contain 100 workers, with ten full-time employees already. The farmer's wife had a realistic business model whereas the cat products company was a victim of Amara's Law, overestimating the short-term power and importance of the Internet, and was itself a victim of the pricking and deflating of the dot-com bubble.

Hype cycle

From the viewpoint of fifteen years later, it is clear that the reason for the dot-com bubble was not because the Internet was never going to be a place where companies can make money, as thousands of incredibly successful and highly prosperous Internet start-ups have shown, but due to overestimating the short-term effects.

The first part of Amara's law has been described as an example of a hype cycle, as explained by the information technology research company Gartner. A high-point of "inflated expectations" is followed by a low-point or "trough of disillusionment", which is what happened during the dot-com bubble. Yet, in the longer run, it turns out that both the high-point and the low-point of the hype cycle were only the beginning of the story. The low-point of 2001 turned into a new and very real high-point in the 2010s, with the importance of the Internet for all types of businesses continuing to grow massively today.

Amara's law applied to data

Data has become one of the big buzzwords for 2016 and is associated with other important themes this year, such as artificial intelligence (AI), which, along with the machine learning which it depends on, is entirely reliant on big data. The massive growth of data are also subject to Amara's law, while data quality which you can trust in will play a central role in how Amara's law works for data. The data only have value when they are given proper data cleaning so that they have data integrity. Without this, even the smartest companies developing AI, and all businesses which are dependent on data, will fail if their data are full of inaccuracies, duplications and other corruptions. Rogue data are about as useful to their owners today as substantial investments in the online cat products market were to financiers and investors in 2000. All the data in your data repository thus need data validation.

The inflated expectations or hype cycle around data, as a manifestation of Amara's Law, are almost certainly going to be followed by a low-point of disillusionment and despair, and this low-point will be caused simply because the data are not of a high enough quality to be trusted. The current big players, who already employ large numbers of data scientists, will find that these scientists have to spend between 60% and 90% of their working lives data cleaning their company's data to remain competitive in an increasingly data-driven world. Others, because their companies are not tech businesses or because they are not yet big enough to employ their own data scientists, are going to find that their initial hopes at the prospect of how data are going to transform their company and drive its growth will hit their own low-point of disillusionment or disappointment. This is because the data they are using are not doing what they should, due to dirty data which have not been cleaned effectively. Unless, of course, they use our unique web-based data quality API to scrub up their data, and thus avoid the gloomy initial predictions of Amara's law.

However, in spite of any low points of disillusionment concerning data, their long-term importance will remain central to the success of most businesses. Just as Google and a few other Internet start-ups survived the dot-com bubble and subsequently became very successful, so those companies which cleanse their data successfully to produce quality data that you can trust in right now are going to be the ones which stand-out as the most successful in a future world dominated by data.

Spotless Data's API solution

Spotless can now offer offline processing of your data to ensure GDPR compliance. You can read our introduction to using our API to validate your data and then try out our service on your my filters page but, for this, you need to log in first. You can take advantage of our free offer of 500Mb of free data cleaning to see how much you like our service. If you haven't already done so, you can sign-up using your email address, Facebook, Google or GitHub accounts. You may also view our video on data cleaning an EPG file, which also explains how to use our API.

We use the https protocol, guaranteeing your data are secure while they are in our care and that they are not accessible by any third party, a responsibility we take very seriously.

Here is a quick link to our FAQ. You can also check out our range of subscription packages and pricing. If you would like to contact us, you can speak to one of our team by pressing on the white square icon with a smile within a blue circle, which you can find in the bottom right-hand corner of any of the web pages on our site.

If your data quality is an issue or you know that you have known sources of dirty data but your files are just too big, and the problems too numerous to be able to fix manually please do log in and try now