Why you should use Spotless Data for data quality

Using a tablet for data quality while organising goods in a factory

Using a tablet to organise a data-driven factory manufacturing goods.

A question we get asked a lot is why should you use Spotless Data's API solution to address data quality issues in your organisation.

This question has two features to it. There is the question of why would you need to use any data cleaning service to be able to trust your data. And then there is the question of why choose Spotless machine learning filters solution, easily accessed through an API in your web-browser, rather than any of the other solutions which are also appearing on the market such as downloading a software application onto your PC or another device which will then clean the data for you.

Spotless Data - The easy-to-use API solution

The simple answer as to why you should make Spotless your preferred data cleansing service is because of the efficiency of our machine learning filters which learn from your particular rogue data issues and allow you to control the specifications of your data cleansing through an easy to use API (see our video illustrating how you can use our API to cleanse an EPG file). The other advantage of our browseable API is that it can be accessed through multiple devices so you can use it at work on your PC and then at home on your tablet or while on the move through your mobile, an option that is much more complicated to set up with a downloadable software application.

If problems during our data cleansing process come up then an automated flag alerts our data science team who then manually review the problem. If necessary they then contact you via your log-in details so that you and they can resolve the issue together, though, with our easy-to-use filters and you, the customer, defining the variables to be cleaned, this kind of situation only arises very occasionally.

Why data quality is important

Data has suddenly ceased being an IT speciality and gone mainstream. This provides huge opportunity for the most agile, forward-looking companies while at the same time offering a big headache for those in charge of the data for those companies, typically a Chief Data Officer (CDO), though many companies with plenty of big data still don't actually have anyone employed in this particular role.

The bottom line is that whatever you try to do with your data will primarily be dependent on the quality of the data your company has. Many companies would struggle even to recognise all the various sources of data that come into their company's data storage repository every day, including dark data. With data quality, your business can solve problems before they occur and without the need to run daily reports or checks. By using Spotless, your operation teams can focus on the core elements in your business without having to worry about the quality of the data.

A manufacturing example

Let us imagine you own an established company manufacturing garden tools. In the past, you and your predecessors built up the business by selling your products wholesale to garden centres, DIY shops, and even local ironmongers. You knew how many items of each product you were likely to sell in a year, based on previous experience, and you made the decisions about how many tools you manufactured in the coming year based on the sales records you had for the previous year. You worried more about producing too little rather than producing too much as anything you couldn't sell this year you would be able to sell next year; after all, garden equipment didn't change much from year to year. And while your engineering, sales and accounts departments did have data, this certainly did not mean you needed an IT department!

Eventually, you found the need to employ a computer expert to install computers and ensure the data you did have went into them rather than remaining in filing cabinets, and then with the rise of the Internet the IT expert had to ensure this data was backed up onto the cloud. You even set up a website.

However, in the last year or so that things have started to change at a dizzying pace, with the main trend being that your company is now swamped with data. Unless you learn how to utilise this data, and quickly, you may find your company is up against the wall and facing bankruptcy. Getting the data quality right with the minimal hassle is the first step towards using this data.

Your latest brand of internet of things garden products, such as sensor systems designed to let gardeners know what is actually going on in their garden and when their plants need watering provide huge amounts of data, which those who buy your products expect to see some tangible benefit from. This also means that last year's products are much more likely to be already out-of-date; your garden equipment is suddenly changing much more rapidly than you ever imagined possible.

The way you sell your products has also changed as your website now sells directly to your customers, and this is going so well that it is now your main source of sales. However, your product team is complaining that they don't know how many items of each product to make next year and last year's data is no use at all because of the rapid changes to the market and within your company.

Your marketing department is complaining that they no longer understand who they are marketing to nor exactly what kinds of new marketing campaigns they need tome up with in this, for them, entirely new environment where they have to engage directly with customers. Your sales team no longer need to visit just the same retail outlets to sell the same products, they need digital skills and to understand targetted advertising.

Then there are all the data from your IoT products, which if you do not manage them skillfully will result in masses of disgruntled customers. All the departments which make up your business are demanding better quality data and better big data analytics.

To make things worse, recruiting a data management team is both expensive and, even with plenty of money to play with, often difficult to achieve. This is because not merely your competitors but a whole range of other businesses that have nothing to do with gardening are also trying to get hold of the best CDOs and data scientists for their businesses because they are all swamped with data they urgently need to make the most of as well, not just those companies in your particular niche market.

The bottom line, as you realise after spending a few days researching the whole issue, is to ensure data validation leading to data quality you can trust in through data cleaning so that all your data have data integrity as the first step in becoming the data-driven gardening manufacturer that you realise is the only way your business will survive the next decade.

How Spotless can help

We at Spotless Data cannot solve all your data problems for you; we are simply the first step, which is cleansing the data and getting the data quality right. All the analysis and business intelligence your business is crying out for can be achieved if your data are no longer rogue data, full of inaccuracies, mismatches and other corruptions that make them so difficult to interpret. With excellent data in place, you are already on the path to embracing this new revolution in the way business is done, the most profound since the industrial revolution of the early 19th Century.

With clean, quality data your website data can then help your marketing and sales teams to understand your customers. Your clean IoT data can give them increased satisfaction while giving you some feedback on how your cstomers actually use your products. This was something unimaginable in the past where, once an item had been sold that was that, unless it came with a guarantee. Even that very simple guarantee data could only be used to measure any technical issues, such as a lawnmower breaking down in its first year of use, nothing compared to the vast amounts of IoT data you now have.

And having good quality data means being able to make the right decisions whereas having bad data means poor business intelligence and making the wrong decisions. Our goal in life as a business is simply to make these bad data a thing of the past.

As you deal with customers directly rather than going through the middlemen of retail outlets, you develop a Single Customer View for each customer, prioritising customer satisfaction and building a relationship with those who buy your products. You then discover through your dark data that a surprisingly large number of visitors to your website come from the Republic of Ireland and so decide to expand operations to that country as well.

Your data warehouse, now thanks to your using Spotless at its point of entry, starts to exceed your expectations rather than having your data pipeline go down whenever it receives a new type of data because the data failed to match the expected specifications, as you have been reliably informed keeps happening to your main rival. Your accountants report phenomenal growth, and you suddenly have a 21st Century business which is leaving your less well-prepared rivals far behind.

And, thanks to Spotless Data, the new data science team, led by an ambitious CDO, are able to focus on solving business problems rather than having their working lives consumed by fixing data mistakes. Among other advantages this has made the recruitment of these people much easier as most data scientists would much prefer not be fixing rogue data issues.

On using Spotless Data's machine learning filters

Our introduction to using our API. You can try using our service on your my filters page but need to be logged in here first. You can sign-up using your email address, facebook, google or github accounts. You can also view our videos on data cleaning an EPG file and data cleaning a genre column which explain how to use our API.

We are giving away 500Mb of free data cleaning to each new customer to demonstrate to you how smoothly our API filters work.

We guarantee your data are secure and not accessible to any 3rd parties during the time they are in our care, a responsibility we take very seriously.

Here is a quick link to our FAQ. You can also check out our range of subscription packages and pricing. If you would like to contact us you can speak to one of our team by pressing on the white square icon with a smile within a blue circle, which you can find in the bottom right-hand corner of any of the web pages on our site.

If your data quality is an issue or you know that you have known sources of dirty data but your files are just too big, and the problems too numerous to be able to fix manually please do log in and try now