by Will Gunadi
Regardless the size of your business, when it has been running for a while, you will have at hand what all businesses accumulate over time. No, in this case we are not talking about profits nor debts, rather, it is about: Data.
Just by the fact that they operate, businesses will gather various types of data. More importantly, business data is the lifeblood of every business decision made. The higher the data quality is, the better the decisions you could be making.
Problem is, when it’s not sufficiently reviewed and monitored, all accumulated data will start to lose its integrity and accuracy, resulting in misleading indicators and measurements, which will yield two of the most damaging factors for a business: Loss of opportunity and Hidden costs.
How does business data go bad?
Over time, any computerized business’ systems will start to have bad data. This is not something mysterious, in fact, as a savvy business owner, you should be expecting it. Here are some of the ways for bad data to creep into a system:
- Bad data entered by users that is not caught by the existing validation rules
- Policy or business rule changes
- Program bug due to the implementation of the policy changes
- Structural changes to external data (eg. tax rate, zip code, area code, ISO specifications)
In essence, your computer systems change as your business changes due to modifications to internal policies or to external regulations. It is inevitable that during these modifications, bad data is entered and processed into the systems. Sure, we could minimize the negative impacts with good software quality testing practices, but we would not be able to catch them all.
The only effective way to combat bad data caused by these unavoidable changes, you need a system in place that will allow you to see the data comprehensively in order to locate these bad data early before they become real problems.
Meet OLTP and OLAP systems
In this day and age, Online Transaction Processing (OLTP) systems have become the norm, even for small to medium businesses. Most people nowadays rely on some kind of computer-based ERP (Enterprise Resource Planning) system to run their businesses, maybe a consolidated email/calendar package, an accounting package, some combinations of order entry, invoicing, shipment, warehousing modules, all of which are backed by a database system. These are the typical components of an OLTP system.
This is where most small and medium businesses stop; which is fine apart from the fact that you only have one way to view your business data, and a non-intuitive one at that. Making data quality maintenance harder than it should be.
An OLAP (On-Line Analytic Processing) system was designed to provide you with an alternate way to view and analyze your business data; more accurately, a much more effective way.
Large corporations and big businesses spent billions of dollars every year to develop, utilize, and maintain OLAP systems to complement their OLTP ones. They know that without a constant effort to monitor and measure business data, it can go out of hand very quickly.
Unfortunately, for most small to medium businesses, the value of setting up an OLAP system has not yet become clear. Why? Partly a mindset issue. To understand the value, you have to:
- Realize the two distinct approaches in handling the same business data, one is geared towards data entry and processing (OLTP), the other, data analysis and decision-making (OLAP).
- Accept that analyzing your own data is not a waste of time and resource, rather, it is a vital part of not only running your business but – more importantly – improving it.
Adding insult to injury, the cost of implementing a typical OLAP systems is tantamount to long-term, expensive, and serious commitments. While there is absolutely nothing wrong with long-term and serious commitment to improve data quality, the expensive part prevents a lot of businesses to even try to setup and utilize OLAP systems. Which is a pity.
Why can’t I just use one system?
The difference between OLTP and OLAP starts from the underlying structure that holds your business data. OLTP systems has, as its main purpose, to capture in real-time, your business data such as order entries, invoice generations, shipment records, warehouse inventory movement, accounting journal entries, commission calculations, sales tax, etc.
A good OLTP system would have a solid data model that dictates how the above data are stored and processed. Unfortunately, this very strength causes the data to be extremely efficient for computers to process, but it is not human observable. At least not without a lot of necessary lookups and remembering bits and pieces of information all over the place. Again, something that computers are designed to deal with.
Decreasing data quality happens when no one reviews the business data. Any statistician would tell you that the accuracy of any record is proportional (up to a certain point) to the number of reviews that the record receives.
The same principal applies in this case also. The OLAP system was designed to allows you to review your business data in the manner that is effective for us (humans) to analyze, instead of computers. However because it is a computerized system, it also provides the automation and number-crunching facility that we can use to generate meaningful reports.
Okay, so what can OLAP systems do for my business?
In a nutshell, a fully operational OLAP system should help you to:
- Discover hidden information→
- Your business data resides everywhere, not just in your OLTP system database. And it’s not always obvious how they relate to each other.
- Discovering hidden data may give you an insight on certain aspects of your business that needed attention.
- Identify and fix data integrity problems →
- Keep your business data accurate and up-to-date
- Verify data relationships over long periods of time
- Plan for data expansion, archiving, and storage →
- As your business grow, so will your data accumulation
- Fulfill requirements to keep complete records (example: Legal or Tax auditing purposes)
- Staging area for server movements, upgrades, and failure recovery
- View your data history, trends, and movement with improved clarity and accessibility →
- Discover new opportunities revealed in existing data
- Keep track of aggregate data which is not stored by the OLTP system
- Forecasting based on trends
- Better decision-making ability
Can my business afford one?
As mentioned above, you should view an OLAP system as a tool to grow your business. It is the natural extension to an OLTP system which you have already used.
The good news is, with open source OLAP systems such as Pentaho BI Suite, you now have a good alternative to high-cost systems from SAS, Oracle, Microsoft or IBM. Open source systems are characterized by freely available installations without any licensing restrictions.
Is it any good? As with OLTP systems, it depends on the implementation and subsequently, the implementor. A well implemented Pentaho system should be perfect for small to medium businesses not only because of the zero entrance cost, but the complete set of tools that are customizable down to the source-code level.
Of course there is nothing that prevents big businesses to use Pentaho, but along with available budget, comes options to use the other systems.
If your goal is to run a healthy business, the importance of data quality surrounding it can no longer be dismissed as an overhead. A well-planned implementation of an OLAP system should give you easy access to information that may be hidden in your OLTP systems.
Large corporations have known this for a long time. OLAP systems are not only considered, it is a prominent part within their plans and budgets. Today, with the advent of open source systems such as Pentaho BI Suite, the benefits of OLAP system has been made available to a wider range of business sizes.
There is not a single reason for not considering one. Really.