BI and Custom Development

5 Requirements of a Truly Useful Reports

Posted by Will Gunadi on January 31, 2016  /   Posted in BI and Custom Development, Business, Data Best Practices

Suppose you have followed my suggestion in my previous post where I presented the different levels of Analytics usage that any company can find themselves in: Impaired, Localized, Coordinated, Company-wide, and Competitive.

First, let’s put out again the definition of Analytics that we’re discussing here:

Analytics is the effort to discover and communicate useful patterns within data

You are now aware of the potential power of analyzed data. Here are 5 requirements that will help you to evaluate your existing reports to get them in shape for the next level of Analytics

#1 Reports Must Be Worth The Cost

Measure the time it takes to compile and generate a report. If the time is significant, calculate how much are you paying out of the company’s revenue for the resource to generate that report.

If the time is not significant, measure how long does it take to update the report the next time it’s needed. Add the two measurements together then calculate the cost as the above.

If both measurements added are still insignificant, than you have one of those “magic” report, whose benefits may need to be re-evaluated.

Do you have a budget for company-wide reporting? If the answer is no, it’s time to rethink the way your company does reporting.

#2 Reports Must Be Timely

A truly useful report must be up-to-date when the report consumer needs it. Sounds obvious, doesn’t it?

Unfortunately a lot of truly useful reports must be compiled from multiple data sources. Generating an always up-to-date report is manageable for one source. But how about four? Six?

Each of these sources usually has it’s own timing of when is the best time to retrieve the data, also if you have 5 data sources (example: 3 spreadsheets, and 2 databases), most likely you also have multiple ways to reliably retrieve it.

Without some kind of automation, how many hours of human resource must be dedicated to this?

#3 Reports Must Be Role-Sensitive

A single set of data may be processed and analyzed in many different ways. Before generating a report for a particular role within the company, be it the CEO, Managers, or even a Field Operator, it is prudent to “walk a mile in that person’s shoes.”

If done correctly, each of the role in the company will have their own specific reports that can truly help the person in the role to see more of what is happening and to anticipate and adapt to changes.

Bonus point if you have a screen for the person in the role to input changes in policies/regulations and process, so those can be used to tailor the updates of the said reports. Hint: You need a well designed Data Warehouse to accomplish this.

#4 Reports Must Be Digestible

What do you call a 23 Excel spreadsheets per day reporting? Some may call it impressive,  productive, or “Wow!”

But the receiver of these reports call it “useless.” Being the receiver of a mountain of non-aggregatedindigestible data does not make someone productive.  It makes the person look for an EXIT sign.

Producing a lot of reports must result in an equally significant increase in visibility and decision-making quality and speed. To do this, reports (or even better: Dashboards) need to be consolidated at the appropriate level.

Does your company have a process in place for this?

#5 Reports Must Have A Purpose

Reports that are truly useful have names, which indicate their sole purpose in life. Here are a few examples:

  • Bad Profits -report (Truthfully, I’d be impressed if you currently have this report always available to you as a business owner or CEO)
  • Unnecessary, Untracked, Unknown Expenses -report
  • High Maintenance, Low Revenue Customers -report
  • High Revenue, About-To-Leave Customers -report
  • Financial “Leaks-Busting” -report
  • Low Productivity Team -report
  • Fuel Consumption On Billable Vehicles or Machines -report
  • Where Is The Industry Going and How About Us -report
  • Time To Renegotiate Vendor Contracts -report

In your next Management Meeting, ask your executives and managers to come up with the list similar to the above.  And compare it with your existing reports, if many of the reports don’t have a name and don’t serve a singular, clear, actionable purpose, question their existence.

Be ruthless, life is too short for useless reports.

At nextCoder

We make new reports or tweak existing ones to meet the 5 requirements above. Our clients love truly useful reports that we compile and generate for them because through those reports, they see not only what is going on, but also where to go next. 

In summary, we save our client’s money and increase their company-wide efficiency by:

  1. Replacing expensive human resources with automated electronic “minions” (computer programs).
    So what happened to the human resource? They are now happier (because they don’t do tedious tasks better suited for computers anyway) and they contribute more significantly by doing tasks that is fit for human brains: Analytics.
  2. Using our highly tuned and constantly improved DataWarehouse platform that can serve multiple companies at the same time. Or if you like exclusivity, we’ll clone one just for yours.
  3. Design, and consolidate numerous and diverse data sources into one readable Dashboard. Yes it could be done, and No, it’s not as easy as it looks.

We invite you to contact us if your company is ready to make your data work harder for you.

Is Your Company Ready To Compete?

Posted by Will Gunadi on September 02, 2015  /   Posted in BI and Custom Development, Business

Compete in what? In your industry, your marketplace. Against who? Other companies who are using analytics to gain market advantages.

Analytics is the effort to discover and communicate useful patterns within data

Surely by now this is not news to you anymore: Companies who took the time and made the effort to discover useful information from data patterns stand the chance to use the resulting insights to corner the market, shorten sell cycles, and often times, they seem to be on top of things before the “things” are even known or popular. 

So let’s talk a little bit about what has to be in place to prepare your company for the already-fiercer competition where those who are still oblivious to the potential usefulness of their business data, vs those who have focused and invested in analytics and are reaping the benefits.

(Although in the end, we know that this is not much of a competition, it’s more like watching a long-distance race between a ship without a compass vs one with a GPS system)

The Stages

It’s actually very simple. When it comes to analytics, every company falls into one of these stages:

  1. Impaired - Severely limited insight to what is going on within the company. Lots of vague, subjective, and approximated information. Changes (both internal and external) are dealt with in reactive mode.  And decisions are made based on “gut feelings”
  2. Localized (or “Spotty” to use a more sophisticated term) – Sporadic exercises of analytics. Usually isolated within certain departments, these efforts get no acknowledgment — let alone support — from upper-management. There is no large  impact on how business is conducted, and as the result, no long term benefits.
  3. Coordinated - At this stage, the upper-management are onboard, and the departments start to get the support for establishing enterprise-wide metrics and analysis. Processes are tweaked based on metrics evaluations.  Decision-makers have access to objective information that they need.
  4. Company-wide - Analytics has been absorbed into the culture, it guides the creation and maintenance of processes and systems. The company can cope with unexpected changes because everyone knows what is coming and are in the position to proactively deal with it.  Decisions are made based on objective *and* timely information due to continuous access to up-to-date data.
  5. Competitive - The company is ready to compete for the spot of market dominance using analytics to outrun and outmaneuver competitors. At this stage, analytics is not limited to internal data, but also external trends, market swing, and the foresight on game-changing — buzzword: Disruptive — technologies or processes.  This kind of power can only be materialized with a cohesive analytics effort from top-to-bottom within the company

In Short

Which one of the above stages is your company in currently?

Despite what we hear in the media today about the buzz surrounding analytics and Business Intelligence, most companies in the world are still operating within Stage 1-3.  

Case in point, I regularly encounter company owners who declared that they are doing very well without all these “Data Analysis.”

Let’s think about this. The only reason the above statement is true at the moment is because most of their competitors are still operating the same way (ship without a compass).  But at the rate that even small companies started to put in place processes and systems to gather metrics (data warehouses, business alerts, Truly Functional Dashboards(TM), IoT systems, API integration, etc.) things will soon accelerate towards analytical competition.

The real question is: Who is going to be the first to install the GPS on the ship? Them or their competitors? 

So what a company to do? In my next post, let’s discuss what you can do to take your companies to the later stages. What sort of personnel would you need, how much coordination and buy-in would you require from the executive team and what role would that play, and lastly, what technology is available to us today and beyond.

Until next time.

Do You “See” Your Customers?

Posted by Will Gunadi on June 17, 2015  /   Posted in BI and Custom Development, Business

In my last post, we’ve established that the right level of visibility is crucial to be able to react — or even better, anticipate – changes that can hit your business from any angles.

Let’s take one of the most important aspects of a business: Customers, and see how we can design a system that provides the right level of visibility to customer information through out the company.

What “System” Are We Talking About?

The word “system” has its own share of abuse, so to prevent that here, we’ll narrow it down to a system that tracks Customer Scores only.

Why Customer Scores?
A Customer Score is a number we assign to a particular customer or customer type.  When it is applied to all customers, then we can rank them from the most valuable to the least.

This is one of the most important component of running a successful business, yet it is one of the most delayed if not avoided. Reason? Because it is tricky to figure out consistently , it takes a lot of thinking, and nobody has time for that, furthermore, after you figure out how to do it, then you have to come up with a continuous process to cope with the myriad parameter changes

Now, really, who has time for that? Our most successful clients do find the time.

Before we begin, the system we are prescribing here (in truth, all the systems we at nextCoder build and design for our clients) is designed with these rules:

  1. Any information is entered once and only once into the system
  2. The system adapts to the different roles within the company
  3. The information (customer in this case) is consistently represented to each company department (the key here is “consistent” not “identical”)

Having established what system we need to deal with Customer Scores tracking, let’s get started by discussing what does it take to calculate a score for a customer.

How Much Is Each Customer Worth?

Notice that I am not simply asking how much does a particular customer contribute to your revenue, that is just the first part of the equation, but what matters equally is how much does it cost to retain them.

At first blush, the first question (how much do they contribute) seems to be an easy one. We just have to ask the Account Receivable department, right? Until we start to ask the follow-up question, are they current on their invoices. Suddenly what seems to be an easy question started to become tricky to figure out.

The second question is much harder to answer. In reality, it cannot be answered until you have a system that asks the right question to every single department in the company involved in servicing the said customer.

The third question is how long has the customer been with the company? This question matters especially when the business depends on customer retention.

But what if my customers comes and go (and I have 100,000 of them)?

If the business only touches the customer for a short period of time (to buy a product, for instance), then a better question would be how likely for the customer to come back or to recommend the product. TIP: E-Commerce systems and their hooks to analytic services is your friend here. (Yes, we help our clients with this also, it’s quite amazing to gather data from six marketing platforms to answer a single question).

Ready for number four? What is the projected value of a customer? The earlier in the sales process we are able to assign the score to a customer, the better visibility we would have.  In this context, visibility means the ability to forecast and predict revenue pipelines more accurately.  This is crucial for business with long sale cycle and hard to predict trends.

NOTE: Obviously, depending on the nature of the business, you may end up with 5th, 6th, or even more aspects that you have to consider. But for the space that we have here, this will get us going.

Out of this exercise, we get Customer Scores formulas.

The System Itself

What does a good Customer Scores system look like? For starters it would have these components:

  1. Screens for each department to enter information that is relevant for each customer. And other sources within the department that can be automatically extracted from.
  2. A nightly (or hourly) process that gathers the data entered at the screens above. The data is then processed according to the Customer Scores formula while at the same time making sure that the three systems rules above are followed. If not, a business alert would be generated
  3. A set of Dashboards for different roles within the company.  From floor-supervisors to CEOs (even Board members), visibility is manifested in an always up-to-date, always accessible centralized online system
  4. A set of automatically generated reports are sent via email, or get dropped-off a remote server (for example vendors’ or banks’ or government-agencies’)

Once this system is running, everyone in the company will slowly gain the right level of visibility on Customers. Only at this point ideas, innovations, shortcuts without compromise on quality could emerge organically (unforced).  Why? Because everyone is thinking about how they can help serve the customer better within the constraint of their own department

This is the fruit of visibility.

Furthermore, when everyone in the company has this kind of visibility, then as a company, you would be able to anticipate industry-wide changes and you will be in a position to withstand crises that would otherwise bring you down.

In Short

Do you have a Customer Scores system? Wait. Scratch that… Do you have at least a Customer Scores system? How about Employee Scores system, Vendors Scores system? How about Profit Scores? there are bad profits as well as good profits, right? Is there a good scoring system for Costs?

In this post we focus on the Customer only because it will take a book to discuss the others aspects.  But the thrust is the same. Whether you have 10 or 100,000 customers, you still have to “See” them in the right context that is relevant to your business growth.

So it comes down to: Do you want your business to grow? If so, you need visibility, all the time. Without a system that is designed to provide that for you, how can you maintain the visibility over time?

Too many companies are too busy running and not looking for information that can result in better directions.  This is why nextCoder was founded, to come alongside companies and build systems that give them visibility where it matters.

If any of the above resonates with one of the needs within your company, hit me with a note and let’s discuss. And of course, follow this space for more Business Intelligence tips.  This post is also published at our website: nextcoder.com

Why Visibility is Useful

Posted by Will Gunadi on May 18, 2015  /   Posted in BI and Custom Development, Business, Data Best Practices

What is Visibility?

In my previous post I’ve established why visibility is very important in running and growing a business. And how the lack of visibility if both a vicious cycle that can result in the decline of a business, and that visibility is both ways, top-down and bottom-up.

Now let’s explore further what exactly do we mean by visibility? In other words, what needs to be visible and to whom?

The short (and useful) definition of visibility:

Being able to see at anytime what is going on in the business that is relevant to the question or task at hand

From Top to Bottom

Executives have the task of making strategic decisions that should affect the business positively. From this vantage point, visibility is being able to see what is going on to make not only good decisions, but timely ones.  In practicality, this translates to knowing what trends are in effect within the business. In other words where are things going based on the events that happens at certain periods of time.

NOTE: Sampling of data taken at certain intervals is a common way to get this kind of visibility. The cross-section of the company where the sample is taken can also provide important insights.

From Bottom – Up

Employees doing daily tasks are the main source of data in which the sampling can be performed. Every aspects of the tasks that can be measured should be measured. Visibility here means more than just being able to perform the task, but also to know why a task is required and how the task affect other areas across the company.

The more successful a company in providing this visibility, the more it can work as a unit rather than disparate departments busy protecting their own interests.

This is truly what is meant by the term Business Intelligence.

What do we do with visibility?

When we can see clearly what is going on in the business, we can decide which action to take and the timing of which the action is taken. So the main motivation to increasing visibility is to know what to do when certain things transpire in the business that needs attention, way before it becomes a problem.

More importantly, is the long term effect of visibility:
The increase in the overall performance of the company in conducting business. At the extreme level, this could mean becoming a game-changer in the industry. I don’t need to belabor why this is important, it’s what makes a company valuable, to anyone who deals with it.

Enough with Theory!

Let’s take an example, let’s say company A found out that the revenue is increased over the same period previously but so does the cost. Let’s use the concepts we learned above about visibility to find out why.

In this situation, the first step is to find out what caused the revenue increase. Let’s say that the increase of revenue is caused by more new customers purchase the products.

Then we take measurements across departments in the company on how much each department either contribute or is affected by the increased revenue (new customers).

NOTE: This is a good example of looking at things differently, instead of focusing on the Sales Department vs Accounting, we start to get the feel of what is taking place at other departments because of this known fact (increased revenue).

Upon analyzing the result of the measurement, we noticed that the Customer Service calls spiked up during the period and the procurement department also are busy sending out part replacements.

I turned out the new customers are not equipped with enough knowledge about the product, resulting in higher refund rate (dissatisfaction) or parts replacement due to damage.

Knowing that, the company tasked the marketing department to come up with materials that can be incorporated with the product packaging, so the customer has to go through them in order to enjoy the product.

So What?

The above scenario sounds like an easy and typical problem to solve. But consider this:

  1. What raised the red-flag and triggered the “investigation” ?
  2. How did the company found out quickly that the revenue increase is caused by new customers?
  3. How long would it take to take the measurements out of each department in the company? Without a conscious and concentrated effort to gather the data, this can take months or longer
  4. How long would it take for Accounting to approve the budget for the new marketing materials?

Interesting questions, isn’t it? And it’s a set of questions that is rarely talked about.  Let’s consider some answers:

  1. What triggered the investigation is the Executives having visibility to what happened to both revenue and cost over a period of time. Contrary to what most believe, this is not the job of the accounting department.  Without a conscious and concentrated effort, no one would have time to do this
  2. The company quickly can pinpoint the source of the revenue increase because it knows the customers. Only companies who took the time to know their customers via analyzing their data would be nimble enough to use the insight when the need arises
  3. The reason this company were able to analyze the situation is because of the visibility across different departments. Each of the department has indicators that everyone can see when their tasks (and cost) spiked up
  4. Lastly, the reason this company were able to address the problem quickly is because the accounting department has visibility to what the marketing department is capable of in terms of designing the new packaging, and the ability of the production department to dole it out.

As we can see clearly, without visibility, this whole process would be much slower if it happened at all.

In Short

Now let’s make this closer to home. Were the above story to happen within your company (or the company you work for), what would be the outcome?

Do you have enough visibility?

In a form of a list, this is Visibility for starters:

  1. Periodical sampling of data to discover trends
  2. Transparent cross-department measurements
  3. Customer tracking and scoring
  4. Employees performance scoring
  5. Vendors tracking and scoring
  6. Documented Time

Visibility is something that we don’t miss until bad things start to be noticeable. Companies who are proactively putting in place a system — Business Intelligence tools such as a Data Warehouse, Dashboards, fall into this category — that allows for visibility to be achieved, will find out that the more they use it, the more effortless for them to not only react, but also to anticipate changes and “roll with the punches” so to speak.

Next post, we’ll see what is the make up of a good system that provides the 6 aspects of Visibility listed above.

Change The Way You See Your Business

Posted by admin on October 02, 2014  /   Posted in BI and Custom Development, Business, Data Best Practices

What do we tend to do when we are too focused into something? That’s right, we tend to lose sight of what the big picture looks like, and lacking that crucial perspective can be costly when you are in charge to run and grow a business.

As the saying goes “Don’t miss the forest for the trees,” as business owners and managers, we are to be vigilant in maintaining a clear vision and a good sense of where our business currently are.  Only when we managed to do so, then we would be in the position to determine where to go next and how to get there.  Furthermore, it helps us to focus on the right detail, the ones that has a big impact on the business.

The good news is, there are a plethora of tools and techniques out there that can help us in this regard.

In this article, let me try to illustrate this concept using a simple example:

mini_theater

Take a look at the chart above. What did you notice? Bubbles, different shapes, and colors.  For those of us who are data-geeks, this is one variation of a bubble chart.

Now the story behind this chart: Once upon a time, there was an imaginary theater company, who performed in their own facility, three times a week, on Monday and Thursday afternoons, and on Saturday mornings (represented on the vertical axis of the chart).  Each performance slot is indicated on the horizontal axis.

So if we do a mapping between the time-slot and the day, we would find a single performance — a bubble and a number — representing the available seating for that particular performance.  In other words the seating capacity of the theater minus the ones that are already sold for the performance.

Now that you know what the chart is about, take a moment and look at the chart again and let’s see what jumps out at you.

All done? Now put yourself in the position of the business owner of this theater company and let’s compare notes:

  • The size of the bubble intuitively indicates the seats left (which means tickets unsold)
  • The color of the bubble indicates the type of seatings available (main seatings have more capacity than the balcony or the box)
  • The larger the bubble, the worst it is from business perspective, again, intuitively, you want to shrink the bubble to 0 available seats, which means a full-house performance
  • What about those large negative numbers on some of the bubbles? Those indicate overbooking.  Something that must be dealt with before the actual performance, otherwise two or more annoyed audiences will be the result — which in today’s world will likely translate into negative tweets and facebook posts (bad for our reputation)
  • The chart represents one week, it would be easy to produce one for next week, the next two weeks, etc.  In fact, the chart can be an effective planning tool for promoting the performances based on which time slot still have a lot of seatings available
  • Once we have multiple of these charts, we start to see the forest, which weeks of the months is the best time to promote a particular performance.  Is this the time to introduce a Season’s Pass ticket? How far are we from having to get a new building?

Do you see what we’ve accomplished? With a single chart, as a business owner, we can see:

  1. What is going on
  2. Where to focus our attentions to
  3. What bad things that would happen
  4. Which options are available for us
  5. How to create a strategy to do something about it
  6. What does the big picture look like

That is the power of Data Analytics and Visualization.

What About Real-world Scenarios?

Here is the kicker, although I  created a fictitious story for the chart, in actuality, it is based on a real-world visualization that we produce for one of our clients.  It is a real business tool which takes the concept of charting to a whole new level.  That is, we made it a user interface where the business owner can actually reserve a seat, or upgrade it right there on the chart, without having to go through extra screens.

Our client loves this view of their business because it gives them visibility way beyond hundreds of numbers in a spreadsheet that they are used to deal with.  And it make efficient use of both their time and strategic thinking, so they can arrive to better decisions, faster.

Ever wonder why military generals over the ages plan their next moves by analyzing a map?

Let us know what you think about this, and let’s work towards changing the way we see our businesses.

-WG

How To Build and Maintain Multiple Online Platforms

Posted by admin on May 14, 2014  /   Posted in BI and Custom Development, Business, Online Platform

The other day, out of curiosity, one of our clients asked me: “How are you guys able to build and maintain the quality on so many platforms simultaneously?”

That question prompted me to pause and think about this:

What are some of the things that make it possible to build and maintain multiple online platforms simultaneously without overwhelming the capacity?

As I started to write out the answer, I decided to go ahead and share two “ingredients” or “secret-sauces” that works for us and enable us to serve multiple clients simultaneously without compromising on either quality or support.

So What’s The First Secret Sauce?

Just like in the movie “Kung-Fu Panda,” — don’t read the rest of this sentence if you haven’t seen the movie yet — the “secret” turns out to be a blank scroll.

No really, the reason we at nextCoder are able to crank out a lot of platforms is not because of a secret-sauce recipe, rather, our carefully (and painstakingly) selected and molded “ingredients” (think building blocks) that we continually evaluate, test for fit, improve and build upon.

Because we are intentional about it, these building blocks are ready to be attached together to become the online platform that serves a particular industry domain. But in their disassembled state, they are just like “a blank scroll.”

See, back when dinosaurs roamed the earth — that’s the era of DLL-thunking, MS C++ vs Borland C++ vs Intel C++ to you, young programmers — the choices to make when developing an application is which compiler, which sorting algorithm, which expensive optimized libraries, even which memory manager.  The concepts such as Open Source code repository, data sharding, virtualization, JIT compilation, UI-templates, Aspect programming, had yet to become mainstream.

Today, we have the luxury to pick and choose from literally hundreds of already-made “blocks of code” to build our platforms with.  Some are Open Source, some have to be purchased, some are as complex as a whole application or platform framework, some packaged as neat little libraries, and the others, a network of code snippets and functions.  Internet browsers (big or small, desktop or mobile) is everywhere, and themselves, serve as a platform that does the heavy-lifting.

Therefore as platform builders, we can focus more on helping businesses map their needs through the use of data modeling, system configuration, deployment models (see Docker.io if you want to see an impressive technology) and of course: Business Intelligence and Data Analytics.

Ok, That’s The First One, Out with The Second

The second ingredient has to do with how we fit our client’s business model into the building blocks we just talked about.  Having done a lot of deployments across different business domains, we found that there are a lot of businesses out there whose data and information can fit the following model:

Building Blocks 800px

Basically, beyond infrastructure facilities such as security, user, configuration, email, role-specific dashboards, scheduling, notification system, full-text indexing which comes with every online platform worth its salt, we usually can organize data in the business in one or more grouping hierarchies, which typically ends with the main business object. What is this main business object? you may ask.

The main business object is typically tied to how the business serve their customers.  So for example, in the legal industry, the main business object would probably be legal cases, for a bookstore, the books, for a landscape-maintenance company, it’s probably the customer’s property, for a gymnasium, its member’s goals (lose weight, gain weight, get fit, etc.).  The Main Business Object is what most people associate with what the system manages.

Then predictably, almost all of the businesses that we came across associate the main business object with the same concepts such as trackable / auditable notes, links and tags, custom fields (who does not need a handful of these…?) and the ability to sort, search, and filter under various contexts.

Speaking of contexts, all of us know that chronological context is used all over the place, “when” something happens, and what else happens “after that” will always be the primary way in which we process information.  Thanks to the Geographic Information System (GIS) technological advances nowadays, Geospatial data is fast to become the next super-context which will find its place in many aspects of our lives.

Then after all this are put together using the building blocks, we begin the implementation of the proprietary process owned / copyrighted / trademarked by the client’s business or personally. We have been very blessed to work with innovative business owners whose brilliant process is just waiting for a good home to be built.

This process fits the definition of an Online Platform where construction and enhancements never really ends.  All of our platforms exhibit this characteristics.

In Summary

The combination of carefully selected building blocks and the right level of abstraction (at the business object level) gives us the ability to serve many clients simultaneously through our platforms.

This is not something that we take lightly, in fact every processes that we created and use takes this as one of the main goals, by which everything else is taken into consideration.  Why is this so important? Because this is the only way we can extend two biggest benefits to our clients:

  1. Continuous technical improvements, an improvement created for one platform can and will be replicated to the others
  2. Consistent quality and ease of maintenance, which translates into lower cost

Given that most of our platforms are priced in accordance to the number of usages, the growth of our clients’ business means growth for us as well. This is, to us, the prime motivation to go the distance towards excellence.

As I mentioned in my previous blog entry, we are living in the “golden-hours” of the Information technology, where the efforts of many finds its uses to benefit a much larger portion of the population than ever before.

We are the beneficiary of many-many hours spent by dedicated and incredibly smart people. At nextCoder, we intend to pass along these benefits to help as many business as possible out there. In other words, it’s time for the “secret-sauce” to be enjoyed by many, many more.

- Will Gunadi

School Districts are Putting Up Dashboards

Posted by admin on March 13, 2014  /   Posted in BI and Custom Development

Think Dashboards are only for Executives and Businesses?

Well, think again.

Today, even school districts are doing it.  First, let me be clear, it should come to no surprise that organizations within the education sector, especially the ones focusing on educating our children, should take advantage of any information technologies available today.

What really surprised me is how they use the Dashboard. Have a look here:

The Plano Independent School District (PISD) put up a dashboard which compares them with the neighboring cities’ ISD. The above image is just one of the elements on the Dashboard, by the way.  The rest of it looks just as impressive, as they seem to use every bells-and-whistles available in the charting world (not necessarily the best approach, but at least it is out there serving some purpose).

But while the Dashboard is put-together well, what impressed me the most is that the ISD gets it. They know that in order to get ahead of their peers, they not only have to perform, but also take the time to showcase their achievements.

This realization puts them — quite sadly — ahead of how most business owners think today.

Where does the data come from?

Or more importantly, how hard or how easy it is to put up the Dashboard, given the limitation of resources and information.  And this is what makes the situation even more ironic. We are living in the golden era of publicly accessible information. What is not available from within the organization, could easily be obtained either through some governing bodies or data collected independently.

And if a business owner still think that it is impossible for them to get the data about their competitors or customers or vendors/suppliers or everything else, that’s an outdated view and should be discarded.

What is it really for?

To make it easier for interested parties to make decisions. In this particular case, the interested parties are parents of students or young people who are looking for a place to move into or to start their lives.  By utilizing the Dashboard to serve relevant information in various dimension, the Plano ISD has make it that much easier for people to consider their options.

The same case can be made strongly for almost every businesses out there. When people are considering your products and businesses, the more ways you can showcase your distinguishing features, the more you can grab the mind of interested parties (read: potential customers).

Go Publish!

With today’s technologies, and the accessibility of potentially relevant external data, for example the General Society Survey, or  The Gapminder site, and many more, it is a crime not to have your own Business Dashboards. As the ISD’s have shown, an effort to present relevant up-to-date and relevant information is not limited to a one-shot deal, it is a continuous, planned, and deliberate program that should be included in your day-to-day business operations.

Business Dashboards – The Next Generation

Posted by admin on February 13, 2014  /   Posted in BI and Custom Development, Business, Data Best Practices

Dashboards Today

Unfortunately, most vendors that provide dashboard software have done little to encourage the effective use of this medium. They focus their marketing efforts on flash and dazzle that subvert the goals of clear communication.

They fight to win our interest by maximizing sizzle, highlighting flashy display mechanisms that appeal to our desire to be entertained. Once implemented, however, these cute displays lose their spark in a matter of days and become just plain annoying.

An effective dashboard is the product not of cute gauges, meters, and traffic lights, but rather of informed design: more science than art, more simplicity than dazzle. It is, above all else, about communication.

- Stephen Few, Information Dashboard Design, O’Reilly, 2006

Amazingly this comment is still pretty much true even today. But it doesn’t have to be. The next generation of effective dashboards are closer to a mixing panel of a music studio than plain, static dashboards we see today. And we at nextCoder are going to make sure that you business leaders are well-informed about it.

Recap: What is a Business Dashboard?

Nowadays you see fancy dashboards in cars that rival a desktop computer, almost. But what does the most basic dashboard in your car actually do? It tells you, the driver — in real time, at least these four information:

  1. How fast are you going (the odometer)
  2. How far can you go (tank full of gas or almost empty)
  3. Alerting you to pending breakage (weak battery, engine oil empty, engine needs checking)
  4. Alerting you of what is going on (turning signal, headlamp indicator)

Running a business, just like driving a car, requires your full attention in real time, but unfortunately, a business does not come with a built-in dashboard the way cars do.  Either you have to build one yourself, or have someone build it for you.

And just like a car dashboard that is connected at all times to the sensors that feeds it with data, a Business Dashboard is connected to the data sources located either within your business or coming from outside sources.

The information that you receive from a Business Dashboard is actually pretty similar:

  1. How fast is the business growing? (is it growing in the right direction?)
  2. How long can the business last given the current situation?
  3. Are we paying too much to our vendors/suppliers?
  4. Are we serving our customers the best we could?
  5. And many, many more

But … believe it or not, as useful as knowing all of these are, that list merely covers the basic usage of a Business Dashboard. If we stop at this level, we are missing the full potential of what the next generation of dashboards can do for us.

What is brewing?

Thanks to the ever-increasing popularity of game-changing web technologies such as jQuery, HTML5/CSS and the growing list of charting and/or visualization libraries, we enjoy the power of data visualization unlike any other computing era before.

The new way to develop dashboards allows us to produce dashboards that will rival web-applications in terms of information flow. Gone are the days of static, read-only dashboards. Say “Hello” to the new generation that takes interaction to a whole new level:

  • Built in mapping: With the numerous and Geocoding API available for us to use, it is unthinkable that a dashboard should be without one. Most businesses could benefit from geographically-mapped data. Imagine being able to visually see where your products are being purchased,  or your technicians on their service routes, or your suppliers to optimize material or component shipments.  And many more uses.
  • User Inputs: Ever seen a dashboard that is not read-only? If yes, you are in a good company. More and more executives, managers, customer support personnel ask that they are able to punch in data in real-time. Why do they have to switch to another application to do that? We at nextCoder agrees.
  • User-specific Business Rules: Lets face it, a business owner or CEO has his or her own “rules” that allow them to determine whether the business is doing okay or is it floundering.  These “rules” are not for everyone to see and for a very good reason: Panic prevention (just kidding… a little bit). But the fact remains, if a dashboard cannot even contain user-specific rules, then we are shortchanging the users. Plain and simple.
  • External Data: A lot of smaller businesses assume that just because their data volume is not gigantic, they do not have any use for aggregated data analysis. This was true in the past, not anymore. Today, there are volumes of data about any businesses, but it does not originate within the business itself, rather, the vast social-media network. You could be surprised at what your customers broadcast about your product or service to their friends and family, good and bad. And for the sake of your business, monitoring it is a good course of action.

These are just the tip of the iceberg when it comes to what a Business Dashboard is capable of serving with the current technology. Our goal is to deliver these features to our clients with each dashboard we build for them. In the next blog entries, we’ll have a peek on how to do just that.

Pentaho 5 CE Hands-on Review

Posted by admin on January 16, 2014  /   Posted in BI and Custom Development, Data Best Practices

One of the most exciting software release towards the end of 2013 is the Pentaho 5.0 CE (Community Edition) which was rolled out on November 18th 2013. While the EE (Enterprise Edition) was released a couple months prior, the CE version has always been my favorite both to work on and especially to be part of the community who is always full of new (and wonderful) ideas, and actually have the brain power to realize those.  Truly one of the most interesting Open Source communities.

As a BI Consultant, I had several requests to review this new release, so without further ado, let’s take a look.

As usual, I get the zip files for each Pentaho BI Suite components from here. This is what my folder looks like this when I was done downloading:

  • pad-ce-5.0.1-stable.zip – Pentaho Aggregation Designer (missing as of the time of this review, no idea where it went)
  • psw-ce-3.6.1.zip – Pentaho Schema Workbench
  • biserver-ce-5.0.1-stable.zip – Pentaho BI Server
  • pdi-ce-5.0.1-stable.zip – Pentaho Data Integration (Kettle and Spoon)
  • pme-ce-5.0.1-stable.zip – Pentaho Metadata Editor
  • prd-ce-5.0.1-stable.zip – Pentaho Report Designer

Unzipping any of these zip files will “install” the component. Simple as that.
I haven’t had the time to look at PAD or PME, so we’ll review this in the future. For now let’s start with PRD.

The new Pentaho Report Designer has a very convenient and useful item on the Wizard which you see when you started the report-designer.sh (or .bat on Windows) script. It’s called “What’s New”.

It’s basically a report that we can Preview and it listed all the new features in this 5.0 release; very handy to read about the improvements. What piqued my interest especially is they seem to improve the creation of interactive HTML reports, which now can serve links to other reports within Pentaho.  Maybe a new way to serve content that is somewhere between reports, dashboard, and wizard pages.

The popular Kettle (or Spoon or PDI) increase the number of steps including one that I have been waiting for: OpenERP Input and Output. Speaking of OpenERP, I need to contribute the custom OpenERP step that we developed last year into the community.

I’m also eager to try out the MongoDB steps as I started to use it for our projects.  I’ll have more to say about these two wonderful tools in upcoming articles.  These two are big enough to have their own reviews.

Pentaho BI Server

But the biggest changes are truly visible in the Pentaho BI Server itself. After unzipping the biserver-ce-5.0.1-stable.zip, dive into biserver-ce director and issue ./start-pentaho.sh if you are on UNIX or start-pentaho.bat if you are on Windows.

By starting the BI Server from this location, the starting scripts already set the memory allocation and other environment parameters to more reasonable values than the ones that comes default with Apache Tomcat.

After starting the server and wait for a while — or if you are familiar with Tomcat logging features, on UNIX do:

tail -f biserver-ce/tomcat/log/catalina.out

Which will allow you to see if the server starts correctly or failed with errors. On Windows, use the Tomcat Start/Shutdown application to see the logs. When you see the log files stops scrolling, bring up a browser (on the same computer) and try to hit the Tomcat server by entering http://localhost:8080/pentaho if you use the default settings. And you should see:

puclogin_custom_photo

Yes, that’s what our version of the Pentaho User Console (PUC) looks like after a couple of customization steps:

  • Change the login image:
    – user@server:~/pentaho5/biserver-ce/pentaho-solutions/system/common-ui/resources/themes/crystal/images$ mv ~/your-own-similarly-sized-image.jpg ./login-crystal-bg.jpeg
  • Change Pentaho to nextCoder
    – user@server:~/pentaho5/biserver-ce/pentaho-solutions/system/common-ui/resources/themes/images$ mv ~/your_logo.png puc-login-logo.png
  • Change the wordings of the Login page:
    – user@server:~/pentaho5/biserver-ce$ vi tomcat/webapps/pentaho/jsp/PUCLogin.jsp

Gone are the usual ‘joe’ user, replaced by ‘admin’ with the same default password ‘password’. Use these to get in and you’d be greeted by the Home screen:

home_screen

Again, with some modifications, you can tailor the Home screen to suit your purposes. In this case the customization step is:

  • Change the content of Home
    – beruin@yamato:~/pentaho5$ vi ./biserver-ce/tomcat/webapps/pentaho/mantle/home/content/welcome/index.html

If you notice, gone is the compartmentalized panes of the old PUC, replaced by a much better-flowing (plenty of white space) minimalistic-style layout.

Another paradigm switch is the central navigation (it says ‘Home’ in the above screenshot). When you click on it, a dropdown will be displayed showing the available mode.  The Home -mode is what you see above, next is the Browse File -mode that looks like this:

browse_file

This is another departure from the file-based pentaho-solution repository to this JCR-based one. What is JCR? Java Content Repository is a database-based content (files) repository specification that is implemented by among others Apache Jackrabbit project, which is the one being used here by Pentaho.

What does this all mean to users? In a way, it has its advantages being a database-based repository in terms of better control of metadata and versioning of the files without sacrificing ease of use, but which also means that we have to use a plugin if we want to synchronize this repository with a file system so we can Version Control our files on our own.  It remains to be seen if this switch will yield its fruit down the line.

One question for the Pentaho team: Why can’t I select multiple files and do some actions with them?

Speaking of plugins, which is source of productivity in the platform, the next mode we’ll talk about is the Marketplace -mode:

marketplace

In version 4.8 and before, we have to install plugins such as Saiku, CDE, CDA, CDF, etc. manually either by using the ctool-install.sh script or by unzipping files at the right folders and hope it’ll work.

The new Marketplace -mode provides a more organized way to manage plugins and their versions.  Although you still have to restart the server manually after installing or upgrading these plugins, it is still miles ahead and more importantly, just a month or two after release, we started to see plugins written by developers outside of Pentaho, which is wonderful and in-line with the spirit of the community.

Next up, is the Opened -mode, which is basically a mode where it retains all of the files we are working on (both editing or opening).  This mode is somewhat similar to the new Microsoft Office paradigm (starting with Office 2010).

The Scheduled -mode is an improved user interface to schedule ETL runs:

schedules

A new feature introduced is the ability to define block-out time, within which scheduled ETL will not be run.  This is useful for scheduled downtime or maintenance for the host servers.

The last mode is the Administration -mode:

admin_screen

This is the answer to “Where is PAC?” The old Pentaho Administration Console is gone, it is now reborn as this mode. I can’t tell you how many times (with the previous version) I received raised eyebrows or dumbfounded-look when I had to explain that you have to run another server just to create a new user or assign roles. This is definitely a very welcome improvement!

Now, how about some real work. The plugins now take center stage as Pentaho CE matures as a real platform. Old favorites like CDE:

cde_dashboard_edit

Improved with the much more professional-looking “Crystal” theme as the default. You could still switch to the old “Onyx” theme if you like.

Another good tool is Saiku Analytics, returned also thankfully:

saiku_graphsThe charting ability of Saiku Analytics has been improved tremendously. I  almost couldn’t believe my eyes when I see the various charts glide along visualizing the data effortlessly.

A promising newcomer in the Analytics tool called Pivot4J is also available to install through the Marketplace -mode:

pivot4j_editing

The Pivot4J has one thing that has been missing in all of the Pentaho Analytic tools, the ability to render Aggregates at the last row or column.  You have no idea how many times this little feature is asked by my clients.  Yes, business people loves their totals, those helped them to make better decisions.  So good job for this Pivot4J team!

Is there any negatives? Yes, the charting in Pivot4J is not intuitive to me. Take a look at the above screenshot, you see four columns. When you click the interface that will generate the bar chart representation of the table, what would you expect? I expect one bar chart, with four bars each representing the columns.  What did Pivot4J gave me? Four bar charts. Why?? And I don’t see any ways to merge them or change those in any way.

Summary

In summary, I couldn’t be happier with this new 5.0 release of the Pentaho CE. There is enough new features here that warrants companies to consider upgrading their Data Warehouses.  What is the most exciting trend for me is the third-party plugins that starts to become available through the Marketplace.  This can signal a real growth in quantity and quality of what is already one of the most useful BI suites in the market.

So to Mr. Pedro Alves and his team, big kudos, thank you, and good job. 2014 is looking like another stellar years for Open Source BI, starting with Pentaho 5.0 CE.

What You Should Expect from Your Data Warehouse

Posted by admin on October 01, 2013  /   Posted in BI and Custom Development, Business, Data Best Practices

When it comes to the benefits of having a Data Warehouse (DW from here on), most of our clients are already cognizant of its critical role as part of a business system.  What many still don’t realize is that there are good DWs and not so good ones.  Of course everyone is aware of their existence (good vs not so useful), but not everyone has the time thinking of the ways to distinguish between those two.

A Good Data Warehouse Goes (Far) Beyond a Database

The number one question that I get when I talked to anyone about the services we provide to our customers is the difference between a Database and a Data Warehouse.  A lot of people associate a data warehouse with storing data (just like a physical warehouse)  — and that’s about it.

Truth is, a good DW serves the business in at least five areas in addition to hosting business data:

  1. Automatically apply business rules
  2. Automatically generate reports (both on-demand and periodical)
  3. Automatically notify parties of data events of interest
  4. Allows data owners to look into their data in multiple dimensions
  5. Allows for data error detection, investigation, and corrections

When you are building a DW, the above functions must be present in the planning, implementations, and testing phases.  Otherwise, you’d be left with basically just a database.

A Good Data Warehouse Grows into Maturity

In the next blog entry, I’ll be discussing in specifics, the different levels of DW maturity. A lot of misconception out there regarding how to assess the reliability and usefulness of a DW. Most people expect a DW to become mature at the end of the initial implementation project. This is not realistic given the typical complexity of meta-data, data, and business rules surrounding it. The correct way to proceed with a DW-building project is to realize that there are certain phases that a DW goes through which brings it along the path into maturity.

A Good Data Warehouse Is Redundant

An experienced BI consultant would build one or more copies of the DW that can be switched at any given time.  Not only will this provide a good development and testing environment for analytic outputs (reports, dashboards, ETLs), but it would also

A Good Data Warehouse Highlights Data Problems

Due to its proximity with people who did the accounting, and forecasting in the organization, a good DW is usually the first place where these people notice data inconsistencies.   A good way to take advantage of this characteristics, is to put in place monitoring ETLs that runs periodically, executing queries that is designed to check the consistencies of data.

A Good Data Warehouse Reduces the Load on the Transactional System

A lot of the systems that maintains the transactional data input from users are inundated with processes that shouldn’t belong there in the first place.  These processes typically does checking, applying business rules, and monitor the validity of the data, which takes away resources from what Transactional Systems should be mainly about: Collecting data and preserving its integrity.

Now the follow up question is: How do we know which part of the Transactional System should be offload to the DW?

A good boundary rule-of-thumb is whenever a process can be executed against any data in batch-mode. This is an indication that it should be taken off the transaction system and moved to the DW. And by the way, it is not a taboo for a DW to pump data back into the Transactional System when a valid use-case can be made.

In Summary
There are many other ways to assess a DW to see if it’s “good” or “not good enough,” but all of them revolves around the relationship between the DW and the existing Transaction Systems and the other flip side of the coin, the decision maker team. A good DW should always bring clarity and bridge the communication barrier caused by incongruent view of data held by different parts of the business. Hopefully this can get you started in the correct line of thinking as you embark on your DW projects.


We serve businesses of any type and size
Contact Us Today