Pentaho is an amazing system. Built upon countless hours of man hours from all over the world, it is one of the testament to the effectiveness of the open-source SDLC paradigm.

But here’s the rub, for those of us Community Edition users who can’t afford the Enterprise Edition, we are left on our own when it comes to the more “advanced” features … such as SSO integration.

After messing with this for the last week, with a lot of help from my colleague and probably one of the most fun (yes, fun!) online user community that I’ve dealt with (#pentaho irc channel), I finally cracked the proverbial nut.

So I decided to write up what I have found with my boss’ blessing.  Also with the rising awareness of the benefits of BI even for small to medium corporations, I have no doubt that this information would be useful for someone somewhere.

Due to variance in SSO setup, I am not implying that the way I set it up will work for yours. That’s all for the obligatory mini-DISCLAIMER.

The Need

If your organization does not have a Single-Sign-On implemented for your enterprise applications, then this writeup is irrelevant. The fact is, SSO is a useful feature for users (and developers too) that, while almost always a major pain to setup, the payback is usually worth the hassle.

In this writeup, my scenario revolves around trying to fit Pentaho (Community Edition), which I’ll refer to as PCE from here on, into an existing SSO implementation.

Now, for those who are familiar with setting up an SSO system, the next question will be a basic one:

Which SSO implementation did you use?

The SSO Setup

The one that we setup at work utilizes Microsoft Active Directory to authenticate users coming from the website.

While there are some documentation on the Pentaho Wiki on plugging in Pentaho into SiteMinder or CAS, less can be found when you search for Microsoft Active Directory. Which is a shame, because despite being a Linux guy myself, I have to admit that when configured correctly, their implementation of SSO, from the users’ perspective, works fairly well.

NOTE: You should see how my fellow programmer and sysadmin and my Kerberos teacher (that you, Earl) excel in this area of configuration.

With MSAD, once it authenticated you via the usual login screen, it will push information through AJP protocol (1.3) to the Tomcat server that hosts the Pentaho biserver-ce.

For the sake of brevity and clarity, we won’t be discussing how to setup AJP to work with IIS.  Suffice to say that it uses one of the ISAPI Filter extensions called: isapi_redirect.dll to accomplish this.

The first thing to do is to modify the conf/workers.properties and conf/uriworkermap.properties.  The way these works is routing by URL pattern.  It should be self-explanatory to modify, contact me if you need more info.

Gotcha(TM) #1: On my Ubuntu development server, somehow Tomcat AJP listener isn’t really listening to requests coming via tcp (that’s TCP for IP4 clients), rather it waits on tcp6.  And this is not obvious either, especially when you use netstat.  Netstat will show tcp6 whether Tomcat is listening to tpc *and* tcp6 OR just on tcp6.

So now what to do? After searching for answers online, I came onto a tip to specify the actual address where the server is listening on.   Somehow this forces Tomcat to listen to both tcp and tcp6 on 8009 (AJP protocol).  To be specific: add an attribute address=”your.server.ip.address” to the <Connector> tag that configures the 8009 port in the Tomcat’s server.xml

After that roundabout sidetrack, I finally was able to receive AJP requests  from IIS to Tomcat which in turn dutifully re-routes them to /pentaho where the PCE lives.

The Switch

At this point, all the PCE can do is to throw a fit because it does not know what to do with the AJP request coming from IIS (i.e the user) plus it has no idea that there is an authenticated user information within the request.

So we need to initiate the switch from the pre-installed JDBC-based authentication/authorization to the one based on LDAP, of which the MS Active Directory is an implementation.

To do this, you can follow the information in the link that I’m about to give you. But, come back here after you read it because while it listed the steps, it does not give you a clue on what those modifications are really for.  Well, unless you are an LDAP and Spring Security -expert.

Here’s the link.

In summary, here are the list of files you need to touch and modify:

Under biserver-ce/pentaho-solution/system directory:

• pentaho-spring-beans.xml – the big switch, this is where you tell Pentaho to use LDAP instead of JDBC authentication/authorization system.
• applicationContext-security-ldap.properties – this file basically is the center of the modification, we will talk about this file in depth on the next section.
• applicationContext-spring-security.xml – this is where ACL (Access Control Level) is setup at URL level.  Search for <property name=”objectDefinitionSource”>.  Scarily, the actual URL patterns and permitted roles are defined within a hardcoded CDATA block (!!).  What’s wrong with another .properties file guys?  All you need to do here is to substitute the default Pentaho roles such as Admin, Authenticated, etc. with the new ones from LDAP.
• applicationContext-spring-security-ldap.xml – this is where the majority of the values in the above .properties file are being used.  As far as I recall, I didn’t change this file at all, which is always a good thing.
• applicationContext-pentaho-security-ldap.xml – this file contains the two queries that populates the Pentaho UI when we select assign permissions to Users or Roles.
• pentaho.xml – this file governs who can do what to the solutions in the repositories.  All you need to modify in this file is replacing the default roles with the ones you define in LDAP.

Under biserver-ce/pentaho-solution/system/data-access-plugin directory:

• settings.xml – this file governs who can do what to the defined data sources (y’know, the all important source of  data for your ad-hoc reports and cubes).  All you need to modify in this file is replacing the default roles with the ones you define in LDAP.  A #pentaho irc channel community member (pstoellberger) helped me out on this one.  Without his quick source sleuthing, there’s no telling how many more hours I’d spend figuring this out.

Under biserver-ce/tomcat/webapps/pentaho/WEB-INF directory:

• web.xml – this file configures the pentaho web application.  Since we are plugging Pentaho into an existing enterprise application, we need to configure it to reflect this.  All you need to do is to make sure that this section of the file is properly defined (see highlighted part below):
  • <context-param><param-name>base-url</param-name>
  • <param-value>http://www.yourwebsite.entry.point.com/pentaho</param-value>
  • </context-param>

Now let’s talk about some of these modifications in more depth.

The Property File

applicationContext-security-ldap.properties to be exact.  This is the only property file that you need to modify for this purpose.

The values in this file are being used in three Spring-Security bean definition files.  To clarify, Pentaho uses Spring-Security to implement their authentication/authorization layer.  A wise decision that pays rather handsomely as you can see later in this article.

Let’s walk through these values:

• contextSource.providerUrl – this should contain your ldap: URL, between you and your sysadmin, this shouldn’t be a piece of cake to get.  Example: ldap://ldapsrv.acme.com:389
• contextSource.userDn – set the value to the distinguishedName (DN) of the read-only user that you use for accessing LDAP tree.  Example: CN=LDAP Searchdog,CN=Users,DC=acme,DC=com
• contextSource.password – the password of the read-only user above

This section takes care of the basic connecting to LDAP server.  This is used in various places as expected.

Next, we’ll fill out info for single user search (for authentication):

• userSearch.searchBase – this value points to the root of the LDAP tree where you want the search to commence from. Example: DC=acme,DC=com
• userSearch.searchFilter – this is the LDAP attribute that will be matched against the supplied parameter (typically the user name).  Example: (sAMAccountName={0})  <– the {0} is where the parameter would be substituted.

Next, we’ll specify how to fetch the roles of a given user (setup on LDAP):

• populator.convertToUpperCase – when this value is ‘true’ the roles coming from LDAP will be converted into all upper case.  Not sure what this buys us, but it’s important to be consistent.  Don’t set this to true and then forgot to capitalize the roles wherever it’s defined.
• populator.groupRoleAttribute – which LDAP attribute held the roles. Example: cn
• populator.groupSearchBase – same as the userSearch.searchBase above
• populator.groupSearchFilter – specifies the condition for the search, that is using the username  to get the roles he/she is associated with
• populator.rolePrefix – if you need a prefix, I haven’t found out why would I need one.
• populator.searchSubtree – another boolean value that indicates whether to search into the LDAP subtrees or not.

Lastly, we give the proper info for searching available roles in LDAP.  This is an important query that will actually populate the Pentaho UI where we select Roles to assign permissions to certain Reports or Cubes (or ‘Solutions’ if we use Pentaho’s lingo.

• allAuthoritiesSearch.roleAttribute – which LDAP attribute held the value for the roles.  Example: cn
• allAuthoritiesSearch.searchBase – where you’d want to search to begin.  IMPORTANT: the way my LDAP server is organized, when this property is set to the root of the tree (DC=acme,DC=com), the subsequent pentaho code failed to populate the UI control that allows us to select these roles.  Only when I specify a subtree that has only the roles, would this work.  Example: OU=Some Subgroup,DC=acme,DC=com
• allAuthoritiesSearch.searchFilter – this is the criteria that is shared by all the roles we want to pull from the LDAP server.  Example: (objectClass=group)

The Usage

Consistency is the keyword here, once you have defined a set of new roles in MS Active Directory to be used with Pentaho, then you *must* substitute default Pentaho roles (Admin, Authenticated, etc.) in the aforementioned configuration files with the appropriate new roles.

I don’t see the point of belaboring on this as the application would be unique to your own authentication/authorization needs.  Just be aware that a single typo will bring the system to a halt.  Involving some kind of version control is highly recommended when modifying these files.

The most unexpected and quite amazing fact in this whole thing is that Spring-Security automatically handles the authenticated user information that was sent from IIS to Tomcat without any intervention on my side.

Lesser security libraries would probably require some property tweaking or custom-written filters to do this seemingly trivial but important step.   This to me has proven one of the reasons for the maturity of Spring as one of the few Java frameworks that is truly enterprise -worthy.

The Loose Ends

Some miscellaneous random bit of info that may save you some time.  These would save me some had I known about it when I started on this task:

• The log file for Pentaho is located in: biserver-ce/tomcat/bin/pentaho.log
• To find out about problems with your ISAPI Filters, view the log files located where the extension .dll file is.  In my case it’s called isapi_redirect.log
• Turn the log4j.xml logging level for spring-security to INFO or even DEBUG to follow what’s happening if the modifications do not seem to take effect.  This is quiet obvious, but when you’re busy pulling your hair out, it’s easy to forget.
• Don’t forget to turn it back to WARN or ERROR when your modifications *do* work.
• Oh, and Pentaho Administrator Console is useless once you switched to LDAP, it is only configured to work with the JDBC user/role management.

Today I did. I forgot the password to the only user on my three-weeks old laptop. Forgot as in no way to remember it, and the hint didn’t help either. I am sure it’s not one of my regular ones because I was trying to be cute and came up with a new word.

And it so happen that in MS-Windows world, if you don’t have administrator account enabled, and you forgot your user password, you’re toast. I guess its the same thing with UNIX as well, except in UNIX, you will always have a root (same as administrator) user account.

Now what?

My help came from a rather weird source. Youtube. Where a lot of people apparently had solved the same problem. Most of them are .iso files which can be burned into a CD-RW and you boot into the CD instead of the hard drive. Ironically almost all of them are built around some kind of Linux distribution.

I tried one called Ophcrack which attempted to actually decode the Windows password hash. Of course after cranking the CPU to 100% for 15 minutes, it failed to retrieve my password. I didn’t say I came up with weak passwords :)

Finally I tried what’s called the Trinity Rescue Kit (just google it, it’s better than my linking to a website that may not be there when you needed it). A 149MB .iso files which I burned into a CD-RW disc.  It gave me a command prompt after booting on my laptop.

I proceeded to type:
winpass -u [my_username] and I was presented with a couple of options, one of which is to clear the password. I selected it and less than one second later it announced that my password has been cleared. Holding my breath, I issued shutdown -r now and waited.

And Voila! my laptop booted right into the account without asking for the password. Whew! that was a relief.

NOTE: Had I been able to enable the administrator account, I may have been spared from this exercise, but as I found out later, my version of Windows 7 that came with the laptop does not provide a way to enable the administrator account. Strange!

So  my thanks to those white-hackers who wrote the winpass utility and to those distro-jockeys who made it available in the .iso format.

And this time I remembered to create a password reset usb-keychain drive.  Without it, I’d be repeating this story all over again.

And sometimes thanks to our tinkering, we discover good solutions that are applicable to the task given to us at work.  Think about it as giving our employers a bonus.

One of the most important rules in running a server is to never expose unnecessary information publicly.  Want an example? how about an obvious one, your router’s administration application.  This should never be accessible from outside of your home network for an obvious reason, obviously.

But the benefit of this approach is also its own downside.  Consider the following (highly likely) case:

You need to urgently change a setting on your router, while you are not at home.

Yep, you can’t.  Not without doing something extra anyways.

And that something extra is SSH tunneling.  Now, there are at least two ways that I know of on how to accomplish this.  For simplicity’s sake, let’s talk about one now:

If you are on a Windows machine, get yourself Putty and follow the steps on this website while replacing the forwarded port numbers with the one that you are trying to use.

http://www.cs.uu.nl/technical/services/ssh/putty/puttyfw.html

Basically, you are telling Putty to tunnel port number x from a machine within your home network to port number y on the machine where you are working on.

So in the above example, to access your router’s admin application, you can setup the tunnel from port 80 on your router’s ip-address, to let’s say port 8080 on the local machine (where you are working from).

In the UNIX world, that translates into the following:

ssh -L 8080:your_router_ip:80 your_username@your_home_server.net

After successfully loggin into your home serve remotely, you can start a browser (on the machine where you’re working on currently), then go to localhost:8080 and voila! you’ll see your router’s administration application as though you are at home.

Pretty handy, eh?

If you were to ask me that a week ago, I would not be able to come up with a good business scenario.  Other than the fact that I did it at home, because my main computer is a Windows box and my development stuff is on my Linux server.

But at the moment I am stuck with an old laptop at work — which, being a good corporate laptop, of course is running Windows XP — because the giant four-letter computer company has messed up the shipment for my new computer.

So to make lemonade out of a lemon, I have enlisted two powerful free software that can get me out of this jam: PuTTY and Xming.

The idea is to use a powerful Linux machine to host and run the GUI programs that I need to work with from my very resource-limited old laptop (Pentium 4 Mobile with 512 MB, anyone?).

What applications? you say…

Let’s pick a big one, how about the Eclipse IDE.

Step one:

You install Xming and fire up XLaunch.  Out of the three modes that Xming offered: 1) multiple apps in one window, 2) each app running in a separate window, 3) fullscreen mode, mode 2 is one that works the best in this scenario.

So I went ahead and created an .xlaunch file that basically fired up Xming in the background, ready to accept X-window connections on the old laptop. NOTE: Don’t forget to check a box that says “No Access Control” otherwise your Linux application will be rejected when it tried to connect to Xming.

NOTE: Xming has a file called X0.hosts (that’s X and zero, not the letter ‘o’) which is usually located in the Xming install directory.  This file may need to be modified to include the IP address of the linux box.  But only if it connects to Xming via another network domain (other than the primary one).

For example, I setup a virtual linux guest on my windows 7 machine, the local network subdomain looks like 192.168.56.x unlike the default network domain which is 192.168.1.x.  The latter is where Xming waits for connections from.  Therefore I need to add 192.168.1.101 (the IP address of the linux machine) to the end of X0.hosts file (a line by itself), then Xming will work as expected.

A subnote, on my machine, windows 7 forces me to become administrator user to edit this file.  Although not as elegant as UNIX’s sudo command, the Switch User facility in windows makes this not as painful as it could have been.

Next, I fired up PuTTy.  I followed the steps to configure a PuTTy session that will allow X-forwarding and all that good stuff on this website.

Once you are connected to the Linux server, in which I already have downloaded the appropriate Eclipse distribution, ready to go.

All I need to do once I connected via PuTTy is:

1. Set the DISPLAY environment variable to point to the laptop’s Xming instance: export DISPLAY=192.168.x.x:10.0 (the 192.168.x.x being the laptop’s internal IP address, of course).
2. Run Eclipse IDE: ./eclipse &

And voila! Eclipse is running on the Linux server with every bit of its UI tunneled via X onto the old laptop.  Of course the performance depends on the network connection, but it is surprisingly snappy on a typical 100Mbps Gigabit Ethernet.  And I am talking about Eclipse loaded with a large project.

Now if this article does not make you run to the storerooms in the back and pull out those old laptops and give those out to your developers, I don’t know what else will :)

An Integrated Development Environment (IDE) is basically a multi-document text editor that assist you in managing a project.

Assist me how? you might ask.

Beyond reliably allowing you to edit your text files — be it your collection of barbecue recipes, love-letters, or your latest Cake PHP project — a good IDE should have the following features:

• File-type recognition.  The IDE should be somewhat aware of what type of file you are editing.  The most popular file formats are a must, in my book, such as .html, .php, .java, .c, .py, .xml, .css, et cetera.
• Syntax highlighting.  As part of the above feature, an IDE should highlight the common components of the said file type.   This is helpful especially when you are dealing with more than one type of files (typical if you are a software developer/web designer).
• Auto-completion.  Another form of file-type recognition is to provide suggestions when we write a couple of letters.  For example, if I type “clearT”, the IDE should do a lookup for functions visible from the current code context and present it to me.  In this case, “clearThroat()” and “clearTupperware()” should be selectable.
• Pair matching.  Pairs of what? Well, brackets, braces, HTML tags, multi-line comments, you name it.  Along with folding, this feature really makes figuring out your colleague’s code much easier.  Or your own code written six-months ago, for that matter.  Some IDEs out there go as far as auto-completing your brackets.  I find this annoying fifty-percent of the time, though.
• Refined indentation control.  Try highlighting a few lines of code in a basic text editor like Gedit or Notepad and then hit the ‘Tab’ key.  If you are a developer, unconsciously you’d expect the highlighted lines to be indented.  Instead, you are rewarded with a blank space.  A good IDE also allows you to select between tabs and spaces.  I like spaces, four of them per indentation level.  No, not Fortran, Python.
• Multiple-file Text Search. For you UNIX users, think of a built-in ‘grep’ command within the IDE.
• Terminal/Console facility.  At the very least, a good IDE should have a way to execute compilers or other command-line tools that you may need to run your text file through.  At the most, an IDE can act as a GUI for your favorite application/web server.
• Shortcut-keystrokes.  Preferably intuitive ones. As far as developer goes, I am an old-school, hands on keyboard, only reach for the mouse when needed -kinda guy.  I love shortcuts using key combinations.  They make tedious coding sessions a breeze.
• Scriptable Macro Engine.  If you know even what this means, you have been coding long enough.  One of the “badges” of an veteran programmer is to be able to brag about how they wire their text editors to jump through hoops and play dead at their whim.  Yeah, nerdy; but in a good way.

Also, some features that are “nice-to-haves” like built-in graphical debugger, version control integration, or library-dependency manager.

In short, if an IDE does not do enough to relieve you from the most mundane editing tasks, then it’s useless.  On the other hand, if the IDE is so complicated to use that you’re losing hair trying to figure it out, it’s worse than useless.

A good IDE should be intuitive.  If I issued an Open file command from an active document, then I should be presented with the contents of that same directory.  Some editors I use always show me what’s on my home directory.

A Few Good IDEs

Now let’s take a look at some good IDEs out there.

• Geany

This lightweight IDE runs on Linux and Windows. OS X and BSD users should be able to compile and run, in fact, I’d be surprised if there are not port install for it.

The lightweight-ness of this particular IDE cannot be stressed enough.  I am running it remotely inside Xming (an X-windows server for Windows) and I never noticed any performance hit at all.

Two things impressed me right of the bat:

1) it recognizes any file types I’ve thrown at it so far, and when I need to map a new file extension to an existing file format, I went to Tools > Configurations > filetype_extensions.conf.

It opened up the configuration file (formatted intuitively), add the new file extension, save and close, then Tools > Reload Configuration.  Done!

2) configurable shortcut-keystroke system, with defaults that make sense.  For instance, I can use Ctrl-PgDn to go to the next document on the right, Ctrl-PgUp to the left.  Just like how I navigate the tabs on Firefox. Very intuitive.

Most of the information that separates an IDE from an ordinary text editor is displayed on the lower panels.  Geany does not skimp on the  features.  As you can see below, the lower part of the IDE sports application status messages, compiler outputs, search outputs, a scribbling space, and a full blown terminal (or ‘console’ for you modern UNIX users):

A few features that I am missing: 1) a file browser/manager, 2) integration with souce-code version control systems such as SVN, 3) a scriptable macro engine.

Which, brings me to the last feature that I want to highlight.  Although billing itself as a lightweight IDE, the Geany developers are forward-thinking enough to embed a plugin infrastructure.  Perfect for implementing the ‘missing’ features I mentioned above.

It is my hope that as this IDE gets more popular, people would start writing plugins for it to make it even more powerful.  For now, I’m going to enjoy using it.


function mapName($th_name, $th_order)
{
return(array($th_name => $th_order));
}
$order = array_map("mapName", $_POST['thFilename'], $_POST['raw_orders']);
$order_collapsed = array();
foreach ($order as $k => $v) {
foreach ($v as $k2 => $v2) {
$order_collapsed[$k2] = $v2;
}
}


… I found out that I can achieve the same goal by doing this:


$order_collapsed = array_combine($_POST['thFilename'], $_POST['raw_orders']);


To make a lemonade out of lemons, I did learn quite a bit about Array handling in PHP.

Got to hand it to the PHP community,due to their maturity (I’m using PHP 5, not even the latest 6), they have quite a collection of some nifty stuff that you often need to do with arrays. It’s just a matter of knowing what’s in the API before reinventing the wheel.

I had the pleasure of reading through this book — yes all 1,000,000 pages in it — at my favorite Barnes & Nobles bookstore while preparing for a session. One and a half hour later, I find myself strangely educated.

Despite the bad reviews that I read about this book, I think it strikes just the right level of complexity and I appreciate the realistic problems that is tackled throughout the book. Such as how to measure the performance of your Hibernate layer via JMX.

This book is perfect for those who would like to have it as a reference that you can flip back and forth depending on which section of Spring Framework you’re tackling at the moment.

By definition, a Linux server is designed to host long-running processes unattended.  It is the opposite of a Linux-based desktop environment.

Yet a lot of time you do need or miss the convenience of GUI tools when working on your Linux servers.  Or sometimes you use your Linux server as your development platform; and development tools, unless you’re a masochist, are typically GUI-based.

A relevant example would be in a setting where you and your co-workers need to connect to a cloud-computing instance.  Obviously you want the minimal GUI without having to clog the network with unnecessary eye-candy and “smooth-rounded edges” of the mainstream GUI, or maybe you just prefer minimalistic and quick-response over polish.   Either way, you can use this article as a stepping stone to join the fun.

NOTE: You can apply this to any modern UNIX system as long as you can get to the programs that we are discussing below.

As the world’s most flexible and (arguably) powerful OS, UNIX and its derivatives do not disappoint in this regard.  Discovering the right combination of so many good programs out there is a big part of the thrill of putting together an environment.

In this article, I’m using Ubuntu 9.10 (Karmic).  You can substitute almost 90% of apt-get command with yum if you’re using Fedora, or emerge for those of you Gentoo wizards.

First Part: Server-side setup

The reason I came across this exercise is a few months ago, I was upgrading my server to the latest Ubuntu release.  A power outage shutdown the server when it’s in limbo-mode, rather than trying to recover from the unknown state, I decided to went ahead and re-install Ubuntu from scratch and rebuild the server.

TIP: I was able to “wing-it” because I have done the due diligence of backing up my data, otherwise, I’d have way less hair to worry about when combing.

Anyways, on to the history lesson (a practice of using UNIX history command to learn a great deal about how to setup a server):
3  apt-get update
4  apt-get upgrade
5  apt-get install apache2
10  apt-get install mysql-server-5.0
11  apt-get install libapache2-mod-auth-mysql phpmyadmin
19  apt-get install lamp-server^
20  apt-get update
21  apt-get install openbox
22  apt-get install obconf
23  obconf
30  vi /etc/xdg/openbox/menu.xml


Here we start by a pristine Ubuntu server install, nothing on it, and we add to it the LAMP stack, which gives us Apache 2.x, MySQL, and PHP.

Next here’s the lightweight window manager: Openbox. It’s not only a window manager that can run on top of Gnome or KDE, it’s can also run as a stand alone session. And this is the path we’re going in this article, since our goal is to go lean.

Two words: 1) ~/.xinitrc and 2) openbox-session. Put the second one inside the first one; and the next time you start Xwindows you’ll be running Openbox. Like this:

Follow this thread for much more details and interesting aspects of Openbox.

Next, what file manager?…