Custom Software Development

Installing Perl modules on a UNIX virtual server when you don’t have root or administrator privledges.

Before you begin: In order to install modules, you have to have SSH or Telnet access to the server. If you don’t have this access (or you don’t know what SSH/Telnet means) then you’ll probably have to get your system administrator to install modules for you.

I recently wanted to install the DBD::CSV module on my virtual host server (UNIX FreeBSD) and found that it was more challenging than I would have liked. I’m not positive I have the best approach, but it did work for me in the end. Following are the steps that I had to take to get the module and all those on which it depends installed. The primary problem is that on the virtual server, I don’t have root/administrator privledges. So, I have to install Perl modules in an alternate directory. The scripts have to be altered to point to that directory as well.

To install my beloved DBD::CSV, I notice that I also have to have all of the following modules installed and working:

DBI
File::Spec
Text::CSV_XS
SQL::Statement

So, we will install those first, then finally install DBD::CSV.

First, I set up MCPAN. This is a Perl utility that automatically downloads, unpacks, compiles and installs Perl modules. Since I don’t have root, I was only able to utilize MCPAN for the downloading and upacking part. I still had to compile and install the modules manually. If you don’t have access to, or don’t want to bother with MCPAN, then you can just download the modules directly from http://www.cpan.org and unpack them yourself.

To run MCPAN, you enter something like this (this will start the process of installing DBI, the first required module):

perl -MCPAN -e 'install DBI'

The first time I ran this utility, I was automatically taken through an installation process. A directory ~/.cpan was created in my home directory and I was prompted with several questions. I simply used all the defaults. There was only one question that I had trouble with, which was something like “What is your favorite CPAN mirror.” This seemed to be asking where to look for downloadable modules. I wish I remembered what I put, but it took me several tries to find one that worked. I finally wound up entering a server in Japan, which I’m sure is not good. But it worked. You might check for possible server locations here: http://www.perl.com/CPAN

Anyway, after running that the first time, the DBI module installation files were saved in ~/.cpan/build/DBI-1.14. (in case you don’t know ~/ usually means your home directory), however the installation failed because it tried to write some stuff to the main Perl installation location and, again, I don’t have permission to write there. however, when you are manually installing modules, I do know that you can specify an alternate directory. So, I went into the ~/.cpan/build/DBI-1.14 directory and manually installed it like so:

cd ~/.cpan/build/DBI-1.14
perl Makefile.PL INSTALLDIRS=site INSTALLSITELIB=/usr/www/users/myaccount/cgi-bin/lib
make
make test
make install

When I installed it, I see that I got a few errors and such. Upon closer inspection, it seems that the POD documentation and other stuff is still trying to go in the main Perl library location. I ignore these errors, since it doesn’t seem to effect anything but the documentation. Obviously, you want to replace /usr/www/users/myaccount/cgi-bin/lib with the location on your server where you want to save the modules.

Anyway, now I try the File::Spec module:

perl -MCPAN -e 'install File::Spec'
cd ~/.cpan/build/File-Spec-0.82
perl Makefile.PL INSTALLDIRS=site INSTALLSITELIB=/usr/www/users/myaccount/cgi-bin/lib
make
make test
make install

That one was easy. I still got errors, but I just ignore them. Now I try Text::CSV_XS:

perl -MCPAN -e 'install Text::CSV_XS'
cd ~/.cpan/build/Text-CSV_XS-0.21
perl Makefile.PL INSTALLDIRS=site INSTALLSITELIB=/usr/www/users/myaccount/cgi-bin/lib
make
make test
make install

Same old thing. More errors. Moving along to SQL::Statement…

perl -MCPAN -e 'install SQL::Statement'
cd ~/.cpan/build/SQL-Statement-0.1016
perl Makefile.PL INSTALLDIRS=site INSTALLSITELIB=/usr/www/users/myaccount/cgi-bin/lib
make
make test
make install

Another one bites the dust. The final one, DBD::CSV is more tricky because when I tried the same thing, it doesn’t work. I can’t do the Makefile.PL part of it because it says I am missing some required modules. What is happening is that the installer is looking in the standard Perl path – however we have been installing all of our goodies to an alternate location and Perl doesn’t know about it. My solution is that I simply comment out the lines in Makefile.PL where it checks for all the required modules! That’s a crappy way to do it, but it works.

perl -MCPAN -e 'install DBD::CSV'
cd ~/.cpan/build/DBD-CSV-0.1024

Now I edit Makefile.PL and comment out these lines (about 20 lines down from the top or so)

# $ok &&= CheckModule('DBI', '1.00');
# $ok &&= CheckModule('Text::CSV_XS', '0.16');
# $ok &&= CheckModule('SQL::Statement', '0.1011');

That’s it. the rest is the same.

perl Makefile.PL INSTALLDIRS=site INSTALLSITELIB=/usr/www/users/myaccount/cgi-bin/lib
make
make test
make install

All done at last! I tried my script and it worked just fine. One adjustment that you might have to make to your script is to add the location of your libraries to your script. You do this by putting the location in a BEGIN clause in your script like so:

BEGIN {
unshift(@INC,"/usr/www/users/myaccount/cgi-bin/lib");
}

Put this at the top of your script before any other subroutines or anything. That tells your script to look there when loading modules. Since you use the “unshift” function, your path is added to the front of the list. If you would rather your script look in the regular places first, and look in your custom path last, then you can use “push” instead of “unshift.”

One final thing. I went into ~/.cpan/build/ and deleted all the files and folders. I get charged for disk space overusage, so no use keeping all those installation files, right?

Installing Perl Modules on MS Windows Servers (the Easy Way)

This article explains a bit about the process of installing Perl modules and why the typical instructions found in a module’s ReadMe file do not work on Windows servers. If you want to skip all of the details and just get to the installation part, skip to Option 1

Before you begin: In order to install modules, you have to have access to the DOS prompt on the server. If you don’t have access to the DOS prompt (or you don’t understand what this means) then you will probably have to ask your system administrator to install the modules for you.
The Problem:

Do these lines look familiar to you:

perl Makefile.PL
make
make test
make install

Of course, you’ve seen them in the ReadMe file for a Perl modules that you want to install on your Windows server. However, when you get to line 2 and try to “make” the module, you get a “Bad command or filename” error. (If your Windows PATH is not set right, you might not even get past line one!) You look a little deeper into the ReadMe and see that, oh yes, for Windows machines you might have to say “nmake” or “dmake” instead. Still, the results are the same – or you get a bunch of errors. Why is this? Well, the authors of Perl mods often don’t use Windows at all, so they don’t really bother to tell you some important details. In fact, they secretly want you to install Linux instead, so they purposely make it tough on you!

Important Detail:

The ReadMe file mentions the command “make” as if it will just magically work. However, “make” is not a standard DOS command. (It’s actually not even a standard UNIX command) It is a utility program that comes with a C compiler. Although you can get C compilers from a number of places, Windows does not come with one pre-installed. So, you can type “make” until your fingers get blisters, but it won’t do you any good.

Hey, I thought I was programming in Perl – why do I need a C compiler, you ask? Good question. The first reason is that this is simply a familiar way for UNIX admins to install software, even though there may not even be any code to compile. The second reason is that some Perl modules make system calls or other low-level stuff that is not available using pure Perl code. So, they write a some of the code in C to go along with the Perl module. You may already know that C code needs to be compiled into binary form for each specific operating system. It’s not possible for the module author to provide pre-compiled versions for every operating system known to man. So, they just give you the C source code instead and let you compile it yourself for your own particular system. This is no big deal for most UNIX admins because, as I mentioned, this is a familiar way to install software. In fact, UNIX comes with a C compiler all set up and ready to use.

For Windows administrators though, this procedure is probably unfamiliar. Windows software typically comes pre-compiled, so there is no need to compile it. Windows doesn’t even have a C compiler built-in, so you have to buy or download one on your own. It is possible that you are a Windows NT guru and have set up umpteen gigantic corporate networks without ever once compiling a single program.

The good news, holmes, is that you have 4 (count ‘em, 4) options for installing modules on windows. The bad news is that there is no garauntee that any of them will work! Some modules were programmed on UNIX machines and never tested on Windows. They may not even compile correctly on Windows at all without code modification. Hopefully this is not the case for your module, but be prepared to either just accept it, or get really involved in porting that module to Windows!

Option 1

Option 1 is to download nmake.exe from Microsoft, run the executable (which will extract 3 files) and save these 3 files in your Windows directory. Next, download UNIX Utilities for Windows and extract all the files. Locate tar.exe, gunzip.exe and gzip.exe and copy them to your Windows directory as well.

Now your machine is ready to do some mod installing! At this point you should be able to follow the regular Perl module installation instructions, except where it says “make” you type “nmake” instead. Cross your fingers and hope that the module installs properly!! If it does, you’re all set. If not, move on to Option 2 and hope for the best…

Option 2

Option 2 is to use the MCPAN feature that is built into Perl. To do that, you need to have already done Option 1, because MCPAN still requires nmake to be installed on your machine. so, at the command line, you type:

perl -MCPAN -e "YOUR::MODNAME"

(substitute “YOUR::MODNAME” with the name of the mod you’re installing, of course) The first time you run this utility, it will ask if you are ready to continue with the manual configuration. Unless you know your networking stuff pretty well, i’d just hit No at this point and let Perl auto-configure it for you. After that it will attempt to load the module from CPAN and install it. It will look for required modules and try to install those for you as well.

Option 3

Option 3 is actually easier than Option 1 and 2, but only if you have installed ActiveState Perl ( http://www.activestate.com/ ). ActiveState Perl is a nice Windows port and it includes a utility called PPM which is used to install modules. The modules that PPM supports have been compiled for windows and uploaded to the ActiveState module repository. (Note: If you installed Indigo Perl, they created a browser based interface for installing modules which is really nice – check the docs for that). For this step, you do NOT need to have done Option 1 because PPM used pre-compiled modules – you don’t have to compile them yourself. That is one nice feature if Option 2 is crashing during the compilation process.

To use PPM, just go to the DOS command line and type “PPM” (without the quotes). You will be at the PPM command promt. (I should mention that you need to be connected to to the Internet at this point).

At the PPM promt, enter “install YOUR::MODNAME” – you will be prompted if you want to continue. At that point, PPM will connect to ActiveState and see if the module you requested is available in a pre-compiled form. If it is, it will install the module and you are all done! PPM is especially nice because it will even install other required modules for you.

If you get an error message that the module was not found, then it’s not available from ActiveState. You’re once again out of luck. Try Option 1 and 2 if you did not already. Otherwise move to Option 4.

Option 4

Option 4 is a kind of last resort that may not really work. Basically, some modules force you to go about all the MakeFile business, but in reality it is merely one or more .pm files. If this is the case, you can often just copy the .pm file(s) to your C:\Perl\Lib or C:\Perl\Site\Lib directory. Make sure you maintain the directory structure, for example Crypt::RC4 would be saved in C:\Perl\Site\Lib\Crypt\RC4.pm. So you might have to create some sub-directories to maintain the structure.

That’s It!