Skip to content

Archive

Category: Systems Administration

Backing up MySQL and the source code to your websites is absolutely critical as a systems administrator. I have been learning that while often you can rely on the MIS mechanisms that are in place for many large organizations, you never want to fully rely on them. Unfortuneately, we recently lost a small amount of data because the Oracle backups being sent to tape where not properly configured. I will take credit for the query that lost the small amount of data; however, it should have been easily recoverable.

In order to beef up the backup processes for our Linux server, I have now created a cronjob using a BASH script to backup the critical files and ftp them to an external server. In creating this script, I made use of the the scripts found on these two websites, and then added some additional functionality.

http://bash.cyberciti.biz/backup/backup-mysql-database-server-2/comment-page-1/#comment-500
http://linux.derkeiler.com/Newsgroups/alt.os.linux.suse/2005-12/msg01959.html

The script from this first website was incredible. Without having to update the script everytime you add a new database, this script will do a mysqldump and save them all in .gz format. My addition simply tarballed the source code for the websites including the http.conf file. Lastly, the script sends the files to a remote server using FTP.

Save the below code in a file called backupscript.sh. Then set your crontab to run it every evening – sh backupscript.sh.

#!/bin/bash

MyUSER=”databaseusername”
MyPASS=”databasepassword”
MyHOST=”databasehost”

# Linux bin paths, change this if it can’t be autodetected via which command
MYSQL=”$(which mysql)”
MYSQLDUMP=”$(which mysqldump)”
CHOWN=”$(which chown)”
CHMOD=”$(which chmod)”
GZIP=”$(which gzip)”

# Backup Dest directory, change this if you have someother location
DEST=”/backupdirectory”

# Main directory where backup will be stored
MBD=”$DEST/mysql”

# Get hostname
HOST=”$(hostname)”

# Get data in dd-mm-yyyy format
NOW=”$(date +”%d-%m-%Y”)”

# File to store current backup file
FILE=”"
# Store list of databases
DBS=”"

# DO NOT BACKUP these databases
IGGY=”test”

[ ! -d $MBD ] && mkdir -p $MBD || :

# Only root can access it!
$CHOWN 0.0 -R $DEST
$CHMOD 0600 $DEST

# Get all database list first
DBS=”$($MYSQL -u $MyUSER -h $MyHOST -p$MyPASS -Bse ’show databases’)”

for db in $DBS
do
skipdb=-1
if [ "$IGGY" != "" ];
then
for i in $IGGY
do
[ "$db" == "$i" ] && skipdb=1 || :
done
fi

if [ "$skipdb" == "-1" ] ; then
FILE=”$MBD/$db.$HOST.$NOW.gz”
# do all inone job in pipe,
# connect to mysql using mysqldump for select mysql database
# and pipe it out to gz file in backup dir
$MYSQLDUMP -u $MyUSER -h $MyHOST -p$MyPASS $db | $GZIP -9 > $FILE
fi
done

FILE=”$MBD/hostway_$NOW.gz”
tar -cvf $FILE /var/www/html/*
FILE=”$MBD/httpd_conf_$NOW.gz”
tar -cvf $FILE /etc/httpd/*

host=”ftpserverhost”
user=”ftpusername”
password=”ftppassword”
file1=”*.gz”
srcdir=”/backupdirectory/mysql”
FTPLOG=”ftp.log”
ftp -d -v -n $host < $FTPLOG; ERR=$?
user $user $password
lcd $srcdir
prompt off
mput $file1
bye
END
if [[ $ERR -ne 0 ]]; then
echo “Failure …$?”
echo “Debug – File $FTPLOG”
fi
exit
#End

I am not an expert at the PIX Firewall. In fact, I don’t even know if you could say that I am a beginner at it. I am certified at working with the Astaro; but even that firewall I am now rusty on as I have not been working with it for a couple of years now. But I was put into a position today where I needed to figure out how to add a new external IP Address to the PIX for it to be network address translated to the latest website I have been creating called http://easytrekking.com. Now you could easily ask why I did not simply use the Godaddy web interface created for the PIX firewall. Well, I use to. However, since I blocked off some of the non standard ports that Godaddy by default left open, I may have locked myself out of the web interface. After putting in a call to Godaddy, we determined that I still had ssh access and that I should be content to do the administration from the console. By the way, I am using PIX version 6.3.

Okay. The problem is that I have never before learned the scripting language used by the PIX. So this was my first attempt. It took me a few hours to figure out this little bit that I will show you here.

1. Connect to the firewall by SSH using the IP Address given to you by your provider – Godaddy in this case. It is interesting to note that you are not using the same IP Address as you would use with say your first website on your server. This is a difference between the PIX and the Astaro. Evidently, when setting up the PIX, you need one static IP Address for the firewall, and one static IP Address for your first website. At least, that is how it appears to me.

2. After logging in, type ‘enable‘. You will be prompted for your password again.

3. Type ‘configure terminal‘. Now you are able to make configuration changes to the firewall.

4. Type ‘show xlate‘. This will show you the existing network address translations currently at work.

5. Type ‘show running config‘. This will show you the settings currently in place. You need to analyze it to see exactly how yours is setup. In my case, I needed to add four lines.

pdm location 192.168.0.4 255.255.255.255 inside
pdm location 97.74.198.125 255.255.255.255 outside
static (inside,outside) 97.74.198.125 192.168.0.4 netmask 255.255.255.255 0 0
static (outside,inside) 192.168.0.4 97.74.198.125 netmask 255.255.255.255 0 0

Simply customize these four lines to match your own environment and drop them onto the console. Afterwards, once again type ‘show running config‘ and you will see your changes added.

6. Now type ‘clear xlate‘.

7. Type ‘show xlate‘ again and you will see the entries cleared. Now access at least one website on each of your IP Addresses and then type again ‘show xlate‘. You should then find your list populated again as well as seeing your new IP Address listed as being translated. Hopefully, your website on the new IP Address will show up in the browser as well.

Now, some of you may be questioning the wisdom of me posting something about my firewall. From where I stand, if you really want to hack my stuff, you will probably find a way. It just depends on how badly you want in. Names and IP Addresses have been changed to protect my stupidity.

Okay, so I was installing an SSL certificate using Godaddy.com as the certificate provider. I followed their instructions for generating the key, creating the CSR and downloading and installing the certificate. Then I go to restart apache and apache will not restart with the following error:

Enter pass phrase:Apache:mod_ssl:Error: Private key not found.

On my Redhat machine, I am not even prompted to put in the password. It simply fails on me.

While it would be preferred to find a way to configure the passphrase into the startup files, I am not able to figure out how to do it. However, I do learn that you can eliminate the passphrase from the key file and this is what many on the internet recommend doing. This is the command to eliminate the passphrase.

# openssl rsa -in www.key -out new.key

Thanks MNX Solutions…it took me an over an hour to find you.

http://www.mnxsolutions.com/blog/apache/removing-a-passphrase-from-an-ssl-key.html

Affect on Page Rank
Recently for Quest Software, we changed the domain name of one of the communities. It was a fairly new community that had plans to become one of the most prolific communities for Quest Software. Prior to the name change, the site already had the Google Page Rank of 3/10 showing up in the Google Toolbar. Not long thereafter, the new URL was showing up with a Page Rank of 0/10. I was quite ashamed as I was the one who recommended the change to the shorter URL. I felt that I had damaged the site. Then someone from outside of the communities pointed out that my redirects where not 301 redirects, they were 302 redirects.

Frankly, at this time, I had never heard of a 301 redirect. But being open to learn new things, I researched the 301 redirects and found this post by Matt Cutts, http://www.mattcutts.com/blog/seo-advice-discussing-302-redirects/.

In a nutshell, a 301 redirect is a permenant redirect. A 302 redirect is a temporary redirect. The new domain name recieved a 302 redirect from the original domain name and hence a penalty in Google probably for being duplicate content of an existing domain. The original domain name was simply being considered as temporarily offline.

So I dug in and figured out that the redirects I had set up in IIS did not have the “a permanent redirection for this resource” box checked. Problem solved? At least now I have the redirect set up properly.

Here is an example of what a 301 redirect looks like in IIS.

Here are a couple of examples of doing 301 redirects when using Apache / Linux.
http://trisummit.net/2008/07/01/using-htaccess-to-redirect-a-single-webpage/
http://trisummit.net/2008/05/30/how-to-redirect-a-website-to-remove-the-www/

It still took another 6 weeks or so for the page rank in the toolbar to change. It remained showing up in the toolbar with a ranking of 0/10 throughout this time. Then finally on July 31, it changed to a ranking of 4/10.

Affect on Search Engine Results
While I do not have clear knowledge of how the Quest site was affected in the search engine results, I do have a second example where I do know how it was affected.

My tire chain website had a name which I chose to get rid of. Originally, this obsolete website was the catalog for my new site. I was reluctant to ditch the orginal name at first for one reason–the placement on the long tail search results for “Rud tire chains” was good. While the results for “Tire Chains” was horrible on the original domain name being on page 35, the results for “Rud tire chains” was perfect in the number 2 spot on page 1–right behind the manufacturer’s web page. Now the new domain name showed up on page 15 for “Tire chains” – much better. However, it as in position 5 on page 1 for “Rud tire chains”. I did not want to lose that number 2 spot which the original domain name had. Also, I had already changed about as many of the links as I was going to get changed to the new domain name and I believe that mostly all of the link changes were already saturated in Google for about a two week time period.

I took a chance to see what would happen if I put up a 301 redirect from the old domain name to the catalog directory of the new domain name and placed all of the contents of the old site into the catalog directory of the new site.

There was no change in Google for about ten days. Then finally the 301 redirect kicked in with Google. The old domain name disappeared as I had expected. Then new domain name jumped up into the coveted number two spot that I was afraid to lose. I even jumped up two pages on Google for the keyword search of “Tire Chains” to where I am now on page 13. This was a total win for me. Now I have the domain name I like, the positive ranking of the old domain name, and improved ranking of the new domain name.

Conclusion
A 301 redirect is a critical piece that all SEO experts must be familiar with. Google appears to handle it very nicely. I have been warned in other blog posts that the other search engines may not be so friendly to the 301 redirect. Since the vast majority of my traffic comes from Google anyways, I am not all that concerned with it.

Ran into this issue where the section on the main page called Recently Updated was not updating.

The solution was to go into Administration -> Content Indexing and Rebuild the Index.

The official instructions for Confluence is that if that does not work, then remove the <confluence-home>/index directory, restart the application server and then rebuild the index.

http://confluence.atlassian.com/display/CONF27/Rebuild+index+from+scratch

This will be a very simple entry. If you need to redirect one web page to another web page and desire to get a 301 redirect, you simply add one line to the .htaccess file.

Redirect 301 /old.php http://domainname.com/new.php

If you do not have an .htaccess file, simply create the file in the directory that holds the html page and add that line.

This solution only works for Linux / Unix based servers. I use it on the Red Hat flavor of Linux machines.

My intent in this blog is to help jumpstart someone who is new to wikis and has been tasked to learn or prepare to deploy a wiki.

1. What is a Wiki?
A wiki is a collaborative tool used to allow a group of people to jointly contribute to a document. Typically, how it works is that say a person wants documentation on a software program. The company will task several individuals to write the documentation. But instead of using Microsoft Word, the IS group puts up a wiki and each of the writers write their content to a wiki page. Now eventually, the jobs of the writers will cross. They may wish to contribute to the pages that the other writers have written because perhaps they know something the original author did not know. The wiki allows all writers to do just that. They can put in thier own two cents wherever they want to in the document.

Now suppose the manager of the document does not like the content of one of the contributors, and he or she feels that the latest changes to the document where really messed up. No problem. The manager of the document will have the ability to roll back the document to any previous version. It is very difficult to break a document in a wiki because the managers always have the ability to take out the trash.

2. Which Software Package is Best
There are some really good potential packages you can use. This is the list of wikis I have now used.

  • Sharepoint
  • JSPWiki
  • PHPWiki
  • MediaWiki
  • Confluence

Sharepoint – Now Sharepoint is not just a wiki. Sharepoint is the collaboration tool that Microsoft wants everyone to use. It is an application that is quite customizable with a lot of out of the box web widgets that can create a really good project managment and organization tool for you. A Wiki is just one of its available web parts.

Personally, I am quite fond of Sharepoint. The last opportunity I had to project manage, I used Sharepoint as the team collaboration tool. All of the writeups for future reference where done with the Sharepoint wiki. For example, one person was tasked to make this integrate image support into the forums. After he built the functionality, he had to write up his specifications on a wiki page. Now, 6 months later, I still have his write up ready for reference. If things have changed, anyone with access can modify the contents.

The Sharepoint Wiki functionality is good for small groups. However, it is not near as robust as some of the other available wikis. What you get is a list of pages and the ability for anybody on the team to edit the page. There is no further organization of your wiki pages. If Sharepoint is your existing collaborative tool, I do recommend using it. However, if you are looking for collaborative wiki for wide scale use with a lot more really useful functionality, I would say that this is not the right tool.

JSPWiki – At first, I was rather impressed with this free open source package built on the Java Platform. I initially put it up for the Powergui team as a beta to see if it would serve their purposes for a wiki. We ran into some pitfalls with it.

  • It did not run well on all platforms. It did best in a Linux / Apache / Tomcat environment. I could not get it to work on Resin. Lastly, it seemed to cause instability when I combined it on the same tomcat application server with Jive Integrated running on a Windows 2003 server sitting behind IIS.
  • It was functionality deficient. You could roll back pages, but it did not have a good diff utility. Also, you could not set up email notifications for page changes. The users also complained about the inability to escape certain characters.
  • On the positive side, the support team including Janne Jalkanin were very helpful even though they were working for free. My hats off to them.

Unfortuneatly, we had to scrap using JSPWiki and turn to a different platform.

PHPWiki – After scraping the JSPWiki project, I downloaded PHPWiki and installed it on a Red Hat Linux ES 5.0 server. This did not go well at all. PHPWiki required some PHP_DBA functions in order to work. Those PHP_DBA functions are not in the out of the box RPM installation of PHP on Red Hat. I searched and searched for the proper RPM packages but could not find it. In order to get those functions up, I would have needed to uninstall PHP via the RPMs and reinstall php by compiling the source code with all of the required flags to make it work. That was just too much work for my feeble mind so I said “Forget It”!!! I have time on my hands, but not that much time.

MediaWiki – After failing now on two Wikis for the Powergui community, I dug up one called Media Wiki. Media Wiki is the wiki software used by Wikipedia. After my frustrations with PHPWiki, this one was a breath of fresh air. Installation was probably the simplest I have ever seen for any out of the box web application installation. This wiki is functionality rich having all the items that JSPWiki did not have. Administration is user freindly. It has good internationization support as well. The Powergui Team thus far has been rather pleased with this choice. I give this application two thumbs up for a wide scale collaborative tool – especially in terms of being an external wiki for documenting a software package. Also, this software package is a free open source application. Nice job!!!

http://mediawiki.org

Confluence – There are certain groups in Quest Software that make use of this Wiki made by Atlassian and I have recently been assigned to take over the administration of this application. It is a very good platform built on Java and is probably the most functionality rich of all Wikis. It includes watch support, tree structure organization, rich permission functionality, and even the ability to integrate with your active directory. It has the best user interface making it simple for anyone to use. It is very expandable and organizable and works well for large organizations with many teams. It also supports a rich scripting which for those who have taken the time to learn gives then far greater power at organizing content. This platform from my perspective is the darling of all wikis. The only weakness is that it does cost. Last time I checked it was about $7,500 for an unlimited use external collaboration license. If that is not a factor to you, you will not go wrong with this one. http://www.atlassian.com/

In the end, I give two thumbs up for both Media Wiki and Confluence.

Some instability on one of the servers I manage has brought up the need to setup automated monitoring that will send email alerts in the event that a website is down. In looking at possible solutions, I was considering several alternatives. Two of them that stood out the strongest are as follows:

1. Use Powershell on a Windows Server.
2. Use a scripting language on one of our externally hosted Linux servers.

Using Powershell did look like a viable alternative. The only problem is that I have never used Powershell before and I decided that the learning curve to implement a solution was not one that I am ready to go through. Maybe later. Nevertheless, Dmitry Sotnikov has a good blog where he describes how he did it for the http://powergui.org website using Powershell.

Monitor Website Availability

What I choose to do instead was use Perl and the Crontab to set up monitoring.

I tested the setup on three different Linux Servers–all three are Red Hat Linux Enterprise Servers version 4.0 or 5.0. Below is the script that I created using Perl:


$mailprogpath='/usr/sbin/sendmail';

use lib '/var/www/cgi-bin/libwww-perl-5.69/lib'; 
use LWP::Simple;

$location = "http://mywebsite.com";

if(get($location))
{
    print "success";
}else
{
    # need to send email
    open (MAIL, "|$mailprogpath -t") or die ("Can't access $mailprogpath\n");
    $recipient_address = "brian.nettles\@domainname.com";
    print MAIL "To: $recipient_address\n";
    print MAIL "From: donotreply\@domainname.com\n";
    print MAIL "Subject:  $location Down\n\n";
    print MAIL "$location request timed out.  Website may be down.\n";
    close (MAIL);
}

So on the Linux server, I went into the directory where I store my crontab’d files and created a file called monitorwebsites.pland populated the file with the above script. I then tested the script by calling the following:

perl monitorwebsites.pl

It works well.

On one of the servers, I had an issue with use LWP::Simple;. The perl script would not run on that particular server as the LWP module was not loaded. I found some good instructions on how to load that module at the following url:

http://forums.dixiesys.com/index.php?showtopic=2588

The next thing I needed was to schedule the Perl script in the Crontab. If you don’t know what the Crontab is, that is a scheduling device built into Linux/Unix systems to run automated tasks. I found some great instructions on how to setup the Crontab on Linux machines at the following URL:

http://kevin.vanzonneveld.net/techblog/article/schedule_tasks_on_linux_using_crontab/

The first command here opens up the crontab so that you can add an entry. The second line below is the entry that calls on the script every fifteen minutes.

sudo crontab -e
*/15 * * * * perl /directoryforscripts/monitorwebsites.pl

This was pretty simple. Took me about 4 hours total to research, write, and implement this.

The first question is whether you should use Tomcat or Resin as your application server. Here are some quick pros and cons with each.

  • Pro for Tomcat – Third party software is more likely to run on Tomcat than Resin
  • Pro for Tomcat – Sessions don’t get lost on server restarts
  • Pro for Tomcat – Tomcat is built on open source standards and has more documentation
  • Pro for Tomcat – Tomcat is free
  • Pro for Resin – Resin is a lot easier to integrate with IIS than Tomcat

I have now set up Tomcat to work behind IIS more than once. I know the challenges that go along with it. If you miss one small item on the list that needs to be configured, your installation will fail. And the logs don’t always point you in the right direction. It truely is a challenge to make Tomcat and IIS work together, but once it does, you will be glad you did it.

1. Install Tomcat as a Windows Service
The simplest place to start is with tomcat. Make certain you download the version using the Windows installer and go ahead and install it as a service. Make certain you remove all spaces that are in your installation. After Tomcat is installed, point your browser to it at http://localhost:8080/. You should then get your Tomcat home page. Now lets get IIS to work with it prior to adding your own application to tomcat.

2. Add the Configuration and Connector Files
Create a directory in the Tomcat Home directory calle connectors. In the connectors directory, add a subdirectory for the name of your application such as “root”. Now in this connectors/root directory, add two files.

1. isapi_redirect-1.2.26.dll (or whatever the latest version is)
2. isapi_redirect-1.2.26.properties

You will need to download this isapi_redirect-1.2.xx.dll file from apache.org. Currently the url is http://tomcat.apache.org/download-connectors.cgi.

A sample of the isapi_redirect-1.2.26.properties files contains the following:

# Configuration file for the Jakarta ISAPI Redirector

# The path to the ISAPI Redirector Extension, relative to the website
# This must be in a virtual directory with execute privileges
extension_uri=/jakarta/isapi_redirect-1.2.26.dll

# Full path to the log file for the ISAPI Redirector
log_file=C:\Tomcat-5.5.25\logs\isapi_redirect.log

# Log level (debug, info, warn, error or trace)
log_level=debug

# Full path to the workers.properties file
worker_file=C:\Tomcat-5.5.25\conf\workers.properties

# Full path to the uriworkermap.properties file
worker_mount_file=C:\Tomcat-5.5.25\conf\uriworkermap.properties

Now we have two more configuration files to add. These two you should place in the conf directory:

1. workers.properties
2. uriworkermap.properties

A sample of the workers.properties is given below:

# Define 1 real worker using ajp13
worker.list=worker1

# Set properties for worker1 (ajp13)
worker.worker1.type=ajp13
worker.worker1.host=localhost
worker.worker1.port=8009

A sample of the uriworkermap.properties is given below:

/*=worker1
!/images/*=worker1
!*.jpg=worker1
!*.xmls=worker1
!*.gif=worker1
!*.bmp=worker1

For more information on these two files, search the following two urls:
http://tomcat.apache.org/connectors-doc/reference/workers.html
http://tomcat.apache.org/connectors-doc/reference/uriworkermap.html

3. Make the Registry Entries
Follow the example on this image by going into clicking start -> run -> regedit and the entries just like on this image. Click on the image to get a larger view of it.

IIS-Tomcat-Registries

4. Configure IIS
Before you do anything with IIS, make certain you can see your web pages without interference from tomcat. Start by turning Tomcat and create a website and point it to the ROOT directory in the webapps directory of Tomcat. See if you can see the images when referencing them direct by url. ex. http://localhost/tomcat.gif. If you see the image, then you know IIS is working.

a. Add a new application pool that is copied from the default application pool and name is something you recognize – mysitenamePool.

b. Add a webservice extension and point it to the connectors.

web-service-extensions

c. Add a jakarta virtual directory

virtualdirectory

d. Make certain that every reference in this website to the application pool all point to the same newly created application pool.

pool

e. Setup the isapi filter by calling it jakarta and point it to the isapi_redirect-1.2.26.dll file. After doing so, restart IIS and make certain you have a green arrow in place for the filter.

jakarta

Now try the url http://localhost without the reference to port 8080. If you see the default Tomcat application, you are in business.

5. Add your own application
Turn off Tomcat and rename the ROOT directory ROOT_ORIGINAL. Now recreate the ROOT folder and add your application into the ROOT directory. Now you can restart Tomcat, Restart IIS and check to see if your application shows up at http://localhost

If it doesn’t work, carefully check all of your settings.

6. Heap Size
Sometimes, your newly established application ends up running out of memory and therefore you need to adjust the heap size. This is done in the registries. Take a look at the image here. You may need to add a couple of entries to the registries as displayed here.

heap

Ran into this issue where the section on the main page called Recently Updated was not updating.

The solution was to go into Administration -> Content Indexing and Rebuild the Index.

The official instructions for Confluence is that if that does not work, then remove the <confluence-home>/index directory, restart the application server and then rebuild the index.

http://confluence.atlassian.com/display/CONF27/Rebuild+index+from+scratch