Using a Script to Edit a conf File

Well, because I have been uber busy here at work, I haven't been able to write any blog posts. But, since I've been super productive at work, I get to write up a few posts about some more tools I've written to add to my arsenal. w00t!

So my latest task involved editing some of our web server's Apache confs. It consisted of adding a new alias so that we can call in some new app someone else wrote. Okay cool. Alias? Sure not a problem. Er...wait! The alias requires the site's directory name as part of the path! Oh no! That means I can't create one global alias!

Ah, see that's when I thought of writing this script that will look for a specific line in the vhost entries and will enter the new line after it. Pretty nifty eh? :)

So basically, what we have here is a list of all sites in a text file. Now we can use the site's directory name read in as a variable. It will then use awk to search for a specific line, and then print the new line you need right below it!

So if you take a look at the code, I want to add this line:
"Alias /newalias /mnt/www/$site" (where $site is the site directory name variable)
right after each line that fits the regex that searches for any line that has "Alias" and "reports" in one line followed by the site dir name at the end of the line. Obviously if you want to utilize this script you should edit the regex to your needs.

And that's it! Now think of all of the time saved rather than adding a new line manually to EACH vhost entry on EACH server! whoa....scary...


#!/bin/bash

exec < sites.txt
while read site ; do
echo "processing $site..."
awk '{print $0} '"/Alias.+reports.+\/$site"$/'{print " Alias /newalias /mnt/www/'"$site"'"}' httpd.conf > newhttpd.conf && mv newhttpd.conf httpd.conf
echo "done!"
done


Shell Scripting to Read Data From a File

Okay, so here's your problem: you have a file that contains data that you'd like to parse through a script. It's a lot of data and you don't want to sit there and continuously do the same thing over and over!

The data can be set up as one long column of data or multiple columns of data. In this case for simplicity I'll only use one column of data.

So let's say your data file is as follows:

dog
cat
cow
...
...
(long list of data)


These represent parts of filenames that you need to clear out. You don't want to sit there typing cat /dev/null > dog
cat /dev/null > cat
... So you come up with a master plan to script the process. The script will be short, sweet, and absolutely time saving!

You first need to read in the file; so this line does the work for you: exec < files.txt. Obviously, in this example my "files.txt" is the filename that contains the data I want to read. After you've opened up the file for reading, you can now parse through the file with a while loop:
while read file ; do
[do some work here]
done


Now note, "file" is the variable that we're assigning to the column. You can call the variable whatever you want. Also remember that I am being simple with this example and only have one column of data. If I wanted to do multiple columns of data, I would just assign more variables to each column. e.g. while read col01 col02 col03 ; do where $col01, $col02, and $col03 will be my new variables.

That's it! I'll leave the customization to you. Cheers!

Here is the full script example:
#!/usr/bin/bash

exec < files.txt

while read file ; do
if [ -s /mnt/$file ] ; then
cat /dev/null > /mnt/www/vdir/z/ec/$file
else
echo "/mnt/$file already empty!"
fi
done


XML RFC822 and W3CDTF Validation

I've been tasked to clean up XML pages at work. We streamline the process with a perl script (just like many of our other publishing tools) and then a nice neat XML file is produced.

So, I've been spending my time lately validating XML feeds on FeedValidator.org (very useful validator, bookmark it!). Apparently, some of our date formats were not valid. We weren't following the proper RFC822 format for the "pubDate" tag and W3CDTF format for the "dc:date" tag.

Luckily with localtime and strftime, this process is easy. So, I wrote a new subroutine for our perl module and here we have it!

use POSIX qw(strftime);



sub Now2
{
my $a = shift;

if ($a == 1) {
## returns in RFC822 format:
## e.g. Tue, 30 Sep 2008 12:57:06 -0400
my $now_string = strftime "%a, %e %b %Y %H:%M:%S %z", localtime;
return $now_string;
}
if ($a == 2) {
## returns in W3CDTF format:
## e.g. 2008-09-30T13:00:35-0400
my $now_string = strftime "%Y-%m-%dT%H:%M:%S%z", localtime;
return $now_string;
}
else {
return;
}
}


You basically call the subroutine specifying a "1" or "2" depending on which date format you want it in. So for example to get a W3CDTF date format, call the subroutine like this: Now2(2)

Cheers!

xargs Is a Very Useful Tool

I was recently asked how to remove specific files of a specific extension (.tmp in this case). Well, of course you can find them by doing a find . -type f -name "*.tmp". Easy eh? But how to delete them if they span deep into the directory tree? Well, I suppose you can just append the output of your find command into a text file and then have a script do the rm'ing of the files. But, what if you can do all of that work in one command line?

So, I mentioned xargs and mentioned what it can do and gave a brief example and lo and behold, he saved a good 5 minutes of extra work. Awesome!

Now, to exemplify what I just said:
find . -type f -name "*.tmp" | xargs rm

Just think of all the possibilities now that you have all of this functionality! You can cp, tar, mv, etc. etc.

Enjoy! Cheers!

First Android Phone Available October 22

At last, Google/T-mobile have announced a release date of October 22, 2008 for the HTC Dream (aka T-mobile G1) in a press conference today. The phone is priced at $179 with a 2-year contract with T-mobile. I'm curious how much it is without those restrictions...hmmmm...

One major concern for me is the tether-ability; From Ars Technica, "In response to a question about whether the device will be tetherable to a laptop, the company said that the G1 was 'meant to be used as a mobile device, not as a tethered modem.'" Now, I actually love being able to tether my Nokia N810 to my phone just for the accessibility of ssh, email, and the web. Though, I suppose I should be able to get the same functionality via the HTC.

I'm sure another big issue at hand for companies thinking about rolling this device out to they're staff is the support for MS Exchange, which "there is currently no Exchange compatibility". Though, this does not seem to worry any prospective buyers who only want access to their Google Apps as Gmail, Google Maps, etc. will work perfectly fine.

I'm sure the iPhone lovers are curious to compare the similarities and differences that stand out. Right now, it looks like an Amazon (DRM-less) MP3 store and an "Android Market" will compete with the iPhone's iTunes Store and Apple App Store respectively.

I have to admit, I've been anxious about Android ever since rumors flooded the web about a "G-phone", so I'm admitting to being a bit biased. Though, I think that the true test will be when users get their hands on these devices.

Now when does my current cellular contract end... :)

Cheers!

References: T-Mobile, Google finally unveil the first Android phone

Remove ^M Characters in vi

Alright, I've previously blogged this before, but if you can tell from my previous post, I was blogless for a while so, now to fill this new blog with new and old posts.

Anyhow, I'm sure any of you Vi guys & gals out there have run into this wonderful issue of having "^M" appear at the end of each line. This is especially true if you work with *nix and Windows or if your co-workers use Windows.

So, to quickly get rid of this in vi enter the following:
:%s/ctrl+V ctrl+M//g

It will display as: ^M

*Note: Do NOT type in shift+6 shift+m. Remember these are special characters and they need to be entered as so.

That's it! And Bob's your uncle!

Alternatively, you can use sed to make the search/replace. Notice the vi code looks pretty much the same as the sed arguments.

Cheers!

Lorem Ipsum

So, I usually hosted my own sites in the past, but my leisure time to keep up with hosting issues has become a hindrance. And, well I got a little lazy.
So, the internet was calling me back and I needed to write and now here we are. Also, I figure if/when(?) I get my own site up and running again I can easily just have this blog's RSS feed into it. So everything works out!

Anyhow, this will be my "lorem ipsum" post as my blog looked desolate and lonely. :-)

Cheers!