sandip's blog

Using a .forward to send mail to multiple accounts

If you put multiple addresses in a .forward file, a copy will be sent to each.

remoteuser1@domain.tld, remoteuser2@domain.tld, remoteuser3@domain.tld

If you want to keep a copy of each message in the original account without causing a .forward infinite loop, put a backslash in front of the account name.

\localuser, remoteuser1@domain.tld, remoteuser2@domain.tld, remoteuser3@domain.tld

Search bot report

Here is a simple bash script to get a daily report of search bot results of Success (200) and Failed (404) hits:

#!/bin/bash
# bot_report.sh
# usage: ./bot_report.sh [botName] [logPath]
# default: ./bot_report.sh Googlebot /var/log/httpd/access_log

######################################
# Run this in a daily cron           #
# 59 23 * * * /path/to/bot_report.sh #
######################################

# Commands
GREP=/bin/grep
DATE=/bin/date
MKDIR=/bin/mkdir
AWK=/bin/awk
SORT=/bin/sort
UNIQ=/usr/bin/uniq
TMPWATCH=/usr/sbin/tmpwatch
CAT=/bin/cat
MAIL=/bin/mail
ECHO=/bin/echo

# Global Variables
DEFAULT_BOT_NAME=Googlebot
DEFAULT_LOG_FILE=/var/log/httpd/access_log
if [ -z "$1" ]; then
BOT_NAME=${DEFAULT_BOT_NAME}
else
BOT_NAME=$1
fi
if [ -z "$2" ]; then
LOG_FILE=${DEFAULT_LOG_FILE}
else
LOG_FILE=$2
fi
PREFIX_LOG=`$ECHO $LOG_FILE | sed 's/\//_/g'`
TMP_LOG_PATH=/tmp/bot_report
TMP_LOG_FILE=${TMP_LOG_PATH}/${PREFIX_LOG}_`${DATE} +%F`.log
TMP_REPORT_FILE=${TMP_LOG_PATH}/${BOT_NAME}_report.txt
EMAIL=user@domain.tld

###########################
# Nothing to change below #
###########################

# Produce a temp file to work with for todays date
tmp_file_out() {
[ -d "${TMP_LOG_PATH}" ] || $MKDIR ${TMP_LOG_PATH}
if [ ! -f "${TMP_LOG_FILE}" ]; then
$GREP `$DATE +%d/%b/%Y` $LOG_FILE > $TMP_LOG_FILE
fi
}

# Clean up temp file older than a day
tmp_file_clean() {
$TMPWATCH 24 $TMP_LOG_PATH
}

# Create report
report_out() {
$ECHO "######## Success Hits ########" > $TMP_REPORT_FILE
$GREP " 200 " $TMP_LOG_FILE | $AWK -v bot="$BOT_NAME" '$0 ~ bot {print $7}' | $SORT | $UNIQ -c | $SORT -rn >> $TMP_REPORT_FILE
$ECHO >> $TMP_REPORT_FILE
$ECHO "######## Failed Hits ########" >> $TMP_REPORT_FILE
$GREP " 404 " $TMP_LOG_FILE | $AWK -v bot="$BOT_NAME" '$0 ~ bot {print $7}' | $SORT | $UNIQ -c | $SORT -rn >> $TMP_REPORT_FILE
}

# Mail report
mail_report() {
$CAT $TMP_REPORT_FILE | $MAIL -s "bot report: ${BOT_NAME}" $EMAIL
}

#
# Main
#
tmp_file_out
report_out
mail_report
tmp_file_clean

exit 0

php with mysqlnd support

You can now get the latest PHP with mysqlnd (MySQL Native Driver) support via remis' yum repository as mentioned in PHP-5.3,-zts-and-mysqlnd. The blog mentions that this is enabld in php-zts. However, no php-pecl extension are available and neither are some of the extensions thread safe. So I went about rebuilding php package from source for mysqlnd support.

  1. Download the source php rpm from http://rpms.famillecollet.com/SRPMS/ . Note, I have used php-5.3.2 which is the latest as of this writing.
  2. Install and apply the below diff patch to the spec file via `patch -p0 < {new_patch_file}` where "new_patch_file" has the below contents:

    --- php-5.3.2-remi.spec.orig 2010-03-11 23:07:04.000000000 -0600
    +++ php-5.3.2-remi.spec 2010-03-11 23:36:03.000000000 -0600
    @@ -24,6 +24,7 @@
    %global phpversion 5.3.2

    # Optional components; pass "--with mssql" etc to rpmbuild.
    +%define with_mysqlnd %{?_with_mysqlnd:1}%{!?_with_mysqlnd:0}
    %define with_oci8 %{?_with_oci8:1}%{!?_with_oci8:0}
    %define with_ibase %{?_with_ibase:1}%{!?_with_ibase:0}
    %if %{?rhel}%{?fedora} > 4
    @@ -677,6 +678,11 @@
    %if %{?fedora}%{?rhel:99} >= 10
             --with-system-tzdata \
    %endif
    +%if %{with_mysqlnd}
    +        --with-mysql=shared,mysqlnd \
    +        --with-mysqli=shared,mysqlnd \
    +        --with-pdo-mysql=shared,mysqlnd \
    +%endif
    $*
    if test $? != 0; then
       tail -500 config.log
    @@ -704,8 +710,13 @@
           --enable-dba=shared --with-db4=%{_prefix} \
           --with-xmlrpc=shared \
           --with-ldap=shared --with-ldap-sasl \
    +%if %{with_mysqlnd}
    +      --with-mysql=shared,mysqlnd \
    +      --with-mysqli=shared,mysqlnd \
    +%else
           --with-mysql=shared,%{_prefix} \
           --with-mysqli=shared,%{_bindir}/mysql_config \
    +%endif
    %ifarch x86_64
           %{?_with_oci8:--with-oci8=shared,instantclient,%{_libdir}/oracle/%{oraclever}/client64/lib,%{oraclever}} \
    %else
    @@ -725,7 +736,11 @@
           --enable-fastcgi \
           --enable-pdo=shared \
           --with-pdo-odbc=shared,unixODBC,%{_prefix} \
    +%if %{with_mysqlnd}
    +      --with-pdo-mysql=shared,mysqlnd \
    +%else
           --with-pdo-mysql=shared,%{_prefix} \
    +%endif
           --with-pdo-pgsql=shared,%{_prefix} \
           --with-pdo-sqlite=shared,%{_prefix} \
           --with-pdo-dblib=shared,%{_prefix} \
    @@ -756,6 +771,16 @@
           --with-recode=shared,%{_prefix}
    popd

    +%if %{with_mysqlnd}
    +without_shared="--without-gd \
    +      --disable-dom --disable-dba --without-unixODBC \
    +      --disable-xmlreader --disable-xmlwriter \
    +      --without-sqlite \
    +      --disable-phar --disable-fileinfo \
    +      --disable-json --without-pspell --disable-wddx \
    +      --without-curl --disable-posix \
    +      --disable-sysvmsg --disable-sysvshm --disable-sysvsem"
    +%else
    without_shared="--without-mysql --without-gd \
           --disable-dom --disable-dba --without-unixODBC \
           --disable-pdo --disable-xmlreader --disable-xmlwriter \
    @@ -764,6 +789,7 @@
           --disable-json --without-pspell --disable-wddx \
           --without-curl --disable-posix \
           --disable-sysvmsg --disable-sysvshm --disable-sysvsem"
    +%endif

    # Build Apache module, and the CLI SAPI, /usr/bin/php
    pushd build-apache

  3. Package with:
    rpmbuild -bb --with mysqlnd --define "rhel 5" SPECS/php-5.3.2-remi.spec
    
  4. To install via yum, change to the directory where rpms are located and recreate the repodata via:
    createrepo .
  5. Note: you may need to install some of the missing dependent devel packages. I used a combination of base, epel and remi repository to install the dependencies.

In place variable substitution with AWK

The content of the input file becomes stdin for rm and awk. rm ignores the input and removes the file, but its file descriptor remains open until both commands, rm and awk is complete. AWK process this "nameless" file and creates a new file:

  { rm $CSV_FILE && awk -F',' -v stid="$ST_ID" '$1 ~ stid {gsub(/&/,"",$7)}1' > $CSV_FILE; } < $CSV_FILE

  • "-v" sets the awk variable that is passed in via shell script variable.
  • "gsub" replaces with "" all occurrence of & in the 7th field. Use "sub" for single/first occurrence substitution or GNU Awk's gensub for more articulated substitutions.
  • "1" is a shortcut which means print the current record:

pecl runkit with php-5.2.x

As of writing, the current pecl runkit-0.9 package does not compile with PHP 5.2+ . Below is how I got the latest installed from svn trunk on CentOS-5.4 with php-5.2.9.

  • Install runkit:
          svn co http://svn.php.net/repository/pecl/runkit/trunk/
          cd trunk
          phpize
          ./configure
          make
          make install
  • Create "/etc/php.d/runkit.ini":
          extension=runkit.so
  • Restart apache and validate with phpinfo.

Note: you would need php-pear and php-devel installed already.

y2k10 bug in SpamAssassin

If running 3.2.x version of SpamAssassin... (check with `spamassassin -V`) there is a y2k10 bug in the FH_DATE_PAST_20XX rule, causing all mails getting an extra 3.4 points:

Reference: SA Bugzilla – Bug 6269

The fix mentioned is to update the rules using `sa-update` and restart spamd. The rules in the update directory at /var/lib/spamasassin will get updated.

With the fix applied you should no longer see FH_DATE_PAST_20XX in the logs.

For a manual fix, update /usr/share/spamassassin/72_active.cf and change:

header   FH_DATE_PAST_20XX      Date =~ /20[1-9][0-9]/ [if-unset: 2006]

to:

header   FH_DATE_PAST_20XX      Date =~ /20[2-9][0-9]/ [if-unset: 2006]

Related reading on sa-update: SpamAssassin RuleUpdates

Resolving df and du reporting different output

If df and du give different output of disk usage. Then, most probably it is due to "open file descriptors".

Run `lsof +L1` to get a listing of open files that have been unlinked or removed. Note the file size and kill or restart the respective service.

Reference: walkernews.net

HP Deskjet F2430 printing in Ubuntu-9.10

Although the printer HP Deskjet F2430 was automatically detected upon usb connection to my ubuntu-9.10 desktop, the 3.9.8 version of hplip that came via apt repository did not quite get printing to work.

I headed off to hplipopensource.com, downloaded and installed the latest driver (version 3.9.12 as of this writing) and got the all-in-one printer to work just following the install wizard.

I have yet to test the scanner... but that's going to be another update.

Extending ext3 partition

*** Make sure you have a back up prior to proceeding. ***

  1. Boot into rescue mode.
  2. Unmount partition if mounted and check disk:
    umount /dev/sda1
    e2fsck -vn /dev/sda1
  3. Delete /dev/sda1 partition and create a bigger one with fdisk:
    fdisk /dev/sda
  4. Recreate the partition /dev/sda1 with the starting point at the default location and the ending point at highest possible cylinder. (Note: if you are extending by merging two partitions, the data in the second partition is lost so make sure to backup data you need.)
  5. Run partprobe and resize2fs utility with no size arguments:
    partprobe /dev/sda
    resize2fs /dev/sda1
  6. Reboot and check everything in file-system is intact.

Shrinking ext3 partition

*** Make sure you have a back up prior to proceeding. ***

  1. Boot into rescue mode.
  2. Unmount partition if mounted and check disk:
    umount /dev/sda1
    e2fsck -vn /dev/sda1
  3. Remove journaling from ext3 partition and revert to ext2, as resize2fs does not work on ext3 partition. (Note: This step should not be necessary in recent kernels.)
    tune2fs -O ^has_journal /dev/sda1
  4. Force check the partition:
    e2fsck -vf /dev/sda1
  5. Resize the partition making sure that you don't shrink it lesser than the disk space currently used else you may lose data.
    resize2fs /dev/sda1 6000M
  6. Make a note of the blocks and block size. You can also run:
    dumpe2fs /dev/sda1
  7. Delete /dev/sda1 partition and create a smaller one with fdisk:
    fdisk /dev/sda
  8. Recreate the partition /dev/sda1 with the starting point at the default location and the ending point at number of blocks from the resize2fs output (1536000) multiplied by the size of a block (4K). So, the end point would be 1536000 * 4 = 6144000K.
  9. Run partprobe and resize2fs utility with no size arguments:
    partprobe /dev/sda
    resize2fs /dev/sda1

    Note: If resize2fs errors out, you may need to further increase the block size by a small percentage (3 to 5%).

  10. Run a disk check for the final time before restoring the journal.
    e2fsck -vn /dev/sda1
    tune2fs -j /dev/sda1
  11. Reboot and check everything in file-system is intact.

Syndicate content
Comment