Outbound network traffic with multiple interfaces

Why does Red Hat Enterprise Linux 6 invalidate / discard packets when the route for outbound traffic differs from the route of incoming traffic?

Issue Description
Why does Red Hat Enterprise Linux 6 invalidate / discard packets when the route for outbound traffic differs from the route of incoming traffic?
Why does Red Hat Enterprise Linux 6 differ from Red Hat Enterprise Linux 5 in handling asymmetrically routed packets?

Solution posted at access.redhat.com/site/solutions/53031

Red Hat Enterprise Linux (RHEL) 6 Resolution

Temporary change
To accept asymmetrically routed (outgoing routes and incoming routes are different) packets set “rp_filter” to 2 and restart networking, by running the following commands:

echo 2 > /proc/sys/net/ipv4/conf/default/rp_filter
echo 2 > /proc/sys/net/ipv4/conf/all/rp_filter

Persistent change
To make this behaviour persistent across reboots, modify /etc/sysctl.conf and make the following change prior to reboot:

net.ipv4.conf.default.rp_filter = 2

Root Cause

RHEL6 (unlike RHEL5) defaults to using ‘Strict’ Reverse Path Forwarding (RPF) filtering.

The sysctl net.ipv4.conf.default.rp_filter selects the default RPF filtering setting for IPv4 networking. (It can be overridden per network interface through net.ipv4.interfacename.rp_filter).

Both RHEL6 and RHEL5 ship with a default /etc/sysctl.conf that sets this sysctl to 1, but the meaning of this value is different between the RHEL6 and the RHEL5 kernel.

Microsoft Shortcuts and Notes

Searching within Outlook 2010

Useful keyword searches:

category:=”Red Category”
read:no Items that have not been read. You can also use read:false to get the same results.
category:business Items that are categorized as business.
messagesize:enormous Items whose size is larger than 5 megabytes

Windows 7
Launch command window based on specific folder

Open command window here

Make your own Reduce File Size presets for PDF export

Within Preview there is a filter that can be used to reduce the size of PDF files (think of PDF files that are 600 DPI high resolution).  Unfortunately it produces very poor quality images to the point of being unusable. Fortunately there is a way to create and install your own custom quartz filters for use in Preview that give large file size reductions while maintaining good quality.

After some googling, I found a perfect article that explains why the default Mac OS X Reduce File Size filter produces terrible quality images .. and how to fix that:


The filter, which is just a XML file, can be edited with any text or programming editor then saved to the  /System/Library/Filters  directory with a unique filename.  The Reduce File Size (Good) filter is what I use .. rather than posting as a code block and messing around with escaping the XML so the code displays correctly, the file is available for [download here].

Simply download the contents of this file, ensure it is renamed to a .qfilter file, then copy into the system filter directory (so it is available for all users). I chose to use /System/Library/Filters/Reduce File Size Good.qfilter. You may need to be a Mac OS X Administrator to write this file into the shared system library folder. At this point, in Preview you can use this filter to reduce large scanned PDF files by almost a magnitude of order.

Here is the text of the original post:

Make your own Reduce File Size presets for PDF export
Jul 05, ’12 07:30:00AM Contributed by: zpjet

I was never satisfied with results of “Reduce File Size” Quartz filter when trying to make some PDFs smaller before sending them by e-mail. It made them too small, and the graphics were fuzzy.

I eventually found where these filters are:


I was delighted to find out they’re XML files easily editable with TextEdit (or any other text editor). I also found why this particular filter makes quite unusable PDFs, as these parameters were just too low:

Compression Quality 0.0
ImageSizeMax 512

So I copied this file to my Desktop, and then made two more copies of it, and called them Reduce File Size Good, Better and Best. Then I changed the parameters of each file to 0.25, 0.5 and 0.75 for Compression Quality, and used these three values for ImageSizeMax:

842 (that’s A4 at 72dpi)
1684 (A4 at 144dpi)
3508 (A4 at 300dpi)

Finally, I changed the default string for the Name key at the end of each file to reflect the three settings, so they display the names I have given them in the menu.

Then I copied them to a /Library/Filters folder I created (for some reason, ~/Library/Filters doesn’t work in Lion) and now when I open a picture or PDF in Preview, I have the option of four different qualities for reduced file sizes.

As an example, I have a JPEG of scanned A4 invoice at 300dpi and it’s 1.6MB. When exporting to PDF in reduced size, the file is only 27 KB and it’s quite unusable – very fuzzy and hard to read. The Good one is much easier to read, slightly fuzzy and still only 80 KB. Better is 420 KB and clear, and the Best is 600 KB and almost as good as the original even on a laser printer.

Apple Airport Utility 5.6 on Mountain Lion (10.8.4)

Apple has been continuously dumbing down the AirPort Utility to the point where their default AirPort Utility (version 6.x) is virtually useless.  Settings such as syslog destination, NTP settings, etc can only be set with AirPort Utility 5.6 – problem is it cannot be loaded on Mountain Lion (10.8.4).  Discussion here at https://discussions.apple.com/thread/4172563?start=15&tstart=0 shows how to install AirPort Utility 5.6.1 on ML without clobbering version 6.

In case Apple removes that thread (written by Douglas Urner), here it is:

Just in case you’d like to use AirPort Utility 5.6.1 on Mountain Lion (and probably Lion as well), here’s how to install it:

  1. Download the disk image (you can find it here:http://support.apple.com/kb/DL1536).
  2. Mount the disk image and drag the install package (AirPortUtility.pkg) to your desktop.
  3. Fire up Terminal and prepare to show off…
  4. Make a temporary directory and cd into it: mkdir tmp ; cd tmp
  5. Extract the Payload file from the install package with xar, here’s the command: xar -x -f ~/Desktop/AirPortUtility.pkg Payload
  6. The result will be a directory named AirPortUtility.pkg (just like the file, but now you can move into it to get the files you want). Inside will be a file called Payload that is a compressed archive of AirPort Utility.app.
  7. So our next move is to extract the app. Here’s the command: gzcat AirPortUtility.pkg/Payload | tar -xf –
  8. When it finished there will be three new folders Applications, Library, and System. Your nice new copy of AirPort Utility 5.6.1 will be in the Utilities folder inside of the Applications folder. Use Finder to rename it (assuming you want to keep version 6 as well) then drag it to your Applications/Utilities folder.
  9. The other two folders hold the AirPort Base Station Agent and its supporting files. I’m not sure if you need/want these or not. As best I can figure the agent does two things: it checks for updates for AirPort Utility and it monitors AirPort base stations for problems. You probably already have a version running as it comes with the system and it seems to know how to talk to both versions of AirPort Utility (I got nagged about updating).
  10. The final step is to launch AirPort Utility and confirm that it works. You’ll probably want to go into preferences and turn off the option to check for updates. If all is good you can remove the temporary directory: cd .. ; rm -rf tmp (or drag it into the trash with Finder).

That pretty much does it. I hope it helps somebody out there.

Seagate Disk Replacement and RAID1 mdadm Commands

So I’ve had to replace a Seagate disk yet again and spent a frustrating amount of time on their website looking for a link to their warrenty replacement page >> http://www.seagate.com/support/warranty-and-replacements/

At least this time, I’m using Linux software RAID for a RAID1 mirroring configuration. When I determined there was a disk failure, I used the following mdadm commands to remove the bad drive:

# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sda1[0](F) sdb1[2]
5139084 blocks [2/1] [U_]
md1 : active raid1 sda2[0](F) sdb2[2]
9841585344 blocks [2/1] [U_]
unused devices:

– Fail and remove all /dev/sdb partitions (/dev/sdb1, /dev/sdb2)
# mdadm --manage /dev/md0 --fail /dev/sdb1
mdadm: set /dev/sdb1 faulty in /dev/md0
# mdadm --manage /dev/md0 --remove /dev/sdb1
mdadm: hot removed /dev/sdb1
# mdadm --manage /dev/md1 --fail /dev/sdb3
mdadm: set /dev/sdb3 faulty in /dev/md1
# mdadm --manage /dev/md1 --remove /dev/sdb3
mdadm: hot removed /dev/sdb3

– Shutdown and replace the bad disk (assuming you have been able to replace with the exact disk)
– Copy the partition table from the surviving disk
# sfdisk -d /dev/sda | sfdisk /dev/sdb

– Re-attach the partitions from /dev/sdb to the RAID1 mirrors:
# mdadm --manage /dev/md0 --add /dev/sdb1
# mdadm --manage /dev/md1 --add /dev/sdb2

You should now see the md devices syncing up by:
# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sda1[1] sdb1[2]
5139084 blocks [2/1] [U_]
[======>.......] recovery = 49.3% ...

Once the sync completes, install grub on both the drives again:
# grub
grub> root (hd0,0)
grub> setup (hd0)
grub> root (hd1,0)
grub> setup (hd1)

Great reference pages:

How to reset the enable password on a Cisco ASA 5505

How to reset the enable password on an ASA 5505:

The following procedure worked for me to reset the enable password.

Connect to serial port – typically 9600,8,N,1.  On my MacBook Pro, I use a Keyspan USB-Serial adapter, so my command line is:

screen /dev/tty.USA19Hfd13P1.1 9600,8

You can eventually use <ctrl-A><ctrl-\> to kill the screen session.

Power on the device.
When it prompts to interrupt boot sequence, do so (press ESC).

It should prompt

rommon #0>

Type in:
rommon #0> confreg

Should show something like:

Current Configuration Register: 0×00000001
Configuration Summary:
boot default image from Flash

Do you wish to change this configuration? y/n [n]:

Press n (don’t change)

We can have the ASA boot a default config with no password by setting register flags 0×41, so do this:

rommon #2> confreg 0×41
rommon #2> reboot

You now can login as the password has been removed (use <return> as the password).  Be sure to set the enable password with:

config t
enable password new-password-here
config-register 0x1

Ensure you either use the config-register command or interrupt the boot sequence again and reset the boot flags back to 0x1, otherwise the boot loader will continue to boot the default configuration – ignoring your configuration.


Unix, Linux and Mac OS X Notes

Here’s some notable command syntax I use. You can also select the Notes category and you’ll get more specific topics such as Linux LVM and Mac OS X commands.

rsyslog options

Forward syslog events to external host via UDP:
– edit /etc/rsyslog.conf .. add a stanza like the example at the end of the file .. a single @ = UDP forward, @@ = TCP forward

$WorkDirectory /var/lib/rsyslog # where to place spool files
$ActionQueueFileName fwdRule1 # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList # run asynchronously
$ActionResumeRetryCount -1 # infinite retries if host is down
# remote host is: name/ip:port, e.g., port optional
*.* @

– restart the rsyslog daemon
systemctl restart rsyslog.service
service rsyslog restart

Mac OS X syslog to remote syslog server

Forward syslog events on Mac OS X 10.11 to external syslog server via UDP or TCP:
– edit /etc/syslog.conf .. add a line at the end of the file .. a single @ = UDP forward, @@ = TCP forward

*.* @
# remote host is: name or ip:port, e.g., port optional

– restart the OS X syslog daemon
sudo launchctl unload /System/Library/LaunchDaemons/com.apple.syslogd.plist
sudo launchctl load /System/Library/LaunchDaemons/com.apple.syslogd.plist

Write ISO image to USB on Mac

– plug in USB to Mac
– lookup disk number
sudo diskutil list
– unmount the USB
sudo diskutil unmountDisk /dev/disk2
– copy ISO image to USB
sudo dd if=CentOS.iso of=/dev/disk2

NIC MAC change

Changing MAC address of NIC
– RedHat stores this in: /etc/sysconfig
You need to edit the hwaddr in /etc/sysconfig/hwconf and HWADDR in the other locations (some are links).

ssh tunneling of syslog traffic

– Example SSH configuration for tunneling a syslog TCP stream from a remote server back to a local node:

Remote node has TCP client process (rsyslog) running, we want it to write to a local TCP port (15514/tcp), and have that local port forward to the local node we have initiated the ssh connection from to a syslog daemon listening on port 1514/tcp:

Remote node rsyslog.conf:

Event flow is through ssh on the remote node, listening on 15514/tcp and forwarding to the local node via ssh tunnel launched on the local node:
$ ssh -R 15514:localhost:1514 remotehostusername@remote.hostname.domain

To complete the picture, we probably want some sort of process on the local node to detect when the ssh connection has been lost and (1) re-establish the ssh connection, (2) restart rsyslog on the remote host to re-establish the connection from the remote rsyslog daemon to the ssh listener on port 15514/tcp.

YUM Software Repository

– Manually add DVD location/repository by: Using a Red Hat Enterprise Linux Installation DVD as a Software Repository

To use a Red Hat Enterprise Linux installation DVD as a software repository, either in the form of a physical disc, or in the form of an ISO image file.

1. Create a mount point for the repository:
mkdir -p /path/to/repo

Where /path/to/repo is a location for the repository, for example, /mnt/repo. Mount the DVD on the mount point that you just created. If you are using a physical disc, you need to know the device name of your DVD drive. You can find the names of any CD or DVD drives on your system with the command cat /proc/sys/dev/cdrom/info. The first CD or DVD drive on the system is typically named sr0. When you know the device name, mount the DVD:
mount -r -t iso9660 /dev/device_name /path/to/repo
For example: mount -r -t iso9660 /dev/sr0 /mnt/repo

If you are using an ISO image file of a disc, mount the image file like this:
mount -r -t iso9660 -o loop /path/to/image/file.iso /path/to/repo
For example: mount -r -o loop /home/root/Downloads/RHEL6-Server-i386-DVD.iso /mnt/repo

Note that you can only mount an image file if the storage device that holds the image file is itself mounted. For example, if the image file is stored on a hard drive that is not mounted automatically when the system boots, you must mount the hard drive before you mount an image file stored on that hard drive. Consider a hard drive named /dev/sdb that is not automatically mounted at boot time and which has an image file stored in a directory named Downloads on its first partition:

mkdir /mnt/temp
mount /dev/sdb1 /mnt/temp
mkdir /mnt/repo
mount -r -t iso9660 -o loop mount -r -o loop /mnt/temp/Downloads/RHEL6-Server-i386-DVD.iso /mnt/repo

2. Create a new repo file in the /etc/yum.repos.d/ directory:
The name of the file is not important, as long as it ends in .repo. For example, dvd.repo is an obvious choice. Choose a name for the repo file and open it as a new file with the vi text editor. For example:

vi /etc/yum.repos.d/dvd.repo


The name of the repository is specified in square brackets — in this example, [dvd]. The name is not important, but you should choose something that is meaningful and recognizable. The line that specifies the baseurl should contain the path to the mount point that you created previously, suffixed with /Server for a Red Hat Enterprise Linux server installation DVD, or with /Client for a Red Hat Enterprise Linux client installation DVD. NOTE: After installing or upgrading software from the DVD, delete the repo file that you created to get updates from the online sources.

IP Networking

– Manually add IPv4 alias to interface by:
ip addr add dev eth4
– Manually remove that IPv4 alias to interface by (note the subnet mask):
ip addr del dev eth4
– Manually add route for specific host:
route add -host gw

pcap files

– Split large pcap file by using command line tool that comes with Wireshark editcap:
editcap -c 10000 infile.pcap outfile.pcap

tcpdump options

Display only packets with SYN flag set (for host and NOT port 80):
tcpdump 'host  &&  tcp[13]&0x02 = 2  &&  !port 80'

Mac OS X (10.7)

sudo /usr/sbin/sysctl -w net.inet.ip.fw.enable=1
sudo /sbin/ipfw -q /etc/firewall.conf
sudo ifconfig en0 lladdr 00:1e:c2:0f:86:10
sudo ifconfig en1 alias netmask
sudo ifconfig en1 -alias
sudo route add -net

rpm commands:

List files in an rpm file
rpm -qlp package-name.rpm

List files associated with an already installed package
rpm --query –-filesbypkg package-name
How do I find out what rpm provides a file?
yum whatprovides '*bin/grep'
Returns the package that supplies the file, but the repoquery tool (in the yum-utils package) is faster and provides more output as well as do other queries such as listing package contents, dependencies, reverse-dependencies.

sed commands:

Remove specific patterns (delete or remove blank lines):
sed '/^$/d'
sed command matching multiple line pattern (a single log line got split into two lines, the second line beginning with a space):
cat syslog3.txt | sed 'N;s/\n / /' > syslog3a.txt
– matches the end of line (\n) and space at the beginning of the next line, then removes the newline

awk commands:

Print out key value pairs KVP separated by =:
awk /SRC=/ RS=" "
Print out source IP for all iptables entries that contain the keyword recent:
cat /var/log/iptables.log | egrep recent | awk /SRC=/ RS=" " | sort | uniq
Sum column one in a file, giving the average (where NR is the automatically computed number of lines in the file):
./packet_parser analyzer_data.pcap | awk '{print $5}' | sed -e 's/length=//g' | awk 'BEGIN {sum=0} { sum+=$1 } END { print sum/NR }'
Find the number of tabs per line – used to do a sanity check on tab delimited input files
awk -F$'\t' '{print NF-1;}' file | sort -u

sort by some mid-line column

I wanted to sort by the sub-facility message name internal to the dovecot messages, so found the default behavior of sorting by space delimited columns works.

sort -k6 refers to the sixth column with the default delimiter as space.
sort -tx -k1.20,1.25 is an alternative, where ‘x’ is a delimiter character that does not appear anywhere in the line, and character position 20 is the start of the sort key and character position 25 is the end of the sort key.

This sorts by the bold column:
$ sort -k6 dovecot.txt
Oct 7 09:09:31 server1 dovecot: auth: mysql: Connected to (db1)
Oct 7 09:34:03 server1 dovecot: auth: sql(user1@example.com, Password mismatch
Oct 7 09:33:36 server1 dovecot: auth: sql(someuser@example.com, unknown user
Oct 7 09:15:27 server1 dovecot: imap(user1@example.com): Disconnected for inactivity bytes=946/215256
Oct 7 09:21:11 server1 dovecot: imap(user2@example2.com): Disconnected: Logged out bytes=120/12718

dos2unix equivalent with tr

tr -d '\15\32' < windows-file.csv > unix-file.csv

Fedora 16 biosdevname

– Fedora 16 includes a package called “biosdevname” that sets up strange network port names (p3p1 versus eth0) .. since I don’t particilarly care if my ethernet adapter(s) is(are) in a particular PCI slot, remove this nonsense by:

yum erase biosdevname

– to take total control of network interfaces back over (edit /etc/sysconfig/network-scripts/ifcfg-eth?)

– remove NetworkManager

yum erase NetworkManager
chkconfig network on

WordPress notes for pomeroy.us

Production site is www.pomeroy.us
Development site is dev.pomeroy.us

– webserver root directory is /var/web
– production node is called prod
– development node is called dev
– WordPress database is called wpdb

Procedure to copy production WordPress instance to the development node:
1. Copy webserver www root dir via a tarball
tar czf prod-20110808.tgz /var/web

2. Dump the WordPress database to a MySQL dmp file:
mysqldump -u$mysqluser -p$mysqlpass wpdb | \
 gzip -c > prod-20110808.dmp.gz

3. Copy these two backup files to the dev node:
scp prod-20110808* user@dev:.

On the development node:
4. Unpack the webserver tarball:
mv /var/web /var/web.previous
cd /
tar xzvf prod-20110808.tgz

5. Drop the WordPress database and restore the new version:
mysql> drop database wpdb;
mysql> create database wpdp;
$ gunzip prod-20110808.dmp.gz
$ mysql -u$mysqluser -p wpdb < prod-20110808.dmp

6. Update the WordPress 'siteurl' and 'home' options to point to the development node:
update wp_options set option_value='http://dev.pomeroy.us' where option_name='siteurl';
update wp_options set option_value='http://dev.pomeroy.us' where option_name='home';

Should be all done!

Building a new PVR

<Updated Aug 18, 2011 after a successful PVR rollout>

Technology has evolved since the last MythTV PVR I built, as chronicled here.  Here’s the latest techniques and tech that I’ve used to (start) build(ing) my current PVR. I’ll update this article as I go, as there’s been some bumps along the way, so completion of the project has been slower than I anticipated.

Requirements for my new PVR include:

  • Linux operating system for cost and flexibility reasons
  • Quiet! Fan-less operation if at all possible, external power supply ok
  • Small form factor, black case to fit in with my current home theater gear
  • Video capture with MPEG-2 hardware acceleration to help keep the CPU needed as small as possible, in an expansion card format for the most compact physical footprint .. additionally there must be at least two independent tuners
  • Analog tuners, but would be good if they were capable of digital for when I eventually move to digital/HD
  • IR receiver and transmitter capability for easy remote control and ability of the PVR to use my current set-top box as a source (gives me all the cable company movies and channels that are not available via the basic cable connection
  • Ability to schedule at least 10 shows and retain 5 episodes of each show .. also ability to schedule based on show name alone
  • Ability to perform post-recording processing, such as removing commercials or changing formats
  • Should be able to use a pre-packaged distribution for most if not all of the functions .. I know it’s a home-brew, but I’m tired of messing with individual packages, firmware, and custom codes to make it work. Using a distribution package makes it easier to maintain through updates.
  • Want to purchase the parts from the same supplier if possible (ended up using newegg.ca)

Since I already run MythTV, it was an obvious starting point and given I don’t have an affinity to a specific Linux distribution, I looked at Mythbuntu and Mythdora since I’m familiar with and already run both Ubuntu and Fedora distributions.

After downloading the Mythbuntu 10.10 ISO disk image, I discovered I didn’t have my USB DVD drive, so I wanted to create a bootable USB flash disk.  I followed the excellent instructions at https://help.ubuntu.com/community/Installation/FromUSBStick and successfully burned a bootable Mythbuntu disk on a 2GB USB flash disk via a Ubuntu VM running on my MacBook Pro.

Continue reading

MySQL Notes

MySQL Command Line and Configuration Notes

Drop tables with wildcard:

There are multiple ways to specify MySQL credentials, this is not the best, but simply an example of how to drop tables using a wildcard pattern. In this case, command line history such as .bash_history will store your MySQL username and password plaintext, and an extended process listing will also reveal both username and password. When run from the command line like this, the SQL commands and the credentials are not stored in the MySQL history file (.mysql_history).  On closed (private) systems, the risk is low, especially if you clean up after these maintenance activities by purging the command histories.

mysql -u user -p password database -e "show tables" | grep "table_pattern_to_drop_" | awk '{print "drop table " $1 ";"}' | mysql -u user -p password database

Update WordPress home URL

There are times when moving or copying WordPress blogs from one server to another, the owner may want to update the URL associated with the specific site.

A simple MySQL update can match the WordPress blog to a new site URL:

mysql> select option_value from wp_options where option_name = 'siteurl';

| option_value                   |
| http://www.example.com |
1 row in set (0.00 sec)

mysql> select option_value from wp_options where option_name = 'home';

| option_value                   |
| http://www.example.com |
1 row in set (0.00 sec)

mysql> update wp_options set option_value='http://server.newsite.com' where option_name='siteurl';

Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> update wp_options set option_value='http://server.newsite.com' where option_name='home';

Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

Epitome of bad software

There is a reason many people loathe Microsoft software. Before you consider flaming me for that statement, I realize all software has flaws, bugs and eventually crashes. In my experience, even if it’s patched and up to date, the following image happens FAR too frequently with Microsoft software.
Microsoft bugs
I don’t recall having the same issues with Concept Draw, even with complex diagrams. Since I’m just tired of having to redo work over again, good-bye Visio, I’ve just purchased your replacement.

Resetting WordPress user passwords

Resetting WordPress 3.0 user passwords can be done directly within MySQL through the following procedure.  This assumes your installation of WordPress stores user passwords in the wp_users table as MD5 hashes and the unique site prefix for all WordPress tables in MySQL is _x.

Connect to the database via your favorite GUI (phpMyAdmin, Navicat) or command line with either the WordPress role account or any other MySQL user account with select and update privileges on the WordPress database:

update wp_x_users set user_pass = MD5('123abc890') where user_login = 'administrator';

This will update the password for user ‘administrator’ to ‘123abc890’.  Once this has completed, either flush the wp_x_users table or exit the tool used to access the database to cause the updates to be committed.  Sign into WordPress with the new password and optionally change the password via the user interface.

Apple Exemplifies Fine Software Engineering

So I’ve been a recent Apple user for a mere eight years, when I purchased my first iBook  running the new OS X (10.1). I’m a fan of the form engineering that goes several steps beyond the basic function engineering that is so prevalent in consumer technology these days. For Apple, it’s not good enough that there’s windows, they have to look good too – like a master craftsman that puts finishing touches on the product rather than just slapping some cheap molding on and calling it done (or Windows).

This is too fine for words.

After working through successively newer notebooks (iBook, PowerBook, MacBook Pro), I have recently upgraded my first gen MacBook Pro to a new uni-body MBP. All the way through the online store (with the complication of being a grad student and navigating the education part of the online store), the process was pretty painless. But the real wow was when my new MBP showed up three weeks ago and I decided to use the Migrate function to just suck the contents of my old MBP to my shiny new uni-body MBP (thanks for the encouragement, Jonathan). I figured since I didn’t have the time or energy to setup another computer from scratch, I would try this migrate feature – with a heavy dose of battle earned skepticism. When I turned on the power on my new MBP, it seamlessly guided me through the setup .. and asked me if I wanted to migrate from an existing Mac or even a TimeMachine backup of a Mac.  I said yes, hooked the old and the new together .. fully expecting this to not end well and have to restart some install process.  Well a little while later, the migrate was done .. I restarted my new MBP (didn’t have to), and it looked exactly like my old MBP. All of my Applications were there. All my documents where there. iTunes was there. iPhoto was there. The positioning of the icons and documents on my desktop was exactly like my old MBP. Wow. A migrate function that actually worked.  Really. All the way.  Ok, well I did have to re-setup my home wireless connection .. for some reason that didn’t seem to come across, but with the totally customized settings I use, I’m not too surprised although it only added about 120 seconds onto my migrate time.

So at the time I’m writing this, Apple has announced the next generation of the MacBook Pro (the Intel i5 and i7 processors).  Since I’ve only had my shiny new uni-body MBP for a week, I call the folks at Apple and speak to a very pleasant customer service rep (send me an email or website message and I’ll forward his name), who not only cheerfully agrees to accept my new MBP back, but helps me order the new generation. They waived the return shipping and any refurbishment fees, as well as the express shipping for the new unit to me.  Gives me his direct line so if the Apple provided UPS return sticker expires before I get the old-new MBP migrated to the new-new MBP, I can call and get a new label. All this (and I ordered a new mouse) and they refunded a net of nearly $900 back to my credit card.

Well, I’ve just finished the migrate from the old-new MBP to my new-new MBP and again, it was seamless. I don’t think I’ll rebuild a new Mac from scratch any more – this is just too fine for words.  So I can get back to my Master’s thesis and life in general, and not worry about the software out there that is half baked or just barely good enough to get by .. with lots of manual care and feeding.

Thanks Steve and crew – this is why I’m an Apple shareholder.

Accessing Ubuntu desktop from Mac Snow Leopard

Accessing my Ubuntu 9.04 Gnome desktop from the built in Mac OS X 10.6.2 VNC viewer took a bit of tweaking on the Ubuntu Gnome side. I have an OpenVPN SSL tunnel between the Mac and the Ubuntu desktop, however a SSH tunnel could also be used to protect the VNC session. In this post, I’ll just cover the VNC server setup assuming a secure connection between the Mac and the desktop.

Initially I followed the guidance at sanity, inc.”How to OS X Leopard Screen Sharing with Linux“, on Ubuntu I installed tightvnc:

apt-get install tightvncserver

Then tested it out by starting up the vnc server on the Ubuntu system as the user I want to run the remote session as:

tightvncserver -geometry 1024x700 -depth 24 :1

As tightvncserver starts up the VNC service, it will check for a .vncpasswd file in the user home directory. If it doesn’t exist, you will be prompted for a password to use to protect the remote session.  Note VNC is not designed to be used for multi-user remote access.
On the Mac, rather than use Bonjour to automatically discover the Ubuntu screen sharing service, I just referred to the VNC session directly within Finder which invokes the built in VNC viewer. Enter the VNC session password when prompted and the Ubuntu desktop is displayed. connect-to-server Within Finder, either use Go -> Connect to Server or Apple-K to bring up the Connect to Server window.  The server address is the URL that points to the Ubuntu VNC instance vnc:// where the port is 5900 + the display number specified when starting up the tightvncserver (5901).

This all worked fantastic, except for the keyboard mapping within Gnome – it was scrambled.  After googling several possible solutions, the only one that was successful for me was to disable the keyboard plugin in Gnome

Amit Gurdasani wrote on 2008-04-28: #51

I’ve also encountered this issue with TightVNC and the hardy release. My solution was to capture the xmodmap -pke output as ${HOME}/.Xmodmap at the login screen (DISPLAY=:0 XAUTHORITY=/var/lib/gdm/:0.Xauth sudo xmodmap -pke > ${HOME}/.Xmodmap). When gnome-settings-daemon starts up and finds an .Xmodmap, it asks if it should be loaded — I answer yes. As a side effect, if gnome-settings-daemon were to be restarted without the .Xmodmap, it’d scramble the keyboard layout again. With an .Xmodmap in place, it’ll load the .Xmodmap every time.

Due to another issue (#199245, gnome-settings-daemon crashing with BadWindow every time a window is mapped), I disabled the keyboard plugin using gconf-editor, at /apps/gnome_settings_daemon/plugins/keyboard. Since it’s not being loaded, I suspect it might not garble the layout even if I remove the .Xmodmap now.

So maybe disabling the keyboard plugin is a better fix.

On the Ubuntu system, invoke the Gnome configuration editor (gconf-editor on command line), then navigate to apps -> gnome_settings_daemon -> plugins -> keyboard uncheck the Active keyword.  Kill the VNC daemon and relaunch it – problem fixed.

pkill vnc
tightvncserver -geometry 1024x700 -depth 24 :1

Various methods exist to automatically start and kill the VNC server, but for now this will do it for me.

IMAP mailstore migration .. again

So last weekend, I discovered that Spamhaus decided it would be a good idea to place all of the public IP addresses for Slicehost (my Linux VPS hoster) into their Spamhaus block list (SBL). This covered both my slice in Dallas and the one in St. Louis – meaning an impressive chunk of inbound mail to my domains was being trashed by the sending MTA and an even bigger chunk of my outbound mail was being outright rejected since the sending IP’s were on the SBL.  Slicehost worked hard to convince Spamhaus to recind the blocklist, so the Slicehost IP’s got moved over to the less-nasty-but-you’re-still-probably-a-spamming-dirtbag Policy Block list (PBL) assuming affected IP owners would request to be removed from that list.

Sample query to see if you’re on any Spamhaus block list:  http://www.spamhaus.org/query/bl?ip=

It seems it’s time to relinquish the care and feeding of my own Postfix mail system and turn to a hosted solution.  This means I need to migrate about 5GB of IMAP store to another site (again).  Last time I did a wholesale migration, I used imapsync to make the transition painless. In the code example below, an SSL connection to the IMAPS server at imap-server.sourcedomain.com is made with username@sourcedomain.com and the password stored in the plaintext file secret1. An SSL connection is made to the target system (which happens to be the server on which the imapsync tool is running, but could just as easily be another IMAPS server somewhere on a network accessible to the host where imapsync is running). The –delete and –expunge1 arguments will clean the successfully moved messages from IMAP store #1 .. so be sure you have your messages on the target successfully! Imapsync can be run iteratively to ensure you have got all the messages from your source.

/usr/bin/imapsync \
--host1 imap-server.sourcedomain.com \
--ssl1 \
--authmech1 LOGIN \
--user1 username@sourcedomain.com --passfile1 secret1 \
--host2 --user2 username@targetdomain.com --passfile2 secret2 \
--ssl2 \
--delete --expunge1 \

And one can use the


option to just test the process but not actually move any of the messages.

So that’s it – I’m about half way though migrating my current IMAP stores over to a hosted mail solution, so that I don’t need to keep up with the increasing level of care and feeding that running your own mail service requires.  Before I get too many darts about that .. I first started running my own personal MTA in 1995, adding spam and av filtering over time, and adding substantial redundancy (servers, sites, storage) so I could rely on it and fix things that broke as I had time rather than right when they broke (which was always at a bad time).  My new hosted solution takes over from two VPS servers running Postfix, Spamassassin, ClamAV, Greylisting with the IMAP store replicated across data centers in different states (15 minute rsyncs).  So soon, the (hopefully) last Allen Pomeroy owned and operated MTA can be turned off, while I get to work on fun stuff, rather than figuring out why my email is bouncing.  :-)

Update 2012/12/17:

Sometimes manual manipulation of your mailstore via IMAP is needed, so here’s how I deleted a large number of folders I had trashed and were being synced to my new system from the old.  Kinda clunky, since I didn’t get the scripted version to work (just used a copy/paste in an interactive bash session), but got the job done for now.

Connect to the IMAP server using SSL:
openssl s_client -crlf -connect imap.emailsrvr.com:993

* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Server ready director6.mail.ord1a.rsapps.net

Log in with your email credentials:
0 login user@domain.com Password


List the folders you want to remove:
0 list "" "Trash.*"

That didn’t return the list I was expecting, so I listed all folders
0 list "" "*"

… and realized the source mail system adds “INBOX” on the front of the folder names, so then this command worked to list the folders to be deleted:
0 list "" "INBOX.Trash.*"

I copied the output and edited it to insert the folder name into a delete command:
0 delete "INBOX.Trash.Folder1"
0 delete "INBOX.Trash.Folder2"
0 delete "INBOX.Trash.Folder3"

0 OK Delete completed.
0 OK Delete completed.
0 OK Delete completed.

Finish off the session by logging out:
0 logout

* BYE Logging out
0 OK Logout completed.

FreeMind mind mapping tool

Have you ever had a daunting task that just seemed like a nightmare to get your head around how to organize it? If you’re like me, you try to find some patterns in all the individual elements that make up whatever the topic is you’re trying to get a handle on. The patterns may not come easily, and even if they do, it’s usually a pain to try and re-categorize an element as you see fit (ever tried to create lists and categorize things in Excel??).

I came across a tool that one of my clients uses called FreeMind – it’s a Java app that allows you to enter a number of text elements and reorganize them in a hierarchical fashion.

FreeMind example

FreeMind example

Ok, one can do that with an unstructured word processor document or a spreadsheet, but FreeMind allows you to dump all these random ideas onto the page then drag and drop into categories or tags that make sense as you’re rearranging the elements.

So after about an hour of dropping in ideas around areas of improvement for the IT security of one of my clients, I had over 250 elements organized into 8 high level categories and about 18 subcategories. It was grouped well enough to lead discussions on what the current priorities for their programmes should be. If I had attempted this in a spreadsheet (and I had) it would have taken hours and untold frustration – not to mention I probably would have missed relationships that I could see in FreeMind.

If I had attempted this in a spreadsheet (and I had) it would have taken hours and untold frustration

FreeMind icons

FreeMind icons

You can add icons to each element to make labeling and categorization easier. Best to check out the FreeMind home page as it is a feature rich tool. From the project Wiki, typical uses include:

  • Keeping track of projects, including subtasks, state of subtasks and time recording
  • Project workplace, including links to necessary files, executables, source of information and of course information
  • Workplace for internet research using Google and other sources
  • Keeping a collection of small or middle sized notes with links on some area which expands as needed. Such a collection of notes is sometimes called knowledge base.
  • Essay writing and brainstorming, using colors to show which essays are open, completed, not yet started etc, using size of nodes to indicate size of essays. I don’t have one map for one essay, I have one map for all essays. I move parts of some essays to other when it seems appropriate.
  • Keeping a small database of something with structure that is either very dynamic or not known in advance. The main disadvantage of such approach when compared to traditional database applications are poor query possibilities, but I use it that way anyway – contacts, recipes, medical records etc. You learn about the structure from the additional data items you enter. For example, different medical records use different structure and you do not have to analyze all the possible structures before you enter the first medical record.
  • Commented internet favorites or bookmarks, with colors and fonts having the meaning you want

What a great tool .. I’m sure I’ll find more uses for it!

Windows/AD Notes

Find all the AD groups a particular user belongs to:
dsquery user -samid username | dsget user -memberof

Find all members of an AD group:
dsquery group -samid groupname | dsget group -members

Find all inactive users:
dsquery  user -disabled -inactive 12