Hostnames for eximstats Rejections

I use eximstats to report my daily email traffic. I have a fairly high rate of rejections, and wanted hostnames listed in the rejection reports.    To resolve this I developed a patch to capture the hostname related to the IP address, and add this data to the rejection reports.

The enhanced list saves me the effort of looking up IP addresses that were repeatedly addressed.   Occasionally, these are from legitimate servers that have been misconfigured.  DNS problems are often the cause.

There are a some of caveats to this patch:

  • Only the last hostname provided for an IP address is reported.  Multiple names may apply to an IP address, but  this code only reports one name.
  • This patch may break the ability to merge reports. If not, summary reports may not summarize the rejects correctly. I don’t use this feature, and haven’t tested it.
  • Lookups might be done for other reports if  the hostname was an IP address in domain literal format. These really shouldn’t be required (or used) by Internet connected mail servers. By default Exim does not accept domain literals.

The following diff (patch) includes all the functionality for this feature. You may want to apply it to a copy of /usr/sbin/eximstats and test it before replacing the standard program. I keep the eximstats.orig version as well as the patched version.

--- eximstats.orig      2012-06-23 13:09:23.000000000 -0400
+++ eximstats   2012-07-05 21:44:55.000000000 -0400
@@ -615,6 +615,7 @@
 use vars qw(%ham_count_by_ip %spam_count_by_ip);
 use vars qw(%rejected_count_by_ip %rejected_count_by_reason);
 use vars qw(%temporarily_rejected_count_by_ip %temporarily_rejected_count_by_reason);
+use vars qw(%host_by_ip);

 #For use in Speadsheed::WriteExcel
 use vars qw($workbook $ws_global $ws_relayed $ws_errors);
@@ -679,6 +680,29 @@
 #                   Subroutines                  #
 ##################################################

+
+#######################################################################
+# key_with_host(key)
+# Return the league table key with host appended
+#
+# Uses host_by_ip as primary lookup source
+#  this is populated with the hostname used elsewhere
+# Secondary lookup is dns check for IP addresses.
+#  hostname is prefixed with > in this case.
+#
+# Returns other keys unmodified
+#######################################################################
+
+sub key_with_host($) {
+    my ($ltkey) = @_;
+    if (defined $host_by_ip{$ltkey}) {
+       return "$ltkey  $host_by_ip{$ltkey}";
+    } else {
+       return $ltkey;
+    }
+}
+
+
 #######################################################################
 # get_filehandle($file,\%output_files);
 # Return a filehandle writing to $file.
@@ -1425,15 +1449,15 @@
     }

     # write content
-    printf $txt_fh ($txt_format, @content, $key) if $txt_fh;
+    printf $txt_fh ($txt_format, @content, key_with_host($key)) if $txt_fh;

     if ($htm_fh) {
-      my $htmlkey = $key;
+      my $htmlkey = key_with_host($key);
       $htmlkey =~ s/>/\>\;/g;
       $htmlkey =~ s/write(${$row_sref}++, 0, [@content, $key], $f_default) if $xls_fh;
+    $spreadsheet->write(${$row_sref}++, 0, [@content, key_with_host($key)], $f_default) if $xls_fh;

     if (scalar @chartdatanames < $ntopchart) {
       push(@chartdatanames, $key);
@@ -1935,6 +1959,9 @@
      : (/SMTP call from .*?(\[\S+\])/) ? $1
      : "local";
     $host = (/\\bH=(\\S+)/) ? $1 : "local";
+    if ($host ne $ip) {
+        $host_by_ip{$ip} = "$host" ;
+    }

     $domain = "localdomain";  #Domain is localdomain unless otherwise specified.

Lookup code for IP addresses with no hostname

If you reject connections before an HELO or EHLO command is issued, the rejected IP address may not have a hostname included with it.  This replacement key_with_host routine replaces the function used in the patch above. This code was used when I was deferring connections for which rDNS validation failed. In other cases, this patch is likely not very useful.

The change supports both IPv4 and IPv6 addresses and requires the NetAddr::IP::Util package. This package adds support for IPv6 to the Perl network capabilities. To add the package to Ubuntu, use the command sudo apt-get install libnetadd-ip-perl. If you don’t want to add this package, you can replace the IP address look-up code with the traditional IPv4 only look-up code.

Name lookups are only done for IP addresses reported in the top list that didn’t have a hostname associated with them. If you have rejected IPs without names in the rejected IP address lists, the report will take longer to run. This is due to the time taken to do DNS lookups. These look-ups occur once for every IP address which is reported, and does not have a hostname in the log data. Successful lookups are stored back to the list, so that the lookup is only done once. Looked up names are prefixed “>” to indicate that only the PTR record was reported.

sub key_with_host($) {
    my ($ltkey) = @_;
    my ($host);
    if (defined $host_by_ip{$ltkey}) {
        return "$ltkey  $host_by_ip{$ltkey}";
    } else {
        if ( $ltkey =~ m/^\[([^\]]+)\]$/ ) {
            $host = gethostbyaddr(( $1 =~ ":" ) ?
                                  ipv6_aton($1) : inet_aton($1), AF_INET );
        }
        if ($host) {
            $host_by_ip{$ltkey} = ">$host";
            return "$ltkey  >$host";
        } else {
    return $ltkey;
        }
    }
}

You might want to modify this code to report the PTR records for IP addresses that don’t have a verifiable hostname.  These will be reported as hostnames in brackets like “(hostname)”.