Over a year ago I saw mod_mbox. I love they way it provided nice looking archives and an Atom1 feed for a mailbox archived in the mbox2 format. I knew immediately that I wanted this for some Mailman lists which I provide.
Mailman does store its archives in mbox format. Unfortunately mod_mbox wants one mbox file per month, named a very specific way.
Here is how I got there. First, the tools:
- Mhonarc is a mail to html converter used by popular sites such as mail-archive.com and lists.debian.org.
- Mharc is a web-based mail archiving system for multiple mailing lists.
- Mharc has a tool called mbox-month-pack which does (almost)exactly what I want.
- Mailman is already configured.
- mod_mbox just needs to be plugged into a pre-configured apache.
The Procedure:
Mailman stores its archives in /var/lib/mailman/archives/private/ with complete mbox archives in one file in a file named ${list}.mbox/${list}.mbox. I just need to feed that to Mharc’s mbox-month-pack.
Massage the filenames a bit:
for i in ????-?? ; do mv $i ${i/-/}.mbox ; done
Setup Apache as the mod_mbox docs say.
Modify mailman’s list_members program to extract a encrypted password for each member, with output suitable for basic http authentication.
— /var/lib/mailman/bin/list_members 2007-06-12 08:49:33.000000000 -0400
+++ /var/lib/mailman/bin/list_members_pass 2007-11-08 23:04:20.000000000 -0500
@@ -80,6 +80,10 @@
from email.Utils import formataddr
+import crypt
+from random import randint
+import string
+
PROGRAM = sys.argv[0]
ENC = sys.getdefaultencoding()
COMMASPACE = ‘, ‘
@@ -140,6 +144,11 @@
return status <> MemberAdaptor.ENABLED
return status == WHYCHOICES[why]
+salt_chars = ‘./’ + string.ascii_letters + string.digits
+
+def crypt_password(password):
+ salt = salt_chars[randint(0, 63)] + salt_chars[randint(0, 63)]
+ return crypt.crypt(password, salt)
def main():
@@ -258,14 +267,16 @@
rmembers.sort()
for addr in rmembers:
name = fullnames and mlist.getMemberName(addr) or ”
+ password = crypt_password(mlist.getMemberPassword(addr) or ”)
# Filter out nomails
if nomail and not whymatches(mlist, addr, why):
continue
– print >> fp, formataddr((safe(name), addr))
+ print >> fp, formataddr((safe(name), addr)) +”:”+password
if digest:
dmembers.sort()
for addr in dmembers:
name = fullnames and mlist.getMemberName(addr) or ”
+ password = crypt_password(mlist.getMemberPassword(addr) or ”)
# Filter out nomails
if nomail and not whymatches(mlist, addr, why):
continue
@@ -278,7 +289,7 @@
# They’re getting MIME digests
if kind == ‘plain’:
continue
– print >> fp, formataddr((safe(name), addr))
+ print >> fp, formataddr((safe(name), addr)) +”:”+password
Wrap all of this up in a nice little script and insert into cron so that messages, usernames and passwords are refreshed hourly or daily.
#!/bin/bash
MAILMAN=/var/lib/mailman
MBOXMONTHPACK=/usr/local/mharc/bin/mbox-month-pack
LISTLISTS=$MAILMAN/bin/list_lists
LISTMEMBERSPASS=$MAILMAN/bin/list_members_pass
for list in `$LISTLISTS -b` ; do
INPUTFILE=/var/lib/mailman/archives/private/$list.mbox/$list.mbox
OUTDIR=/var/mbox/$list
if [[ ! -d $OUTDIR ]]; then mkdir $OUTDIR; fi
if [[ ! -f $OUTDIR/.htaccess ]]; then
cat >$OUTDIR/.htaccess <<EOD
AuthType Basic
AuthName “mailman Password Required”
AuthUserFile /var/passwords/$list.pwfile
Require valid-user
EOD
fi
$LISTMEMBERSPASS $list > /var/passwords/$list.pwfile
$MBOXMONTHPACK $INPUTFILE -outdir $OUTDIR
pushd $OUTDIR
for i in ????-?? ; do mv $i ${i/-/}.mbox ; done
mod-mbox-util -v -c .
popd
done
libapache2-mod-mbox is available on my Launchpad PPA.
mhonarc is available by default in debian.
mharc you will have to get the source and build.
- Say what you want about RSS vs Atom. I don’t care. Any tool that matters supports both… except mod_mbox.
- Yes, mbox isn’t the ideal mailbox format, but in this case it works
