How to Never Lose Your Email Again

Posted by Prolific Programmer Thu, 08 Jan 2009 22:51:00 GMT

A little while ago, I outlined how to achieve limitless gmail using Amazon's S3 service. Well, over a year later, I finally have code ready, prompted by Doc Searls' report of data corruption with his email. You'll need javamail, JetS3t, and their associated dependencies, in order to run the class below. Tomorrow, I'll probably add SSL support, as most email providers are moving in that direction today.

import java.io.ByteArrayOutputStream;
import java.io.ObjectOutputStream;
import java.util.ArrayList;
import javax.mail.Folder;
import javax.mail.Message;
import javax.mail.Session;
import org.jets3t.service.S3Bucket;
import org.jets3t.service.S3Service;
import org.jets3t.service.security.AWSCredentials;

public class EmailS3 {
    Folder folder;
    String user,hostname;
    String AWSACCESSKEYPROPERTY = "com.amazon.aws.ACCESS_KEY";
    String AWSSECRETKEYPROPERTY = "com.amazon.aws.SECRET_KEY";
    public EmailS3 (String username, String password, String hostname) {
	this.user = username;
	this.hostname = hostname;
	Session session = Session.getDefaultInstance(System.getProperties(),null);
	// assume pop3 as imap is probably already backed up
	store = session.getStore("pop3"); 
	folder = store.getDefaultFolder();
    }

    private ArrayList messages() {
	ArrayList ret  = new ArrayList();
	folder.getFolder("INBOX");
	folder.open(Folder.READ_ONLY);
	Message[] content = folder.getMessages();
	for (int i = 0; i != content.length;i++) {
	    ret.add(content[i]);
	}
	return ret;
    }

    public void toS3() throws Exception {
	ByteArrayOutputStream baos = new ByteArrayOutputStream();
	ObjectOutputStream oos = new ObjectOutputStream(baos);
	oos.writeObject(messages);
        String bucketContents = baos.toString();
	AWSCredentials creds = new AWSCredentials(System.getProperty(AWSACCESSKEYPROPERTY), System.getPropety(AWSSECRETKEYPROPERTY));
	S3Service service = new S3Service(creds);
	byte[] bucketBytes = bucketContents.getBytes();
	int i  = 0;
	while (bucketContents.length() > (5 * 1024 * 1024)) {
	    S3Bucket bucket = new S3Bucket(this.user+'@'+this.hostname+new Integer(i).toString());
	    bucketContents = bucketContents.substring(i*(5*1024*1024), (i+1)*(5*1024*1024));
	    i++;
	    bucket.putObject(bucketContents);
	}
	S3Bucket bucket = new S3Bucket(this.user+'@'+this.hostname);
	bucket.putObject(bucketContents);
    }
    public static void main (String[] args) {
       EmailS3 e = new EmailS3(args[0], args[1], args[2]);
       try {
           e.toS3(); 
       } catch (Exception e1) {
           System.err.println(e1.getMessage());
       }
    }
}

How to Use Akismet for Your Email

Posted by Prolific Programmer Mon, 29 Dec 2008 06:29:00 GMT

In my opinion, the ultimate anti-spam solution is Akismet, by the Wordpress author (Matt Mullenweg)'s company, Automatic. So, if your POP3/IMAP mail provider doesn't use it, download javamail and its associated dependencies, jython and paste the code below into its own file, set your classpath, and run said file, at which point it will tell you if the message is deemed spam or not:

from java.lang import *
class akismet (Object):
    def __init__(self, akismetKey, protocol, hostname):
        self.key = akismetKey
        self.hostname = hostname
        System.setProperty(u'mail.transport.protocol', protocol)
        System.setProperty(u'mail.%s.hostname' % protocol, hostname)
        url = (u'http://%s.rest.akismet.com/1.1/verify-key' % akismetKey)
        data = {}
        data['key']=akismetKey
        data['blog']='blog.prolificprogrammer.com'
        response=self.post(data,url)
        
        
    def checkMessage(self, login, password, message_number):
        from javax.mail import Session
        store = Session.getDefaultInstance(System.getProperties(), None).getStore()
        folder = store.getFolder('INBOX')
        from javax.mail import Folder
        folder.open(Folder.READ_ONLY)
        msg = folder.getMessage(message_number)
        request_data = {}
        request_data['blog']=System.getProperty('mail.%s.hostname' % self.hostname)
        from java.net import InetAddress
        addr = InetAddress.getLocalHost()
        request_data['ip']=String(addr.getAddress(), 'utf-8')
        request_data['comment_content']=msg.getContent().toString()
        authorInfo = msg.getFrom()[0]
        request_data['comment_author']=authorInfo.getPersonal()
        request_data['comment_author_email']=authorInfo.getAddress()
        request_data['comment_author_url']='mailto:%s'%request_data['comment_author_email']
        request_data['permalink']=''
        ret = self.post(request_data, u'http://api-key.rest.akismet.com/1.1/comment-check')
        return ret.lower() == 'false'
    
    def post(self, data, url=self.url):
        import os,stat
        modificationDate = os.stat('akismet.py')[stat.ST_MTIME]
        header = {'User-Agent':'SpamDetect/%s' % str(modificationDate)}
        import urllib2
        req = urllib2.Request(url, data, header)
        response = urllib2.urlopen(req)
        return response.read()

if __name__=='__main__':
    import sys
    try:
        a=akismet (sys.argv[1], sys.argv[2], sys.argv[3])
        resp = a.checkMessage(sys.argv[4], sys.argv[5], sys.argv[6])
        if resp == True:
            print 'not spam'
        else: #resp == False
            print 'spam'
    except IndexError,e:
        print 'Usage: %s akismet-key protocol mailhost login password messagenumber'
        import os
        os.exit()

How to Reduce your Inbox to Zero

Posted by Prolific Programmer Tue, 02 Sep 2008 22:28:00 GMT

Unclutterer has a guest writer muse on reducing your inbox. I've given up on my inbox (34923 unread), but through careful use of labels, expert systems, and tagging, I am able to get all important email. The expert system is very highly customised, using growl to display popups regarding things like eBay and Craigslist notifications, airline travel deals, local stories, evite notices., et al.

How to Encourage OpenID Adoption

Posted by Prolific Programmer Tue, 26 Aug 2008 03:00:00 GMT

Chris Saad says that 'The OpenID User Experience sucks - how do we make it more user friendly?'. His solution is to map them to email addresses. The way Mr Saad proposes is performed using the following python script:

import sys
split_parts = sys.argv[0].split('@')
print (u'http://%s/%s') % (split_parts[1]. split_parts[0])

... And redirecting the browser to that URL.

How to Hear your Email

Posted by Prolific Programmer Mon, 09 Jun 2008 17:14:00 GMT

Lifehacker tells us how to get our email read to us, as a bedtime story, no doubt.

How to Solve the Email Debocle Step-by-Step

Posted by Prolific Programmer Mon, 09 Jun 2008 15:29:00 GMT

Below is a start at instructing users how to email. You'll need javamail and a supported JRE. Remember to change the username and set the password. I'll be refining this further in the coming days.

import java.util.*;
import javax.mail.internet.*;
import javax.mail.*;

public class MailTest {
    Message first;
    public MailTest() {
        Session session; Store store = null; Folder folder;
        try {
        Properties props = new Properties();
        props.put("mail.pop3.host", "pop.gmail.com");
        props.put("mail.pop3.user", "hasan.diwan@gmail.com");
        props.put("mail.pop3.port", "995");
        props.put("mail.pop3.starttls.enable", "true");
        props.put("mail.pop3.auth", "true");
        props.put("mail.pop3.socketFactory.port", "995");
        props.put("mail.pop3.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
        session = Session.getDefaultInstance(props);
        System.err.println("session initialized!");
        store = session.getStore("pop3s");
        System.err.println("store initialized!");
        store.connect("pop.gmail.com", "hasan.diwan@gmail.com", System.getProperty("password")); //
        System.err.println("Connected!");
        folder = store.getFolder("INBOX");
        folder.open(Folder.READ_ONLY);
        System.err.println("inbox opened!");
        first = folder.getMessage(1);
        System.err.println("Message retrieved!");
        if (first.getRecipients(Message.RecipientType.TO).length != 1) {
            System.err.println("God damn! More than one recipient on the to line!"); 
        }
        } catch (Exception e) { System.err.println("EXCEPTION: "+e.getMessage()); }
        finally { 
        try { store.close(); } catch (Exception e) { }
        }
    }

    public static void main (String[] args) throws Exception {
        MailTest m = new MailTest();
    }
}

              

How to Solve the Email Debacle

Posted by Prolific Programmer Thu, 05 Jun 2008 21:24:00 GMT

Seth presents things to consider when sending mass emails, which I wish everyone who wants to send me email would read before they do. Indeed, there's an algorithm I just devised to do this, probably can make a procmail rule, if I were so arsed, but I'll leave that to the readers. Without further ado, then:

  1. If the email is from someone in the approved set (more on this later), then abort this script and process normally.
  2. If the "To:" line of the email contains more than one address and it doesn't end in Mailinator or a similar service, send Seth's email checklist to them, saying that they need to resend their email.
  3. Add the email in "From" to the approved set.
  4. repeat for the next message that comes in.

How to Filter Spam

Posted by Prolific Programmer Fri, 02 Nov 2007 11:32:00 GMT

Inspired by Robert's post on the Akismet spam filtering system and how effective it is. I wonder if it would be possible to have an email reader use the same technology. It would appear that there exists an Akismet API, whereby one can use HTTP to verify whether a piece of text is spam or not. We need the comment-check call with the following parameters:

  • blog: http://blog.prolificprogrammer.com/
  • user_ip: the originating email host's ip address, eg lookup(address.split('@')[1])
  • user-agent: Email Akismet Spam filter
  • comment_author_email the originating email address e.g. message.recipients[0].address
  • comment_author_url: mailto://comment_author_email (from above)
  • comment_content: the actual body of the email, eg CWInternetMessage.content()
  • comment_type: reply
  • comment_author: the natural language part of the email address, e.g. EmailAddress.personal()
The site says to err on the side of too much information, so I've covered every parameter I can conceivably figure out. The calls are borrowed from the open-source Pantomime library, used by GNUMail, among others and the syntax is vaguely pythonic. If this returns true, the message is probably spam. If it returns false, the message needs to be looked at further. Finally, the false negatives and false positives need to be submitted, respectively, using the submit-spam and submit-ham with the same parameters.

How to have a Sense of Humour with Family

Posted by Prolific Programmer Tue, 16 Oct 2007 03:32:00 GMT

I just sent this email to my father, mother, brother, and little sister. See, I'd like to meet up with a friend in Sicily this Christmas, and my little sister wants to go to some posh hotel in Oman or Tunisia. I have zero interest in either. So I had to give her a piece of my mind:

Probably more for that most bossiest of little sisters, but when are you guys thinking of doing your fancy hotel in the rubbish city?

Hope the family gets a laugh out of it. I know me mates did!