Restoring SQL Express Databases

When I set up my latest computer, I paid special attention to making sure everything was backed up and I am really glad I did.  I had several issues with bad hard drives fairly early on and recently had another issue with some older drives that I have on the computer dying.  The only purpose these other drives had was to store the log files of my SQL Express Databases.   When they died, I actually didn't notice for several days until I tried to access one of these databases for some work on a development project.  All of my databases were offline.

Luckily I had set up an automated backup off all of the DBs.  Getting this setup using the Free SQL Express isn't obvious and requires (or at least helps to have) a great app that is out there called ExpressMaint.  This lets me run a Scheduled task that automatically backs up all my databases.   Those backups are then backed up by windows backup to a USB hard drive and by Carbonite online.

These databases were easy to restore after I changed the settings on my server to have the logs be on a different drive and made sure to restore to the good drives.   

Gmail's Priority Inbox (Categorized Important Messages)

I like Gmail's new "important" message feature that they are testing in beta currently.  To use it effectively with my setup that already automatically sorts my Gmail into categories, I had to do a few alterations.  I am using the "Multiple Inboxes" lab feature and I wanted to get the new division of "Unread and Important" and other "Unread".  The priority inbox only shows those emails that are in my inbox in those categories but because I have popfile moving my email to other tags, I needed to change the filters in my multiple inboxes.

I set the my filters to

• is:unread is:important
  (show me unread items with the important flag) 
• is:unread -is:important  
  (show me unread items that DON'T have the important flag)

So now my inbox has all my unread important messages at the top (sorted by category) and all my unread, probably not as important categorized emails in the next section.

Google Apps Mail with DotNetNuke

I was recently having a problem with the mail on the Atenveldt.org site.  I had switched from the host provided email server to Google Apps email, but in some cases email was being delivered to the wrong place.   It turned out that this was because I had the host's email server still set up in the SMTP settings in DotNetNuke.  The aliases on this domain change quite a bit as different officers take over and they keep the same alias but point to a new destination address.  For instance, the King and Queen of Atenveldt rotate twice a year, the crown (at) atenveldt.org address will always be available but point to the new King and Queen as they take over from the old set. 

The issues presented themselves when the DotNetNuke (or a particular module) was trying to send to someone at an atenveldt.org address. The host's email server thought it was still the email service for that domain and didn't bother to check DNS to see if that was still true, so the people who were set up in the old alias were getting that email rather than the people we had changed over to get email from the Google Apps group that we set up.

I couldn't just have the host take off the domain from the server because then the site wouldn't be able to email because it won't relay.

The solution to this was to use Google's SMTP server.  Once I set up a google apps account that was specific to the website, I had DotNetNuke use those credentials to log in to the SMTP server and send email. The email coming from the site would appear to be coming from website (at) atenveldt.org (even if a module tried to send out from a different address.  This is an effect of using the google smtp.)  A potential security hole exists in that all mail sent is now being saved in the website's google apps email box.  This means that when the DNN site sends out passwords and such, a record of it will be in that account's sent items. 

The benifits of using Google Apps for this setup is pretty nice.  Our mail is no longer tied to the host, so if we decide to leave this host and go to a new one, email will be one less thing we have to worry about.  There are some other benefits to the Google Apps for our organization in terms of shared calendar and docs.

Sort your Gmail with Popfile

Background

I get a lot of email.  I am on many mailing lists and I do a lot of volunteer work for the SCA which means a lot of email.   I also get a lot of bacn, catalogs and newsletters that I signed up for and don't mind getting.  I get notifications from FriendFeed, Facebook, Flickr, Twitter, and other online applications, which is also bacn I suppose.  Of course, in addition to all that, I get spam. If you want an indication of how much spam I get, check this out.  Even though GMail claims to clear my spam folder for things older than thirty days, I have spam since Feb 25th, 2009 in there right now a total of 19,060 items.  Since today is 4/25/2009 this will give me a good estimate of my spam per month of 9,500 (across two months).  All in all its fairly easy to estimate that 10,000 items come to me every month, yet I haven't gone insane.  Why?   Well Google does a decent job of catching much of the spam but I have something else in my pocket called Popfile. 

POPFile has been around for a while (their download archive page puts the first release in October of 2002)  and I first heard about it on the ScreenSavers, about 6 years ago.  So you might understand why I was yelling at Leo Laporte in my car as I was listening to a recent podcast when he said that he had too many email's in his inbox.  He was the one who introduced me to POPFile! POPFile is not just a spam killer, but it classifies your email into any number of buckets that you want.  It took care of my bacn, before there was a term for bacn.  It was able to distinguish my SCA email from my personal email, I could categorize responses from forums I participated in as different from newsletters and other notifications.

I had changed my email a couple of times because it was just awash in spam.  Now I had Popfile and was able to use my jeffmartin.com as a mail address without worrying about getting overloaded by the 95% noise I would get over my 5% signal.  This worked great for about 6 years because I was using a POP email account and outlook email client.  But I had just got my G1 Android Phone and it had Gmail built in.  I wanted to join the crowd and get my email on my phone, but I was stuck on the pop account and had used Popfile for so long, I didn't want to give it up.  Luckily, Google and Popfile conspired to help me out.  GMail recently started supporting IMAP and Popfile added an IMAP module to its core installation.  Now I have the power of email anywhere I can log into the Internet and its all sorted for me when I get there so I can focus on what I want to work on.

Why is Popfile so Awesome?

You may be saying, "But Jeff,  my email is already pretty well sorted. I use GMail filters and stuff goes where it needs to."  You might be right, and filters may be enough for the light email user, but you have to spend time setting up each filter and its possible that stuff may slip through that you don't expect.  For instance, if you support a product or service, or have a website that you get email from, you might want all that email sorted into different categories.  You don't know the email address of the people who will be sending to and you can't guarantee the subject will have particular words in it even if you ask people to do it that way.    Maybe you participate in a lot of forums and you have email notifications turned on so that you can see when someone responded to your post.  With Popfile, you can start posting at a new forum and have pretty high confidence that response notices will be correctly sorted without having to set up a new filter for that particular forum.   Here is the really killer feature: Popfile learns from its mistakes (and you can change your mind and reteach it new tricks).  When a piece of email goes to the wrong place (or if Popfile is unsure, it will put it in an Unclassified bucket), you can direct the email to the right place and Popfile will learn from that. (This is a feature of the IMAP module, the normal method of training Popfile is to use the interface to "teach it").   It learns through an algorithm developed by Thomas Bayes around 1720, which appeals to the bit of history buff I have.  This Bayesian algorithm is now the basis for many "learning" computer software programs.

Setup

So here is how you set it up.

What you need to start

1. First you obviously need a GMail account. I use a Google Apps account so I can have GMail delivered to my own domain. 
   (As an aside, I also have a GMail email that I use for Google services that don't support a google apps login, this GMail just forwards to my main account).
2. A computer that can run Popfile. Popfile runs as a service in the background and has a web interface. Popfile runs on windows but has cross platform versions.  I am not a Mac guy or a Linux guy but I don't see any reason why it wouldn't run on those OS's.
   I use my desktop for this task.  Its pretty much always on and connected to the Internet, so it can have my email sorted and ready to go when I want to access GMail.  I don't see why this wouldn't work with a laptop that ran Popfile too but it may take a little time to sort your email if you hadn't been connected to the Internet for a while, so you could potentially log in to GMail and still be waiting for POPFile to sort your inbox.  Also, if you hit the Internet from another computer and your laptop isn't connected from somewhere else, your email won't be sorted.

Directions

1. Set up IMAP in your Gmail account using these instructions.
2. Download and Install Popfile.
3. Setting up Popfile:
   1. Enable the IMAP module by following the directions here.
   2. Once the IMAP module is turned on, you need to configure it with your gmail information.
   3. You also need to have labels already created in Gmail, you can then match buckets in Popfile to Labels in Gmail (you need to create buckets).  Make sure you match your spam bucket to the standard spam label in GMail.
4. You will need to now train POPFile to know what types of email you want in your labels.  Initially all of your incoming email will be delivered to the Unclassified bucket.  After you start training POPFile, you should be pretty happy with the results. According to POPFile's stats, the average accuracy of the sorting is over 98% after 500 emails.

Conclusion

Hopefully you can stop being afraid to leave your email address out in the open now.  Hopefully people can stop going from one free email to another when the spam gets too bad.  Hopefully you found this article helpful.   POPFile is a great way to not only filter out your spam but help you get things done by pre-sorting your mail into the buckets you want.