dennis.ca

RSS Feed

Weblog from Ottawa, Ontario, Canada

Howto: Make Spotlight and Google Desktop Index Thunderbird Messages

April 17th, 2007 · 48 Comments

WARNING: This tutorial has not been tested on a fresh Leopard install. It may or may not work; use at your own risk!

SpotlightOne of the most frustrating things about Spotlight is that it doesn't index messages in Thunderbird. The reason is because Spotlight requires a single search result to be able to point to a single source file.

Since Thunderbird (and other email clients like Eudora) stores message folders within an mbox format file, messages can't be referred to individually.

Until now... Thunderbird 2.0 RC1 introduces a kludgy (but nifty) configuration option named mail.spotlight.enable. Enabling this option causes Thunderbird to create individual text files for all messages. These files can then be indexed by Spotlight, and in turn by Google Desktop.

Google DesktopThe following instructions require that you upgrade your version of Thunderbird to a beta version -- wait for the final version if you're uncomfortable running prerelease software.

I've been running this setup for about a week with no issues whatsoever. As a disclaimer, you're on your own if you somehow manage to mangle your system with these instructions. There's really nothing here that should cause any problems, but if you don't know what you're doing, it'd be best to stop now.

The following instructions should help you on your way to having Spotlight index Thunderbird messages --

  1. Backup your existing Thunderbird profile (usually located in [user directory]/Library/Thunderbird). Do not skip this step!
  2. Install Thunderbird 2.0 RC1 directly over your current version. Test the installation to ensure your messages are properly migrated. If something is botched at this stage, do not proceed.
  3. Download and install mozilla mdimporter into /Library/Spotlight. This file is a plugin that teaches Spotlight how to index Thunderbird mail.
  4. Within Thunderbird, select Preferences | Advanced | Config Editor. Search for the entry mail.spotlight.enable and set the value to true.
  5. Verify that the importer has been registered by entering the following in a terminal: /usr/bin/mdimport -L. About 2/3 of the way through the output of this command you should see the line "/Library/Spotlight/Thunderbird.mdimporter". If not, something is wrong -- do not proceed.
  6. Reboot, start Thunderbird. At this point, the 'text' version of your messages will be created. Allow ample time for this step to complete.
  7. Use Spotlight to search for text within an email message. For some reason (on my system, at least), Spotlight didn't want to index emails until I tried the first search. The indexing processing took quite a while, so be sure to let it churn before using it too heavily. You can use Activity Monitor to check to see if the mdimporter process is still running.

Depending on the size of your mail folders, creating the text files and indexing your messages will take some time. You'll notice lots of disk activity while Thunderbird creates text versions of its messages and Spotlight begins to index them. Be patient!

The side benefit of all this is the following -- Google Desktop uses Spotlight plugins to index the file system. If Spotlight is properly indexing message, Google Desktop will also do so without any added effort on your part.

Have fun!

Tags: · , , , , , ,

48 comments for this entry ↓

  • 1 Andy // Apr 20, 2007 at 9:12 am

    Hey Dennis,

    Great guide and thanks for sharing it with us. Just a quick question. I've followed your steps and managed to get Spotlight to search my local Thunderbird mailboxes. However, when I double-click on the entry inside the Spotlight's result window, trying to open the associated email message, my Thunderbird just closes(crashes) application window itself. Are you having the same problem..? or any idea why this is happening...?

    By the way, I'm using the followings:
    OS X: 10.4.9
    Thunderbird 2.0.0.0 (20070326)

    Cheers,

    Andy

  • 2 dennis // Apr 20, 2007 at 6:56 pm

    Andy: A few things to check --

    1) Make sure that .eml files are set to open by default with Thunderbird. Test this by saving a message from Thunderbird and double-clicking to open.

    2) Make sure .mozeml files are set to open by default with Thunderbird. Test this by opening one of the files within the "mozmsgs" folders that get created in your Thunderbird profile directory during mail indexing.

    3) In the preferences for OSX Mail (and I know this seems counterintuitive), make sure you set Thunderbird as the 'Default Email Reader'.

  • 3 Andy // Apr 23, 2007 at 11:36 am

    Hey Dennis,

    Cheers for the head-up. I didn't think to check whether the .eml is set to be opened by Thunderbird by default or not. It turns out that .eml was set to be opened by MS Entourage. Once I've changed that back to thunderbird, the Spotlight and Thunderbird works perfectly now!!

    Many thanks again and cheers for pointing me to the right direction... :)

  • 4 dennis // Apr 23, 2007 at 4:33 pm

    Andy: Great news, glad to hear it works for you! I had the same issue with Entourage, so I figured it might have something to do with the default handler.

  • 5 Rolando // May 3, 2007 at 8:03 pm

    Dennis-

    Thanks so much for this. It seems to be working well to a point, but I am having the same problem you and Andy had. However, my .eml and .mozeml files are associated with Thunderbird. When I doubleclick on an .eml file, things seem to work well. But when I do so on one of the .mozeml files, Thunderbird unceremoniously crashes. Any idea what else might cause this?

    0 org.mozilla.thunderbird 0x001e5ff8 nsMessengerBootstrap::OpenMessengerWindowWithUri(char const*, char const*, unsigned) + 356

    Thanks,
    Rolando.

  • 6 dennis // May 5, 2007 at 5:02 pm

    Rolando: It almost looks like an overflow error. How long is the filename of the email you're trying to open? I'd suggest submitting this bug to Mozilla.

    If you're not already using the final release version of Thunderbird 2.0, that's definitely worth a shot.

    Sorry I can't be any more help!

  • 7 Will // May 7, 2007 at 3:41 pm

    Is this reversible should I re-set mail.spotlight.enable to the default?

  • 8 dennis // May 7, 2007 at 6:54 pm

    Will: This is completely reversible.

    1) Set "mail.spotlight.enable" to "false".

    2) Delete any "(foldername).mozmsgs" folders from your Thunderbird profile directory located at --

    (your user directory)/Library/Thunderbird/profiles/(your profile name)/mail/(your mail account name)/(mail folder names).mozmsgs

  • 9 Will // May 11, 2007 at 6:15 pm

    cool, so they keep a copy of the emails (new one too) in both format?

    It's nice to have spotlight enabled... Hopefully they will fix the "opening email doesn't work" bug soon.... :D

  • 10 Kjell H A // Jul 9, 2007 at 3:23 pm

    Thanks for a great guide. It worked like a charm. I've been missing Thunderbird since I made the switch to Mac as I wanted Spotlight even more. Now I can have both.

  • 11 Kjell H A // Jul 9, 2007 at 3:44 pm

    Hey, I didn't realize you were into photography until left that comment about my Cat image. I just found your blog while searching for Thunderbird-Spotlight solutions.

    I'll save a bookmark for your galleries, I'm a bit too tired now. And about Akvis Enhancer, you should really give it a try, just be careful not overdoing it. It's easy to get carried away.

    Cheers from Oslo
    Kjell

  • 12 Olivier Gourment // Jul 11, 2007 at 9:29 pm

    It hasn't worked for me. I have Thunderbird 2.0.0.4 and at step 6, after rebooting and starting Thunderbird, nothing happened. I mean that I waited a little, went to check if the text files were created, and the .mozmsgs directory was not created under (my user directory)/Library/Thunderbird/profiles/(your profile name)/mail/(your mail account name)

    Cheers

  • 13 Olivier Gourment // Jul 11, 2007 at 9:34 pm

    OMG! I was such an impatient commenter! After a few minutes, I could see the directories and the spotlight search worked exactly as expected. THANKS!

  • 14 dennis // Jul 11, 2007 at 9:36 pm

    Olivier: Depending on the number of folders and messages you have, the generation of the mozmsgs folders/files could take quite a bit of time. Do you see the mdimporter process running? (make sure Thunderbird is running when you check). If so, give it some time -- it should eventually do its work.

    If not, feel free to report back and perhaps I can help you sort this out via email or IM.

  • 15 CB // Jul 18, 2007 at 6:18 am

    Thanks mucho to dennis for this excellent info.

    Unfortunately, there's a problem: When you use spotlite or GDS to search for a message, then click on the search result; the message appears --- but it's entirely out of context --- there's no way to tell what subfolder the message was filed in. Apple Mail handles this with a "view in message viewer" link.

    The error console shows this line every time I click on a search result:

    Error: [Exception Failure when calling method: [nsICommandLineHandler::handle nsresult: 0x80004005 (NS_ERROR_FAILURE) location: data:no]

    Am I the only one who's having this problem?

    Intel Mac Dual Core
    MacOSX 10.4.10
    Thunderbird 2.0.0.4
    Followed Dennis' instructions pretty carefully, even did a TBird reinstall from scratch.

  • 16 dennis // Jul 18, 2007 at 9:24 pm

    CB: The only thing I can think of is that Thunderbird isn't your default mail handler. Try this --

    1) Make sure that .eml files are set to open by default with Thunderbird. Test this by saving a message from Thunderbird and double-clicking to open.

    2) Further test this by opening one of the files within the "mozmsgs" folders that get created in your Thunderbird profile directory during mail indexing.

    3) In the preferences for OSX Mail (and I know this seems counterintuitive), make sure you set Thunderbird as the Default Email Reader.

  • 17 Spotlight search for Thunderbird emails & OpenOffice.org documents « Entrepreneur Geek // Jul 20, 2007 at 10:14 am

    [...] around some blogs, I found Dennis’s blog. The blog post describes in detail how we can enable Spotlight search for Thunderbird. You can read the post for full details, but the following are the [...]

  • 18 Wie mailt man am Besten? Mit Thunderbird! « Mein Neuer Mac // Aug 3, 2007 at 3:22 am

    [...] Originalbeitrag von Dennis (eng) http://www.dennis.ca/weblog/2007/04/17/howto-make-spotlight-and-google-desktop-index-thunderbird-mes... [...]

  • 19 Marc // Aug 4, 2007 at 8:10 pm

    I hat to quit Thunderbird during initial creation of mozmsgs folders. Now only manually viewed (new) messages will get an correspondent mozmsg file. How can I restart the whole process?

  • 20 James Marshall // Aug 5, 2007 at 5:28 am

    Thanks for a very useful howto page. There's one refinement I'd like to find a way to do... Spotlight display Mail app messages under the 'Mail' list, but for Thunderbird messages these are lumped in with the 'Documents' list. There's clearly some information about which application the documents belong to that is made available to Spotlight, so I wonder if there's a way to tell Spotlight that Thunderbird is the default mail client. I will hunt around more for docs on configuring Spotlight, but if anyone else has any ideas...

  • 21 dennis // Aug 10, 2007 at 4:18 pm

    James: That's a good point; I didn't notice it until you brought it up. Let me know if you find anything, and I'll also search around to see if I can figure anything out.

  • 22 Chris // Aug 11, 2007 at 10:58 pm

    Yeah...it's really working - I thought, but then TH 2.0.0.6 (German version) crashed while trying to open the .eml or .mozeml files :( I checked the file assosiation, if TH is my standard email client and rebooted the system - nothing worked for me. Even the file name doesn't seem to be very long "960215.50935.qm%40web15705.mail.cnb.yahoo.com.mozeml", or is it?

    To find that out I simply renamed this file into "sinal". But nothing happend! TH didn't crash but also didn't open the file. So I tried to open a Mail.app Email and it workd perfectly!
    Seems to as if the exported mails from TH are not 100% compatible with TH.

    Hope anybody can give me a hint on this one.
    Thanks. And special thanks to Dennis for this guide ;)

    Btw. I recongized one strange thing. I actually don't use yahoo.com, but in the file names appear yahoo.com quite ofte, even if I'm with specific mail 100% sure what account it is - not Yahoo.

  • 23 Chris // Aug 11, 2007 at 11:07 pm

    A little mistake in the text above!

    The email which is opened by TH perfectly is not from Mail.app. I must have exported this mail once with TB 1.5.4. But as far as I know there's no possibility to export all mails at once like this.

  • 24 dennis // Aug 12, 2007 at 12:27 am

    Chris: That's odd, I've never run into it before. Do you or anyone else who access your computer use Yahoo mail at all (within the browser)?

    I wonder if it is somehow indexing cached files from your web browser. If you open up one of those files with a text editor, what's in there?

  • 25 Chris // Aug 12, 2007 at 12:43 am

    Hi Dennis
    Thanks for the fast feedback! After checking the header of the mail I realized that the sender used yahoo.com ;) But still the other problems remain. Below you will find a header of the .mozeml mail which I can't open. The with XXX marked parts are just names which aren't important to figure this problem.

    kMDItemLastUsedDateSat Aug 11 22:15:45 2007
    kMDItemTitle
    (non-ASCII removed): Re: XXX
    kMDItemDisplayName
    (non-ASCII removed): Re: XXX
    kMDItemTextContent
    sinal jian <[email protected]> XXX XXX <[email protected]> (non-ASCII removed): Re: XXX

  • 26 dennis // Aug 12, 2007 at 1:03 am

    Chris: That header definitely doesn't look right. It should be an XML file consisting completely of plain text. I would expect to see things like subject encased in 'key' tags.

    Do you have any 'working' .mozeml files to compare to? I'm starting to wonder if this might be a problem specific to the German localized version of Thundebird...

    The only thing I can suggest is to send yourself a test email, wait for it to be indexed, then take a close look at what ends up in the file. I can send you a "known-working" .mozeml file if you want to try it from your system to see what happens.

  • 27 Chris // Aug 12, 2007 at 1:12 am

    Dennis: Here is my email address riss.christian(-at-)gmail.com. I just see that the header I sent to you was changed by this web page. Hope you can write me a mail so I can send you the complete header. Thanks for your help.

  • 28 Roger // Aug 23, 2007 at 10:20 am

    Marc,

    Did you ever get your problem sorted out (after quitting during intial indexing, no extra .mozeml files get auto-generated)? I have the same problem.

    Roger

  • 29 Roger // Aug 23, 2007 at 10:40 am

    Nevermind -- I was able to get it sorted out by reversing the whole process as recommended by Dennis, then starting over.

  • 30 Ajay // Aug 31, 2007 at 10:30 pm

    Worked like a charm! Thanks Dennis.

  • 31 jmr // Sep 8, 2007 at 3:34 am

    Many thanks Denis, it's working perfectly.
    Do you have some solution for "James Marshall said on August 5, 2007 at 5:28 am"
    jmr

  • 32 dennis // Sep 8, 2007 at 3:42 am

    jmr: Haven't figured out a solution for that particular problem yet, but please check back after the weekend; I'll see what I can come up with tomorrow.

  • 33 Joel // Sep 11, 2007 at 6:13 pm

    Dennis, I appreciate this tutorial, but after I did it, I have lots of ".mozeml" files but no ".eml" files. And Spotlight doesn't seem to have indexed any mail. Any suggestions?

  • 34 Cristian // Sep 12, 2007 at 4:00 am

    Thanks for a great Howto!

    CB: One way to get context (folder) to a message found by spotlight is to chose "Show all" in Spotlight, then chose the file and click on the information icon at the right of the filename. That will display the file folder, from which you can find out the mail folder.

    Maybe I found a problem: I had a folder that contained no message, just other folders. That seems to have broken the export process, which continued once I copied a message in there.

    For the impatient, this is how you can monitor the progress of the mail export to mozeml in the Terminal:

    cd yourThunderbirdProfile
    find . -name "*mozeml" | wc

    Repeat the last command, you should see the number of files generated increase.

    Also, in Thunderbird Configuration editor, Thudnerbird displays mail.spotlight.lastFolderIndexedUri, that's another way to monitor, though progress there is slow if you have large folders.

  • 35 Cristian // Sep 12, 2007 at 4:19 am

    CB: still about context: pressing View/Message source (apple-U) when viewing the message in Thunderbird will show the message path in the source window title.

    Further, going to that mail folder will actually show the message selected. If the folder is already open, just open another one and come back... Not ideal but still useful.

  • 36 jmr // Sep 12, 2007 at 6:14 pm

    Hi Dennis, hi james
    my solution is in my question!!!! (not actual solution , but...)

    just type (a part of) t h u n d e r b i r d before (after) key-word in spotlight.... ;-)

    Next,
    spotlight can't (it seem that) read date of *.mozmsg

    some ideas?
    thanks
    jmr (nihil nove sub sole)

  • 37 Bluppy2 // Sep 15, 2007 at 8:32 am

    Hi Dennis!
    Do you have any progress in the problem James mentioned on the 5th of August? :) (showing mail under the 'mail' category instead under the 'document' category)

  • 38 iAquarian.com » Blog Archive » Using Spotlight to search through email in Thunderbird // Oct 15, 2007 at 12:15 am

    [...] being able to search for messages through Mac OS X’s Spotlight. Until I found this link to Dennis’ blog through mehtanirav.com. He has the full step-by-step on his site, but here’s the quick run [...]

  • 39 scf.code » Blog Archive » How I Made (Mac) Spotlight and Google Desktop Index My Thunderbird Messages // Oct 23, 2007 at 6:39 am

    [...] Spotlight Mac desktop search feature. The instructions by Dennis in Canada were most valuable, see ref {1}. Dennis gave us a valuable head start on getting this to work… As a complement I felt it [...]

  • 40 Stephen // Oct 27, 2007 at 4:29 pm

    I wrote up my recipe for this on my blog http://scf.homedns.org/

  • 41 RICHARD MISCHOOK // Nov 7, 2007 at 5:28 pm

    Denis,

    Great tips although on my box (running Leopard) I needed to move Inbox.mozmsgs to my home folder and then create a soft link to it from inside ~/Library/Thunderbird/... I seemed to need to do this to get spotlight to index although I also ended up using mdutil -e / to force Spotlight to re-index the who thing.

    I have noticed that Thunderbird does not seem to be creating eml versions of folders other than Inbox (which I use to filter stuff into)? Do you have any experience of this?

    Thanks

  • 42 Brent // Nov 11, 2007 at 12:52 pm

    Dennis, I appreciate this tutorial, but after I did it, I have lots of .mozeml files but no .eml files. And Spotlight doesn't seem to have indexed any mail. Any suggestions?

  • 43 Piero // Nov 12, 2007 at 1:49 pm

    Does this work with Leopard and Thunderbird 2.0.0.6?
    I followed your instructions, had my mailbox coverted to .mozeml files, but Spotlight ignores them altogether...

    :-(

  • 44 dennis // Nov 25, 2007 at 5:00 pm

    Brent provided a tip that you can force reindexing by navigating to your Library/Spotlight directory and running thunderbird.mdimporter.

    Not sure why Brent's comment won't show up here, but thanks nonetheless!

  • 45 Sergej // Dec 23, 2007 at 7:12 am

    For those who cannot make it on Leopard:
    http://www.tuaw.com/2007/11/13/spotlight-in-10-5-skips-over-user-library-files/

    Basically, Spotlight does index Thunderbird mail, but it won't search inside ~/Library by default. One has to go for Universal Search (Command-Option-Space), and setup a criteria with Other ... -> System Files.

    If the mozeml files are indexed properly (more on this topic you can read here: http://razal.de/~ibn/techblog/mac/thunderbird_n_spotlight.html)
    search results will show up Thunderbird mail.

    Sergej

  • 46 Jeff // Feb 3, 2008 at 5:36 pm

    So I followed the updated directions and now I have tons of .mozeml files, and an updated importer and I've clicked the System File "In menu" option under Other for spotlight, and I still can't find a single email.

    Any other tricks?

    -- Jeff

  • 47 Sylvain // Feb 10, 2008 at 5:36 am

    For me mozeml only are created but only for the messages I have read in TB. no eml files created.
    Futhermore I can't find any message by content. If I search for "thunderbird", all my mozeml appeared in spotlight but with their uselessnames (random string).

    Any hints ?

  • 48 flightless // Mar 19, 2008 at 6:45 pm

    same thing with me, Its just the emails that I read that are written as mozeml. and spotlight won't index them even if I try the command-option-space search.... Looks like we'll have to wait for TB3
    :(

Leave a Comment

-- Why? This question confirms you are a human user.

  • Advertisments