How to use a large mail box in real life?


#1

I see that with 99% of hosting companies you can create your gibibyte email account (or even bigger). I have, but at one point when you have hundreds of emails, your email client gets really really slow if you use IMAP. Also, the search feature doesn’t work very well, you search something that you know should return results but you get no results… If you use POP, you don’t have these problems, but you can’t access your mails from any computer, as you download them onto your local computer (and your mailbox is empty 99% of time).

In the end, Gmail, which is free, works better than a paid mailbox from a web hosting company because your inbox pops up instantly, when you search something, you get the search results in half a second. And you can use several gigs easily, it makes no difference.

How can I use a gibibyte mailbox with thousands of mails, lots of attachments, with a fast/efficient search feature, and have it working as fast as Gmail? Is IMAP flawed? What do you think?
Thanks.


#2

Yes. Yes it does. Especially if you leave lots of mail in your inbox. Moving mail from your inbox to other folders on a regular basis so that your actual inbox is “lean and mean” goes a long way toward addressing that problem.

Frankly, I use POP at Dreamhost in conjunction with a Gmail Account (where I forward stuff I might want to keep available “from anywhere”).

–rlparker


#3

Yes, but I guess this “Archive” directory would get messy the same way and I can’t imagine having to access it… My email client would go crazy and freeze…


#4

Well, it certainly could if you used a single directory. Using a series of directories (by month/year, project, or other “sub-sort” criteria) would help to a point but, at the end of the day, if you just leave it all on the server there will be a trade off between convenience of searching and speed.

–rlparker


#5

Just for the record what are you using as mail client?


use VIC3M for a 70USD discount on all plans (except monthly payments) +10% on storage and bandwith.


#6

I use Opera’s email client.


#7

I’ve personally come to the same conclusion and switched to Google Apps for email, and web stuff here at Dreamhost. I doesn’t seem that (most?) shared hosting setups are directed at supporting moderate to large email stores over IMAP.

Maybe this will get better if/when database-driven email systems become more widely used.


#8

Yes. The problem is that no e-mail client (except webmails) are compatible with database-driven email (they only work with POP & IMAP).


#9

I was thinking more along the lines of a database backend, something along the lines of DBMail - http://www.dbmail.org/index.php?page=overview

I’ve never used DBMail, or seen it used, but I imagine it would scale much better than most of what is currently used. There are other, similar servers out there which do even more for the index and searching aspect of mail storage. But I’m sure it would be a huge task to make such a switch so I doubt it will happen any time in the near future if ever.


#10

Thanks for the link. It looks nice but it stores attachments into the database (it should use the file system). Also, I don’t think I’ll be able to install it in DH as it’s a C++ application, from what I read :frowning:


#11

I find it incredible that web hosting companies accomodate with situation. Maybe they prefer it this way because people kind of “have to” download emails to their own computers, and their email accounts are virtually always empty, using less HDD space on the web host end.


#12

Tat could be, I suppose, but I suspect the HDD space is less of an issue than the processing power required to handle manipulating really large collections of received, sent, stored, etc. email.

It’s interesting to me that web hosting companies are in the “email business” at all. They certainly do not have to be (well, except at present for competitive reasons). Email and Web hosting are two different functions, and I wouldn’t be at all surprised if one day we find web hosting companies just saying “We don’t do email” or “it’s an add-on service, and here is what it costs”. :wink:

–rlparker


#13

Are you serious?!! Eh eh :stuck_out_tongue:
I mean, a company that says this is like saying “We don’t offer PHP, only as an addon”. Everybody uses PHP, everybody uses email :slight_smile:
Not everybody wants to use Google Apps (although it’s great).


#14

Absolutely! he he he

I didn’t say I would like it, and I did point out that competitive forces wouldn’t allow it to happen at present. I said I wouldn’t be “surprised” if it happened; that doesn’t mean that I think it will happen! :wink:

Int he past, there were companies that said one thing or another that we now expect to be “included” was an “add-on” (for example, DH used to have MySQL only as an add-on).

“Email” seems to be particularly suited to such treatment because of the varying levels of service needed for different users, the difficulty of keeping “shared” email server off of blacklists, and the SPAM issues altogether.

Dedicated email services could possibly focus on different things, and provide a wider range of services, than a “hosting” company that is primarily focused on serving websites.

–rlparker


#15

Companies such as DH have made the unfortunate distinction of calling themselves “Webhosts.” Unfortunate in that people think that Web==Internet. A more accurate title would be “Domainhosts.” And typically, a domain consists of email, FTP, web, and the back end to both of these services, including spam filtering and databases. I think shell access is super-cool and goes way beyond simple domain hosting. It makes sense that all services related to mydomain.com are hosted by the same company.

So I don’t think email should be an add-on, and should be part of basic service. DH has already taken the steps to separate the services hardware-, and probably network-wise. In a way, they are offering dedicated email service by isolating the email part of your domain. It would make them even more competitive to build up their email feature offerings. Maybe they’ll even grow to create a PS for email.

-Scott


#16

I agree with you completely on that, though I worry that it may not always be that way. Fortunately, it looks to me like it will remain part of “basic service” for the near future, though moving to “enhanced” dedicated email services is already almost a requirement for many businesses. :wink:

You make a really good point about the segregation that is already in place, and an email PS would be very nice! :slight_smile:

–rlparker


#17

Alright. Anyway, I added a suggestion in the Dreamhost web panel: “Use DBMail as mail server backend”. It doesn’t cost to ask :slight_smile:


#18

From glancing at DBMail’s website it looks like DBMail would be incompatible with Maildir email storage. I get the impression that DBMail uses a database as the mail store, not merely a means of indexing the mail for quick searches.

If DBMail were used merely for searching email server-side (essentially handling only the IMAP search command(s)) it would be nice. I could, theoretically use some program to get quick server-side searches. Perhaps a program better suited for searching large volumes of text (like the text indexers Beagle and Tracker use) would be a better choice than PostgreSQL, MySQL, or SQLite (DBMail’s backends). Such an arrangement would theoretically make Thunderbird’s search feature (which, for IMAP accounts, hands off the real searching work to the server) work without so many timeouts on large mail folders.

But I want the easy/powerful manipulation server-side filtering/sorting Maildir affords me via procmail. If I had any improvements to suggest to Dreamhost it would be that all of my email addresses (current and future) should be exposed to me as Maildir folders on my account so that I can use procmail server-side to manage the email for me before I connect with my IMAP client.

I don’t want to have to connect with an IMAP client to do email filtering and sorting. And no MUA’s filtering I’ve seen can compete with what procmail feature-wise.


#19

To me, the question is whether the service should be disaggregated from the web hosting. In my own personal case, I’d like it to be because I don’t use it - well, I do have a few addresses that I have forwarded to my “primary” non-Dreamhost account.

Given that I don’t use the service, do I want to pay for it? No, if it is a significant part of the price I pay. But do I think it is? No, I don’t, so I’m satisfied with the bundling.

Then again, I see a possible future where web hosts become mere aggregators of primary service providers. For instance, you’d have email provided by gmail, blogs provided by wordpress.com, bulletin boards provided by some specialized phpbb hosting provider, etc. In this world, there’d still be a slice of the pie for honest-to-god apache/html/php/RoR custom sites, but a lot of the services that can be asp’ed away would be.

What are [color=#CC0000]50DISK50[/color], [color=#CC0000]3DOM50[/color], and [color=#CC0000]1IP1DOM50[/color]?
More Dreamhost coupons


#20

One thing that drives “professional” email hosting apart from “webhost” email hosting is the simple fact that proper antivirus and spam-filtering takes humongous amounts of CPU power and memory; on a reasonably busy domain with just a few users, it can easily be more than the “average” dynamic webpage. That particular gap will only widen … It’s simple to provide email service per se, it’s not so simple to provide well-maintained spam-filters, virus-scanners, keep the host(s) out of blacklists, and be reasonably nice to the internet-community at large (which includes the option to not accept emails in the SMTP session due to spamfiltering or antivirus-filtering instead of trying to send a bounce to an unrelated address later on – and this makes it necessary to do filtering in real time).
Even a somewhat optimized setup of spamd (from SpamAssassin), clamd (from ClamAV), procmail (for user-defined filtering – and note there is no daemonized form of this, so every time you load that huge ruleset of yours, almost all the regexes need to be compiled and applied), a variety of realtime blacklists (dnsbls as well as Razor and such) can easily grind a server to a screeching halt with a small flood of mails (which may even be legitimate) – and this doesn’t even take into consideration accessing the whole thing through IMAP and the like (you’d like to have sieve-support, server-side fulltext searches, etc.).

Brave new world, that is.