Am I being too hard on DH

#1

… by expecting better uptime for the mail server? I know DH is trying (what is presumably) its best - see answer to POP3 outage below - but the result it not good enough.

Serious question: Why is it that DH’s mail service uptime is so much poorer than that of the lowly connectivity providers I use? (NTL, Claranet etc.)

Is there something about all-services hosting that prevents good mail service? And is there a good mail-only host I should consider?

Thanks.

Chris

[quote]At time of writing mailboxes chrisjj, chrisjjbackup and ff mailboxes @
mail.chrisjj.com are refusing POP3 connections. What’s up?

Chris

[/quote]

Sorry about this Chris. We had some attacks that made our network very
slow and made services unuseable. Our administration team has dealt with
the attacks, which happened again today, and we are confident we can
pinpoint and remove these attacks in a more timely fashion. Sorry about
the late reply, this issue caused a large amount of support.

#2

I’ve been searching for a good “all-services” host for a few years and I’ve finally decided that there (currently) is no such host. I’ve documented my research on my IMAP Service Providers page. Of the hundreds of providers I’ve looked at, I currently recommend only 5 for mail hosting.

Maybe this will help you find a good mail-hosting provider. Or at least see how much is involved in email hosting these days.

Nancy

Infinite Ink ~ http://www.ii.com
Deflexion & Reflexion ~ http://deflexion.com

#3

[quote]I’ve documented my research on my IMAP Service Providers page.
Of the hundreds of providers I’ve looked at, I currently recommend
only 5 for mail hosting.

[/quote]

Thanks - useful. Since I’m looking for mail hosting not just IMAP hosting, what would be nice in your table is an indication of whether POP3 is provided.

I see you conclude that web and mail should be separately hosted. Why? And why do some ISPs disagree? :wink:

[quote]1: No Auto-Deletion of Messages Unless Requested by User

[/quote]

You might like to know that by default DH removes messages from INBOX after 250 messages or 45 days. The fact that they are recoverable from a separate IMAP mailbox is no consolation to POP3 users! :wink:

#4

Eventually I’ll add that (or let users add it via a wiki!). Almost all IMAP providers also offer POP. The only ones I can think of that do not offer POP are the AOL/AIM free accounts and the FastMail.FM Guest account.

I didn’t originally think that and I still don’t think that they should be separate, it’s just that I have looked very hard to find a provider that is good at both. And there is none that I could find (but let me know if you find one!).

My guess is that most ISPs would love to get out of the email business. It is a huge drain on resources and a thankless job.

Yes, I am interested in this and will add a link to this in the DreamHost section of my page. Is this documented in the DH wiki or KB? I use server-side filters to route my mail to mailboxes other than INBOX so fortunately I haven’t had to deal with that. Does anyone know why DreamHost does that?

Thanks,
Nancy

Infinite Ink ~ http://www.ii.com
Deflexion & Reflexion ~ http://deflexion.com

#5

[quote]My guess is that most ISPs would love to get out of the email business.
It is a huge drain on resources and a thankless job.

[/quote]

It can’t be that bad, or there wouldn’t be email-only providers!. :wink:

Is this documented in the DH wiki or KB?

[/quote]

Wiki, dunno. KB, I thought so, but now I can’t find it. More seriously undocumented are very serious mail security issues unique (in my experience) to DH - these rather make the message removal problem academic for serious mail users.

#6

[quote]You might like to know that by default DH removes messages from INBOX after 250 messages or 45 days

[/quote]

It is annoying that you can’t disable this… I looked at this for my little brother, and I think it’s possible to set the limits so high that they won’t cause any problems in actual practice.

For instance, “remove messages older than 10000 days”.
Remove read messages when inbox reaches 10000 messages

I’m not sure whether the archiver actually respects these high limits, but mailboxes seems to let me set them.

interestingly, the settings (and default limits) aren’t consistent between mailboxes.[domain] and the panel options.

[quote]The fact that they are recoverable from a separate IMAP mailbox is no consolation to POP3 users! :wink:

[/quote]

webmail… can always select all and move back to the inbox from there.

#7

[quote]For instance, “remove messages older than 10000 days”.

[/quote]

The UI says “ALL messages older than 1000 days will be expired, regardless of what you select here.”

[quote]Remove read messages when inbox reaches 10000
messages … but mailboxes seems to let me set them.

[/quote]

ISTM that’s a highly desirable but unintentional feature, and now you’ve told everyone, it will soon disappear! :wink:

Meanwhile though, the sang is that, the panel won’t accept above 2000 (recently reduced from 3000) so setting more on mailboxes will prevent you saving any other changes to that mailbox on panel.

[quote]webmail… can always select all and move back to the inbox from there.

[/quote]

… from where the archiver will soon move it back out again! :wink:

#8

[quote]The UI says “ALL messages older than 1000 days will be expired, regardless of what you select here.”

[/quote]

Well 1000 days is almost 3 years… geesh. When messages get that old, surely someone can either archive them locally or in a separate IMAP folder.

… from where the archiver will soon move it back out again! :wink:

[/quote]

Hrm - I was thinking the files’ ctimes would be updated when they were moved / moved back, however that doesn’t seem to be the case from a quick test. If it’s a shell user, you could “cheat” by using “touch” on ~/Maildir/cur/*

re: POP3 users… I would really not suggest that people leave messages in their inbox for very long using POP3. With POP, the messages should be downloaded to your email client(s), and removed either immediately, after some time period (maybe a week), or when deleting the message out of the local inbox.

If it’s really important to keep read messages server-side, use IMAP instead. And in the case of POP3, it really /is/ a performance problem when a user has tons of old messages lying around.

This definitely isn’t a perfect system (and I’m sure gentle suggestions for improvements / enhancement would be considered), but the bottom line is that if DH didn’t enforce this, there would end up being a lot more performance problems with all users’ mail. I do think it would be good to have a checkbox that says “please don’t expire messages at all if the inbox has < 500 messages” - this lets people who keep a tidy inbox BUT like to keep some old messages around do so.

#9

[quote]n the case of POP3, it really /is/ a performance problem when
a user has tons of old messages lying around.

[/quote]

Surely the only performance hit is when checking the mailbox. And ten times as many messages costs nore more than ten times the check frequency. Yet don’t see people getting berated for frequent checks.

[quote]This definitely isn’t a perfect system

[/quote]

Nothing is. The mystery is why this system is so much less perfect than every other POP3 service I’ve come across. I don’t recall ever seeing one that puts a quantity limit on message. Or a time limit.

#10

We do actually tell people to stop checking their email so frequently if we decide it’s putting an unnecessary strain on the server. Also, it’s not really the case that ten times as many messages costs the same as checking the messages ten times as often. The bottlenecks in a large system like ours are not linear. There are thresholds that once passed will degrade performance significantly.

Also note that we do completely isolate email from web hosting. The two don’t mix well. They do share common file storage so your home directory can be in two places at once, but other than that they are isolated.

Anyway, in this case I think it’s just the servers being too busy. We’ve added some more servers but they’re not fully in use yet. Server loads should drop when those start being used. If they don’t drop enough we’ll add more servers.

  • Dallas
  • DreamHost Head Honcho/Founder
#11

[quote]Also, it’s not really the case that ten times as many messages
costs the same as checking the messages ten times as often.

[/quote]

Indeed. In my (limited) experience it is worse, due to the overhead of opening the mailbox.

[quote]I think it’s just the servers being too busy. We’ve added some more servers

[/quote]

Good to hear. fingers crossed…

#12

Also, we’ve been having some problems with idiotic phishers lately. They’ve been hammering our mail servers at times (including this morning). We’ve been taking steps to prevent it, but we have to be careful to only disrupt legitimate service as little as possible. There’s always trade-offs. Providing email service is not as easy as it used to be.

  • Dallas
  • DreamHost Head Honcho/Founder
#13

It is sad that we have a system under which the service provider has to make value judgements on the content of email addressed to its customers before deciding whether to throw it away rather than deliver it as per the service offering.

Mail delivery should not be treated as a free service to the originator. But as a paid-for service to the addressee. Because that’s what it is.

#14

I’m not sure exactly what you mean, but in the case I was describing the emails are not addressed to our customers. These are emails being sent out by people who have signed up for our service (usually using fraudulent credit cards). They’re using our smtp servers to send out phishing scam emails to random people who may be our customers, but probably not.

I agree with you that it’s a sad state of affairs!

  • Dallas
  • DreamHost Head Honcho/Founder
#15

Hi Dallas,

[quote]… people who have signed up for our service (usually using
… fraudulent credit cards). They’re using our smtp servers

[/quote]

How can this be possible. Don’t you check the validity of CC’s# before opening a service ? I agree with you that, this is a pain…

I’m going to change the frequency of mail checking in Thunderbird parameters.

#16

Sorry, since we were talking about incoming mail service, I assumed it was incoming mail you were talking about. I guess that you’ve done what’s practical to keep outgoing mail load separate.

#17

Yeah, the cards themselves are valid and the charge does go through. It’s just that the people signing up for our services don’t actually own the card! Most people don’t watch their card charges daily so they might not notice until they get the bill. We credit it back when we notice but some slip through. Be careful with your credit card numbers!

We have our own fraud detection system in place but some of them slip through. We’re always fine-tuning it to be as accurate as possible.

  • Dallas
  • DreamHost Head Honcho/Founder
#18

The same sets of servers currently handle outbound email via authenticated smtp as well as POP/IMAP. Different sets of servers do handle incoming mail from remote servers, though. We’re working on gradually isolating email services out from each other more, but there are overlaps. When we were smaller it wasn’t practical at all to completely isolate things, but it’s become more practical as we’ve grown.

  • Dallas
  • DreamHost Head Honcho/Founder
#19

[quote]We’re working on gradually isolating email services out from each other more

[/quote]

Good news - thanks.