Mail getting refused


#1

I’m using a Drupal site with a built-in contact form that sends an email to the selected user. However, I’m getting several bounces from Comcast, saying that access has been denied.

Is the DH mailserver blacklisted somewhere, or is this a problem I can fix by adjusting the MX records or Junk filters?


#2

Several Dreamhost servers, as well as many other legitimate email servers, are periodically listed in any one, or more, “blacklists”. Also, some ISP’s have blocked Dreamhost server (which are shared among many users) becuase of the amount of mail they receive from DH servers (if there is that much mail, it must be “spam”, right?) Doh! You can’t really impact it with “spam filtering”, though I suppose you could via mx settings by changing the email service/servers used by your domian (I don’t really know if that is what your question was referring to.)

Comcast, AOL, ATT&T (possibly others) have, of late, blocked several Dreamhost mail servers. Generally, Dreamhost has had good success in getting blocks removed at AT&T, some success at AOL, and virtually no success with Comcast.

There is considerably discussion of these situations on this forum and on dreamhoststatus.com.

It is important to understand how this all happens; there is very little, ultimately, that Dreamhost can do in the face of ignorant, fascist, and abusive email administration policies.

Customers of these organizations are being poorly served when their own ISP decides arbitrarily what mail they will deliver to their clients. While their “spam control” attempts might be well intentioned, their brain-dead implementations succeed in blocking large amounts of legitimate mail their own users have requested (and, in some cases, even forwarded to their ISP accounts).

It has been my experience that many of these blocks are temporary, thanks to Dreamhost’s efforts, but I believe this whole thing is going to get a lot worse before it gets better.

–rlparker


#3

I am a small business consultant who has several clients using DreamHost, and while I understand that this may not directly be DreamHost’s fault, what am I to tell my clients? Oh well, the problem is on recipients end? This will not fly, especially since so many business live and die by prompt communication via email. The recipient is not aware that the message was blocked, only that they did not receive a message. These blocks are not limited to the AOLs, Comcasts, etc… there are private domains using various databases and spam services blocking DreamHost servers as well.

There has to be a fix sooner than later.


#4

ev1 recently started blocking subscribers to a mailing list I manage.

It’s particularly difficult to deal with as an intermediary. The affected client can’t really do anything effective. Neither can we, really. I suppose it’s all down to the ISP and DH. And SpamCop, which I’m beginning to loathe about on par with spam itself.


#5

Your frustration is completely understandable; I have some of the same issues with some of my clients. Of course, I cannot tell you what you should tell your clients, though I can share with you what I tell mine.

I tell them the truth, irrespective of whether or not it will “fly”. To my way of thinking, to tell them anything else is to do them a disservice. I would be remiss if I failed to point out to my clients that, in this day and age, for a business to “live and die by prompt communication via email” is a fundemental problem with their business model that should be addressed much sooner rather than later.

I say that because, the net is changing and, while in the past we have enjoyed highly reliable and often near immediate delivery of email, the whole email experience may well be going the way of usenet; because of spam, bandwidth load, signal-to-noise ratio, etc. it has become problematic to rely upon it for timely or “mission critical” communication. While in the (recent) past this was not the case (and in most circumstances still is not the case), we should realize that if it “absolutely, positively has to be there” by a particular time, email is often no longer a reasonably robust way to accomplish that in a busness environment.

For all of my clients, email is still useful and the overwhelming majority of email sent, and received, is delivered without incident and in relatively rapid manner. For the other problem email, whether or not we “like” it, there is no way to “make” any mail server “accept” any piece of email; the administrators of each server can accept, or reject, any mail they choose. Issues of contractural obligation to process mail by an ISP or mail service are generally made moot by the Terms of Service or Acceptable Use Policy, which give the service provider these “rights.”

If the users’ market, the customers who are having their mail blocked by their service providers, is not able to prevail with the management of their services, and absent any government intervention (the whole “net neutrality” issue is a much bigger, and more complicated subject - though this is related) I’m pretty sure this situation will not get better in the near and foreseeable future.

What to do? I’m not sure what is best, or what will work best for you, or your clients, but, in addition to (or instead of) educating your clients regarding the realities of email communications’ present shortcomings and developing alternate business processes to account for these, you could throw money at it by maintaing your own static IP addressed mail server which you somehow keep off all the blocklists, and use that for your mail service. You could also change to a dedicated email provider, and hope they have less issues (though I really don’t think that is likely to be of much help).

We are also starting to see some corporate email environments starting to “reject” mail from dynamically assigned “consumer based” IP addresses. They don’t want their mail servers used by their employees, at work, to exchange email with “friends and buddies”, to introduce virii into their network, and/or receive huge amounts of “Pass this letter on” or “Fwd: Great Joke/Video/Jpeg/etc” garbage that makes up such a hugh part of many “email connected” employees’ traffic. I’m not saying it’s “right”, just that it is happening.

For most of my clients, the costs and complications of all that has led to them deciding, upon understanding the nature of the problem and the alternatives, to encourage those relatively few associates impacted by these blocks to arrange for an alternate/additional email provider (easy enough to do) and/or utilize alternate communications methodologies (phone, fax, mail, Fed Ex, etc.). The alternative methods to facilitate a particular issue are myriad, and can include online document repositories (call the client, have them download/upload the file), P2P networking, IM/IRC technologies, RSS, VPN, etc. Granted, none of this is as “convenient” as email the way we are used to using (abusing?) it, but it does provide alternatives. I mean, really, how did we get to the point where we are moving multi-megabytes of media files and documents via email attachments anyway? Email was never designed for the way it is now being generally used, but I digress.

Why? While I would love for this to be true, until the “blocking” entities find an economic incentive to change, why should they not limit their bandwidth/processing loads by such activity? I don’t expect them to change without being “forced” to change. One of the things that could provide such an incentive would be a “revolt” and “exodus” from their servce by impacted users. Another might be strong net-neutrality legislation that would penalize such “packet discrimination” (not that I am arguing for that at this point!). I’m sure there are others. I certainly don’t think that implementing convoluted “forward around” operations and wasteful “duping” of mail through various addresses to find one that “works” for every user (adding considerly to congestion in the pipes) helps as much as it hurts; while it might ultimately get a given piece of email through, it just magnifies the problem in the long run as it increases the “noise to signal” ratio of the email medium (as if it is not high enough already).

I’m sure we are only seeing the beginning of this, and I don’t think it is going to get better anytime soon. To me, it is more useful to point my clients to alternate ways of dealing with the issue, than it is to “rail” against the provider of a shared email server (that I elected to use) because they were victimized by heavy-handed mail administrators. There is nothing “unique” about Dreamhost having servers being blocked by “the AOLs, Comcasts, etc… (and)… domains using various databases and spam services” ; these same databases and spam services routinely block many providers.

Until this whole issue starts to come “full circle”, if it does at all, the best any provider can do , including Dreamhost, is to carefully monitor their servers to prevent them being used by spammers (which I am convinced Dreamhost does very well), eliminate or tune processes as is reasonable to avoid the “worst” of the blocking (control certain forwarding circumstances, email volume limits, etc - while recognizing that you ultimately can not gurantee that someone will not block you in spite of your best efforts!) and educate their users as to where to complain - those implementing the blocks are the ones impacting “our” clients who use them.

That said, there is only so much we can “take on ourselves” as consultants and advisors to our clients. If/since they have chosen to have their email service provided by, or continue to choose, a service that filters so agressively they can can no longer reliably receive mail they desire, there is no reasonable way for us to sheild them from the effects of that decision.

Other than explaining what is happening, and why, and using our business and technical expertise to suggest alternatives, there is little more we can do. Ultimately, the situation is mitigated, or not, by their business decisions.

–rlparker


#6

For sending mail from a normal mail client, you can set the smtp server to your own isp. In your case, sending mail from a web script, that’s a serious problem. I would like to know any remedies myself.

If DH would let us set the smtp server that’s used for mail, that would solve it for me. Well, until my site has so many users and is sending out so many emails that my isp shuts down my mail service. So, I guess that’s not a perfect solution either. Chances are my site will never be sending out that many emails, so it should be okay for me.

From the command line, is there a way to set my smtp server? Php is just using the Unix mail command, right?

Terry


#7

Some PHP software allows you to set an SMTP server instead of using the generic sendmail command. But most third-party scripts don’t, and I don’t think sendmail can be customized unless you are going to install your own version of PHP…

A drastic solution would be just to ban comcast, aol and yahoo domains from being used to subscribe, and tell them to use another email provider - one you know doesn’t block emails. But, effective as it is, it’s equally effective in ticking off your users/customers, so if you are running a business, it’s not the best option.


#8

I was just told by a visitor to my site that any non-DH address used by formmail is refused. Is this true? When did this happen? Was there an announcement?


#9

Jennifer,

I use the Dreamhost-provided formmail.cgi on many sites, and have not experienced any such problem. I am also not aware of any recent changes to that script.

Dreamhost’s formmail.cgi was changed quite some time ago to prevent it being used to process forms hosted outside of Dreamhost, though that should not impact a user visiting a form you are hosting on Dreamhost; it just prevents “outsiders” from using the formmail.cgi as the “action” for a form hosted elsewhere (this is, I think, a "good Thing"tm)

If you want to do some tests yourself, feel free to PM me and I’ll send you a couple of my “non-Dreamhost” addresses to use as a test (or point me to the form, and I’ll test for you) :wink:

–rlparker


#10

I’ve tested it and it is true. When recipient field contains a gmail address, this is what appears upon submission:

"Please make sure you have filled in the required recipient form field, and that the e-mail address used is hosted with DreamHost.

As a spam-prevention matter, we can now only allow our formmail to forward to email addresses hosted with us. If you’d like the recipient to be an email not hosted with us, please just create an address @yourdomain.com to forward there!

The recipient was: [ xx@gmail.com ] "

When I change it to my dh domain address, it works fine.


#11

Thanks for the “Heads Up” on this one, Jennifer. Upon closer checking, it seems I have not experienced the problem precisely because all my forms’ recipient fields are set to a Dreamhost address (although many of those are forwarded ouside of Dreamhost).

Whiile I understand Dreamhost’s reasoning here, they really should update the documentation for the recipient field to reflect this. I can only imagine how frustrating it would be to only find out about it from the submission error message after you have set up a form.

I hope this doesn’t necessitate a whole lot of work on your part to implement the workaround. :slight_smile:

–rlparker


#12

The person visiting your site is the sender–that error refers to the recipient (you), which should be coded into the script itself… not a field on a form.

If people could just enter anything as the recipient, your account would probably be shut down for spamming within 10 minutes. :wink:


:stuck_out_tongue: Save up to $96 at Dreamhost with ALMOST97 promo code (I get $1).
Or save $97 with THEFULL97.


#13

I don’t think anyone said anything about allowing the user to set the recipient value. The recipient value in the formmail script is set within a hidden form field.

In my case, this value is being set dynamically by a javascript. It was very simple to change, just annoying that I didn’t hear of the policy change (if there was one).


#14

I was referring to this: “I was just told by a visitor to my site that any non-DH address used by formmail is refused.” Thinking that you meant if a visitor entered their email address, and it wasn’t hosted at DH.

If you’re generating the recipient address with Javascript, is there a default address in a noscript tag for users with Javascript disabled?

I didn’t think this was a recent change, though (but I haven’t used it). If DH has been letting form mail forward to any address, it wouldn’t surprise me if that had a lot to do with some of the spam blocks… especially if they found it necessary to change it.

In any case, the way they have it set now is definitely a good thing.


:stuck_out_tongue: Save up to $96 at Dreamhost with ALMOST97 promo code (I get $1).
Or save $97 with THEFULL97.


#15

“…is there a default address in a noscript tag for users with Javascript disabled?”

No. I won’t get into my decision about that, because it is off topic. I will only say that my site is about JavaScript, and leave it to you to make assumptions about my reasoning.

Still, that doesn’t explain why DH has made this apparent policy change. Any thoughts from DH insiders?


#16

"Still, that doesn’t explain why DH has made this apparent policy change. "

They should have made it a long time ago. I doubt there are many hosts out there that haven’t already had that setting in place for a long time.

It’s one less way for it to be exploited by a spammer. If someone finds a way to spam through it, they’ll only be able to spam you, which wouldn’t lead to being blacklisted with places like AOL, Comcast, etc.

One thing that is obvious, is that they made the change while being labeled as spammers by a few ISPs… so I’m guessing there have been exploits that maybe went unnoticed for awhile. Either that, or they’re sick of fighting the spam issue and took a pro-active step towards stopping what would have likely been their next problem.


:stuck_out_tongue: Save up to $96 at Dreamhost with ALMOST97 promo code (I get $1).
Or save $97 with THEFULL97.


#17

it should be this way. there are many spammers and phishers (particullarly phishers) who hijack sites. this prevents information being sent to an outside email address.

johngrayson.com
suffocate[us]


#18

And it’s an unfortunate fact that most home-made mail forms are not at all secured against email injection. I haven’t tried DH’s script (I assume it is secure), but most people who make one on the fly don’t take the time to protect it against the various exploits.