Sending beaucoup email from PHP

software development

#1

Hi,

I need advice speeding up execution of a script that sends personalized messages to a fairly large recipient list.

I’m using the phpmailer class from sourceforge to send MIME email to a list of about 50k users. It’s very slow, whether I’m using smtp, sendmail or php’s mail(). Actually it seems to hang completely if the batch size is much larger than 250 recipients.

I read that using smtp would speed things up but I suspect that’s only true if each mail is identical. As I mentioned, mine are customized with recipient’s name and a few other unique items of information.

I’m wondering if anyone can give me some tips on pushing this thru smtp faster. I know that’s the bottleneck as the select query and file reading portions of my script (for building the html message body) run nice and fast.

Any help will be sincerely appreciated!

Barry


#2

Even if the mailing is permission based, you probably shouldn’t be sending mail to that many addresses from one of the hosting machines. We ask that people sending mailings to that many people use our announcement system.

Please also make sure that everyone on that list has specifically asked to receive mail from you, and that they’ve confirmed this fact (and their address) via some sort of closed loop system.

If you’re putting everything on one line, it’s possible that the line size exceeds that which the SMTP spec will allow. You may want to fold the recipient-list onto multiple lines (see also older posts in this forum).

If each mail is unique, wouldn’t each mail have its own (single) recipient list anyway?


#3

I’m guessing Barry has already seen this, but for future reference our spam policy can be viewed here:

http://www.dreamhost.com/spam.html

  • Jeff @ DreamHost
  • DH Discussion Forum Admin

#4

Actually, I’ve got the same issue Barry. My site collects signup information from new members, currently at just under 30k. And they are free to specify whether or not to receive any emails.

As Barry mentioned, I too would like to send an email to each of my members (who have agreed to accept emails), but the contents of each message is a little different, based on their account info, yadda yadda. In the past, I’ve used a PHP script to send out each custom email, one-at-a-time, using PHP’s mail() function. However, I realize that doing so puts a large load on the server, so I have always tried to stagger it - maybe pausing for a couple of seconds after every 30 to 50 emails.

You mentioned using DH’s “Announcement” system, what exactly is that? A regular mailing list is not quite appropriate since each email is customized for that particular member. I really don’t want to affect the load on the server too much, but I also want it to be convenient. Any more ideas?

–Brad Barkhouse
Be a Contender!
http://www.beacontender.com
brad.barkhouse@beacontender.com


#5

The announcement list system is basically what you describe, albeit with little in the way of per-user customization. We use the term “announcement list” to better distinguish it from our “discussion list” feature (which has a totally different function).

In your case … Splitting the distribution into small-ish chunks and sending it over the span of several hours is probably your best bet. We really don’t mind as long as your list follows our spam policies to the letter and it doesn’t cause problems on the server. Unfortunately, with the latter concern it’s sometimes hard for customers to know (as they don’t get the 3 AM pages like our Admins do!).

I’m not an Admin, though, so I should probably defer to Will who is more familiar with what our shared servers can handle without impacting other users. Will?

  • Jeff @ DreamHost
  • DH Discussion Forum Admin

#6

I’m hesitant to give specific numbers, but basically there shouldn’t be any bulk mailing going on from the servers… that is, if you have an application that sends out occasional email, that’s probably fine. If you are sending out small amounts of mail, this is probably going to be fine (assuming people have both asked to receive the mail, and confirmed their presence on the list, and that you have IP addresses and timestamps for each subscription)… however once it gets to the point where it’s impacting the stability of our system, we’ll probably disable it or write you a love note. I’d say anything more than a few hundred messages in the space of an hour or two is probably pushing it.


#7

Thanks for the advice, will. I’ll reconfigure my utility scripts so that I throttle down my bulk emails to 100 per hour. I generally only perform them about once per month, and as I said, it’s only to those who have acknowledged and accepted it when they signed up. Other than running “top”, is there another more reliable way I can monitor the load on the server?

Ideally, I could afford my own dedicated server, and be able to send as much email as I wanted. But before that day, I’ll do my darndest to keep my portion of the server’s load under control. :slight_smile:

Thanks guys, keep up the GREAT work!

–Brad Barkhouse
Be a Contender!
http://www.beacontender.com
brad.barkhouse@beacontender.com