CGI front end for DH mailing lists

software development

#1

Hiya, folks…

I’ve read several kBase entries on mailing lists at DH, and it seems I may be able to do much of what I’d like to, but not all…

I’d like to build a branded front-end for the/a mailing list management interface, which I can install on my web clients’ sites. My main reason for this is to allow the use of confirmed opt-in, which I’m now requiring of all new clients, and to which I’ll be upgrading all existing ones.

From what I can tell, I can do these:

  • Create an opt-in confirmation system by creating a layer in front of the DH CGIs that control list subscriptions. This uses Procmail and Perl (I’m already almost done with this feature).

  • Allow the client to edit Confirmation and Welcome messages (via the above layer)

  • Allow the client to create a special “List Help” address via a regular autoresponder

  • Allow the client to sub and unsub users via the DH CGIs (add_list.cgi, etc?).

But can I allow my client to do the following without forcing them to use the DH CP?

  • Create or delete mailing lists

  • View and edit a list of subscribers, as in the DH CP

I’d rather not have to create and delete lists on behalf of the client, and the DH CP would be totally bewildering to most of the folks I design for (besides the fact that I’d rather not allow them an admin password).


Second subject… I’ve never used MySQL, so I’m wondering what would be better for very small sites, with mailing lists of only 200 subscribers or so-- Flat files and a SendMail interface, DH mailing lists, or some sort of MySQL API?

One major wrinkle is that I’d like to be able to integrate email list info with postal list info and other “user preferences”, and it seems that using the DH mailing list facility might make that a little more complicated than it could otherwise be.

Thanks for all experience anyone might be able to shed!

…Bob W.


#2

hola. . .

You could probably roll your own slick mailing list program using php and mySQL. Perhaps easier still would be to pay a visit to www.hotscripts.com and browse the PHP scripts section on this subtopic until you found one you could install and customize.

good luck

Barry


#3

Hi, Barry…

Thanks for the suggestion, but with what I’ve already written in Perl, the PHP thing won’t be necessary. For now, I’m using a flat file structure, but I’m sure I’ll go with MySQL later.

Thanks again…

…Bob W.


#4

How many peopke are on your mailing list? Maybe the overheads of MySQL will not be as efficent as a small flat file database.


#5

Well, since I’m building a system that I hope to reuse for various clients, it would vary… but I would imagine that the folks I’m aiming at would mostly have fewer than 1000 subscribers… And generally somewhere under 500.

What would you think a top-end would be for maintaining fairly efficient low traffic, low sub and unsub flat-file use?

…Bob W.


#6

Ugh. That’s a tough question. It all depends on your code, the way you use file locking, etc.

The safe bet is to go down the MySQL route. With the new updates to the MySQL server on DH, this is more of a stable option. However if you’re to port this onto different servers, maybe a flat-file would be best for it to be usable by all.

How familiar are you with MySQL? I find it much easier to code and work with MySQL rather than flat-files. I do find flat-files mostly obsolete these days, with more and more hosting companies rolling out MySQL servers.


#7

You wrote:

[quote]Ugh. That’s a tough question. It all depends on your code, the way you use
file locking, etc.

[/quote]

…the bane of my existence.

[quote]The safe bet is to go down the MySQL route. With the new updates to the
MySQL server on DH, this is more of a stable option. However if you’re to
port this onto different servers, maybe a flat-file would be best for it
to be usable by all.

How familiar are you with MySQL?

[/quote]

Not.

[quote]I find it much easier to code and work
with MySQL rather than flat-files. I do find flat-files mostly obsolete
these days, with more and more hosting companies rolling out MySQL servers.

[/quote]

I’ve gotten the feeling that there’s yet another language/app I have to try to learn. :confused:

Speaking of which-- Do you know of any website that might allow auto-generation of SQL statements based on simple selections? Like… Fill in the names of your tables and fields, check some boxes, enter criteria, click “Submit” and the page hands you a string…?

That would certainly make this beginner’s life easier!

…Bob W.


#8

Yes I do … me .

I could give you a hand if you want. MySQL is very simple. The beauty of it is that the database does the work for you, so there’s a lot less work to do using Perl (or PHP or whatever you’re using).

Let me know how I can help you and I’ll be happy to write some code for you!


#9

wil wrote:

[quote]Yes I do … me .

[/quote]

You’re a web form?

[quote]I could give you a hand if you want. MySQL is very simple. The beauty of
it is that the database does the work for you, so there’s a lot less work
to do using Perl (or PHP or whatever you’re using).

Let me know how I can help you and I’ll be happy to write some code for you!

[/quote]

Well, I do appreciate the offer, but I apparently need to learn this stuff myself anyway.

If I get stuck, I may ask for some clues.

Thanks!

…Bob W.


#10

OK, Bob. No problem. I’d be happy to assist you or give you some pointers.

Let me know if you get stuck. My email is wil@fbagroup.co.uk.

Good luck! It’s not that steep a learning curve!

Wil


#11

Bob -

I’ve definitely got to agree with wil on the MySQL issue. It’s so much easier to work with a database than flat files, it’s not even funny.

A really good tutorial on the subject can be found here:

http://www.devshed.com/Server_Side/MySQL/Intro

…with a ton of other useful articles here:

http://www.devshed.com/Server_Side/MySQL

I’d also suggest the O’Reilly book on MySQL. Then again, I pretty much recommend all of the O’Reilly books. :>

Really, MySQL isn’t that hard. It may be learning a new language, but it’s one with not all that much syntax (at least not that much compared to alternatives). If you need any help, just ask in this forum and I’ll try to respond.

  • Jeff @ DreamHost
  • DH Discussion Forum Admin

#12

Hey, Jeff…

You wrote:

[quote]I’ve definitely got to agree with wil on the MySQL issue. It’s so much
easier to work with a database than flat files, it’s not even funny.

[/quote]

Hey, I just finished the bulk of the Perl/flat file implementation, and I ain’t laughin’.

[quote]A really good tutorial on the subject can be found here:

http://www.devshed.com/Server_Side/MySQL/Intro

[/quote]

Yeah, I’ve been there before… The tutorial seems to be based on building and maintaining a database exclusively through Telnet… and that makes it sound like a serious pain.

In order to get to the SQL statements part of the tutorial, you have to go through all this manual labor that isn’t necessary if you’re going to be using a graphical interface that sets up tables and such, as, if I’m not mistaken, Dream Host’s CP does.

Very painful to be foreced to learn stuff you don’t need.

On the other hand, if I’ve gotten the wrong impression of that tutorial, please let me know.

[quote]http://www.devshed.com/Server_Side/MySQL

[/quote]

This is actually more helpful, although again, everything is done from the command line. I’d really rather use a graphical interface, and just learn how to create statements.

[quote]I’d also suggest the O’Reilly book on MySQL. Then again, I pretty much
recommend all of the O’Reilly books. :>

[/quote]

As do I… But I’m trying to cut back on my own habit.

[quote]Really, MySQL isn’t that hard. It may be learning a new language, but it’s
one with not all that much syntax (at least not that much compared to
alternatives). If you need any help, just ask in this forum and I’ll try
to respond.

[/quote]

Thanks… The problem is really one of time. If I learned every new language that people told me I should learn, I’d spend more time learning languages than I would getting work done. :confused:

Anyway, for now, the flat-file system is complete (except for actual mail delivery), but I’m going to keep searching for a way to do this as painlessly as possible.

Speaking of mail delivery, Jeff…

I need to ask you about using SendMail for a mailing list. What’s the maximum traffic you would like to see? The lists I’m dealing with right now will have <1000 and generally <500 subscribers.

I could use the DH mailing list facilities if I could create a client interface with the features I need, but apparently that’s not possible.

I want to make sure I’m not going to overload the server or send up flares as if I were a spammer.

Thanks…

…Bob W.


#13

In order to get to the SQL statements part of the tutorial, you have to go through all this manual labor that isn’t necessary if you’re going to be using a graphical interface that sets up tables and such, as, if I’m not mistaken, Dream Host’s CP does.

We have an interface for creating databases and setting up the permissions, but not for the actual manipulation of the database - that would be a ton of work, especially when others have done a much better job at it. One in particular I recommend is phpMyAdmin (which also happens to be free):

There are some others as well, if you look around. I’m pretty sure it shows the text of the SQL statements, which will probably teach you a thing or two about the syntax without you even realizing it. :>


I need to ask you about using SendMail for a mailing list. What’s the maximum traffic you would like to see? The lists I’m dealing with right now will have <1000 and generally <500 subscribers.

Generally we recommend people use our own distribution list utility, but that has more to do with the fact that many 3rd party list utilities are buggy and take up a ton of resources more than anything (we have nothing against people using sendmail in and of itself, as long as it doesn’t overly impact the server). If you think the most you’ll have to send to is 1000, that’s not too bad I don’t think. Even better if you don’t send them all out in one batch, but stagger them out over a longer period of time (generally a good idea anyhow).


I want to make sure I’m not going to overload the server or send up flares as if I were a spammer.

Overloading the server is the real issue. If you’re worried about doing confirmed opt-in, odds are really good that you’re not a spammer. Regardless, if complaints did come in I would probably be the first to know about it.

There usually isn’t much doubt when a person looks like a spammer. As long as you’re not advertising offshore accounts via forged email, and a disproportionate number of people don’t complain, I wouldn’t worry too much about it. :>

  • Jeff @ DreamHost
  • DH Discussion Forum Admin

#14

Or the best one I’ve found is MySQLMan (MySQL Manager). A very powerful program, that shows you the syntax in a command window while working, which makes it a ‘fun’ learning tool.

http://gossamer-threads.com/scripts/mysqlman/index.htm

Check out the demo on the site, or just download it yourself. You’ll have a database up in no time. It’s also free!

I still haven’t found any good articles about using Perl with MySQL. I’ve seen a ton about using PHP with MySQL. But I’m not the biggest fan of PHP.

If you are a PHP fan however, check out a good article on Webmonkey which should get you started.

http://www.webmonkey.com/

Wil


#15

Jeff writ:

[quote]Bob W. writ:

[quote]I need to ask you about using SendMail for a mailing list. What’s the
maximum traffic you would like to see? The lists I’m dealing with right
now will have <1000 and generally <500 subscribers.

[/quote]

[/quote]

[quote]1000, that’s not too bad I don’t think. Even better if you don’t send them
all out in one batch, but stagger them out over a longer period of time
(generally a good idea anyhow).

[/quote]

Yeah, that was a natural part of my plan.

Overloading the server is the real issue. If you’re worried about doing
confirmed opt-in, odds are really good that you’re not a spammer.
Regardless, if complaints did come in I would probably be the first to
know about it.

[/quote]

I’m not worried that you’re going to think that I’m a spammer-- I just wanted to make sure there weren’t some software checks in place that meter the volume of mail being sent and set off alarms when it reaches some arbitrary level.

[quote]There usually isn’t much doubt when a person looks like a spammer. As long
as you’re not advertising offshore accounts via forged email, and a
disproportionate number of people don’t complain, I wouldn’t worry too
much about it. :>

[/quote]

I don’t, believe me. I’m one of those people that spammers call an “Internet Terrorist”.

Thanks…

…Bob W.


#16

Well, I downloaded MySQLMan and installed it…

Buggy, buggy, buggy.

There are support-forum-documented problems with file paths. I had to hack the ‘root’ variables in the source to get the thing to stop giving me “file does not exist” errors. Some folks never get theirs to work right.

But even with that, the connection and or logins are constantly failing and sending me back to the login page.

I had the files on database.myclientsdomain.com, and thought that might be the problem, so I moved them to (www.)myclientsdomain.com/admin/ …and it didn’t make any difference.

I edited my cookie to kill off any gossamerthreads lines, restarted the browser, but it was even worse.

This is very similar to – or the same as – a problem that others have had.

So far… the experience hasn’t been happy. :confused:

…Bob W.


#17

That’s because the MySQL man version that is available in the download area is buggy. You will have to edit one single line in one file (I think it’s GTTemplate.pdm - not sure).

The latest version (which comes bundled in with the other applications on the site) is fantastic. They will be releasing this to the public next week.

It truly is a great product if you stick with it. I’m not sure about your connection problems. That doesn’t sound like a MySQLMan error to me. More to do with the way you’re set up, maybe?

Wil


#18

wil wrote:

[quote]That’s because the MySQL man version that is available in the download area
is buggy. You will have to edit one single line in one file (I think it’s
GTTemplate.pdm - not sure).

[/quote]

Yep-- That would be GT::Template.pm… and I had to edit two lines: the one mentioned in the support forums, and a second one assigning self->{root}; I had to put the full path to templates/ into both.

[quote]The latest version (which comes bundled in with the other applications on the
site) is fantastic. They will be releasing this to the public next
week.

[/quote]

Well, I’ll give it another shot then…

[quote]It truly is a great product if you stick with it. I’m not sure about your
connection problems. That doesn’t sound like a MySQLMan error to me.
More to do with the way you’re set up, maybe?

[/quote]

Dunno… For the database, the settings are one user, plus the subdomain I created, “database.clientsdomain.com”. Then for the user, I set database.clientsdomain.com as a host to allow access to.

I still have yet to try every combination of machinename.dreamhost.com with the custom subdomain, although that may have nothing to do with it.

What’s weird is that it takes me to the login page and says “Connection Failed” every time to every third time I click on some request. Then if I hit the Back button and try again once or twice, it’ll connect and give me the page I want.

Hmmm…

…Bob W.


#19

Hmm. No problems been reported with the MySQL server today? Mel, is it?