I have put together a mod for phpBB3 that allows one to bring web forums and mailing lists together. Mostly. There are two parts: the Mail-To-Forum gateway and the Forum-To-Mail gateway. Each forum is configured separately and there is an assumption that each forum equates to a separate mailing list. You could configure one gateway without the other but generally you’ll want both.
Mail-To-Forum
The mod can fetch messages for the forum either by polling a drop-path for raw RFC2822 messages or by querying a POP3 mailbox. In both cases the messages are removed after being retrieved.
Messages are parsed in the hope of converting them to basic BBCode. If the author is very careful and tidy when it comes to quoting (contiguous lines prefixed with “>”) then the output isn’t too bad. Signatures are cut off (a blank line followed by two dashes and a space prefix signatures — did you know that?) and line wrapping errors are fixed.
Users are matched based on the e-mail address found in the “From:” header. Optionally new users can be automatically created with a random password or unrecognised senders can be treated as guests.
Forum-To-Mail
Outgoing messages are converted from BBCode into reasonably tidy plain-text and sent using SMTP.
Automation
The gateway operations need a scheduled task to run periodically. You (the site operator) need to call /cron_mailgateway.php regularly. On Windows it is just a matter of scheduling a task and on Unix you need the “at” and “curl” and other things I don’t know about.
Administration page in Forum properties
Download it here: Mail Gateway Module 0.0.3
thanks Elric to that,
in your mind, is there a way to add an htlm2text function for mail with Content-Type: text/html ?
I could probably include one of the open source classes (if licensing permits) to do that but I’d rather not complicate it unnecessarily. There are very few mail clients that don’t include a plain-text version. Do you have a mailing list where this is a problem or do you have something specific in mind?
I would like to create automatically a post from a mailbox. This mailbox receives several newsletters and some are in html.
maybe you could use classe html2text from Roundcube Project wich is under GPL Licence?
http://trac.roundcube.net/browser/trunk/roundcubemail/program/lib/html2text.php
I’ve updated the module with support for HTML. Sorry it took so long.
Aaahhh… the search is over! (I hope.) This is a mod I desperately need but realising it is still in v. 0.0.3 I have not (yet) been brave/foolish enough to test it on a production phpbb3 installation…
Two questions before I take the leap:
1) What happens if someone e-mails or posts an attachment? (Will your mod include or truncate the attachment during the conversion, or will it crash the system and cause havoc?)
2) What happens if “Create Users for Unrecognised Accounts” is disabled and Guest account has no write access? Is the mail left untouched or is it deleted? (I’m trying to avoid spam buildup on the forum and in the inbox.)
Thanks for developing this mod!
1) When someone posts a message with attachments the attachments are discarded. It would be nice to turn them into post attachments but my own requirements haven’t included that yet.
2) I don’t know what would happen if the guest account has no write access but I think the post will still take place.
I quite understand not wanting to try it on a live system but you could create a private test forum not visible to the public and test it there.
Good luck and thank for your support!
Thank you for making this! I’ve been looking for something like this for ages!
Is there anyway to get this to work with SSL POP3? I’m trying to pop the mail from a Google POP3 account (the domain I’m working with has Google Hosted Apps).
I’ve looked in the pop3_collect.php and noticed you used raw sockets instead of the PHP IMAP functions which support SSL POP3. Is there a possibility of porting this function to the built-in IMAP functions, or could you describe exactly what information should be returned by that function so I could port it myself?
Thanks for making this! I’ve been looking for something like this for ages!
Is there a way to get it to work with SSL POP3? I have to use a Google account as the mail receiver, but Google forces SSL.
Is there any chance of porting the function in pop3_collect.php to the built-in PHP IMAP functions? They support POP3 and SSL. If that’s not an option, could you explain exactly what output the function should give so I could take a stab at porting it myself?
Damn it! Sorry for the double post. The site said the first one was a duplicate and didn’t show it to me until I posted a second one… Feel free to delete one of them, since they ask the same thing (and feel free to delete this post too)
Thanks again for such a great mod!
Ok, nevermind. I got it working by adding ssl:// to the front of the host name in pop3_collect.php. You can delete all my messages. Again, thanks for the great mod. It seems to be working!
I am getting my emails from Gmail, who uses only SSL.
Thanks!
Elric, I just stumbled across your efforts here and am excited to give it a try. I was wondering two things.
First, can you or anyone else who may be using this share with me a url so I can see what the end result looks like?
Second, my goal is to sync a forum from a yahoo group, minimally one way/archival. I have used a program called yahoo2mbox that has slowly collected all of the history into an mbox file. With your list experience, can you give me some creative ideas on how I could get this data into a fresh phpbb install before I enable the mail gateway to continue the syncing from then on? I too am running IIS environment and am limited on what I might do to achieve this.
I wonder if the mail gateway could be easily modified to not pop, but instead check an mbox file for changes to process into a forum. Seems like a good solution to the yahoo groups dilemma: Mbox2Phpbb.
For those looking at this, I have figured out the following:
- It does try to “thread” messages based on topic title which is really great when it works. I wish it would also try using (In-Reply-To:).
- You need to enable extension=php_mbstring.dll in php.ini.
- For emails out, it uses php mail function without all of the phpbb stuff, so you need to make sure your php.ini has mail configured correctly.
- It absolutely requires plaintext format messages (Content-Type: text/plain; charset=”ISO-8859-1″), it chokes heavily on messages that have been scraped using Yahoo2Mbox or any other scraping tool when they are (Content-Type: multipart/alternative; boundary=”0-58645656-1287429990=:31883″). This makes it a pain to import an archive.
- It will not pull date from the message and uses the phpbb post date. Another pain when importing from an archive.
- Pop works
- Maildir import works and is a good option when you want speed and also when using a single yahoo account for multiple groups. You can use a pop tool to sort mail based on subject or other header info.
Things I still need to figure out:
- How to import mbox or maildir archive when it has multipart messages sprinkled in. I am wondering if there is a tool that will convert messages to plaintext and thereby fix the mess that would otherwise be left behind if you simply searched and replaced the Content-Type.
- Need to find a pop client/mail agent that can sort mail to folders that will run on Windows, php, perl… Or then resolve to use a different email for each group/froum.
- Come up with some code to clean up emails to remove any references to “groups.yahoo.com” and so forth.
- Come up with code to also remove the very common “[Non-text portions of this message have been removed] ”
- Resolve issue where maildir files are not being deleted. Prolly a server rights thing I havent figured out yet.
Success! I did all of the middleware in perl while learning a lot along the way. There are in-fact various ways to tie a Yahoo Group to your PHPBB3 forum. Absolutely abandon the old and nonexistent mail2forum. Dont give it another thought. I chose using the grabyahoogroup.pl which supersedes yahoo2maildir.pl. The scraper bypasses the need for email as well as multiple email accounts to separate the various groups. I modified it to save a duplicate of every new file into an IMPORT folder to work from, so I still always have the original messages. Then you can clean up the mail files with your own perl and regex magic and then run this importer against it. I will be throwing this up on a few forums immediately and probably any BB3 forum I do in the future. It can easily be a convenience to your niche in one-way or even two-way mode.
February 12, 2013.