Cron jobs for IIS

Recently I’ve had to work on Drupal and phpBB and that has meant quickly learning a lot about PHP. What a pleasant experience!  It is an amazingly productive language and it is easy to get results quickly.  On the other hand, working with Windows servers mean there are few things to deal with that are better understood in the Unix world.  One which keeps cropping up is the need to execute code on the web site at regular intervals.

This is commonly called a “cron job”, referring to the usual technique employed in Unix to get something done automatically.  There are ways to get this done within the web pages themselves.  In phpBB, as an example, one could call a function on every page and have it only perform the necessary operations when the right time has passed.  However sometimes these operations are going to be very time consuming.  The result is that the user browsing the web site who happens to be unlucky enough to trigger the operation has to wait during a page load and that isn’t a good experience.

The best option is to put your code in a specific page and have some means to schedule a “view” of that page.  Unless you want to fiddle with access control, the page needs to be written to handle unnecessary views smoothly and not to return any sensitive information in the output (should someone decide to call it from a regular browser).

My approach is fairly generic and easy to implement on Windows servers: create a Javascript file that calls a web page and use the task scheduler to execute it.

We start with a script. Open notepad and paste this into it:

var xmlDoc = new ActiveXObject("Microsoft.XmlHttp");
xmlDoc.open("GET", WScript.Arguments.Item(0), false);
xmlDoc.send();
WScript.Echo(xmlDoc.responseText);

Then save this somewhere as, say “C:\cron.js”. No go to the task scheduler and add a new task.

Create a New Task in Windows 2003

I’m using Windows 2003 in the above example but the idea is fairly similar in Windows 2008.

Windows 2003 New Task Wizard

You’ll need to select “cscript.exe” as the program to schedule and this usually lives in “c:\windows\system32\”.

Select cscript as the program to run

The next step is deciding when the schedule it. If you want to have the operation run more often than once per day then select “daily”. I’ll show you how to schedule the operation every five minutes.

Enter a descriptive name and set the schedule type

Select the time the first occurrence will start during the day. Select Every Day if you need more than one occurrence per day.

Next you’ll need to enter the credentials for the account the task will run under. This is necessary because the operation will occur even when nobody is logged in.

Enter the credentials for the user account this task will run under. An administrator account would be best.

Check “Open advanced properties…” before clicking “Finish”

Windows 2003 Task Wizard Finish

Check the "Open advanced properties..." before finishing

In the “Run:” section add two parameters. The first parameter is the full path to the “cron.js” file you created earlier. The second parameter should be the full URL to the cron page you wish to execute.

For example, you might use

cscript.exe "c:\cron.js" "http://www.example.com/cron.php"

Surround the parameters with quotation marks just in case your paths includes spaces.

Set the Run command

If you need your operation to run more often than once per day, here is how you do it. Click on the “Schedule” tab and then click “Advanced”.

Check “Repeat task” and set it to run as often as you want. Check “Duration” and set it to 24 hours.

Check Repeat Until and set the duration to 24 hours

Now you can click “OK” on this dialogue and and then “OK” on the task properties dialogue to finalise the task. You’ll probably have to re-enter your credentials.

Don’t forget to right click on the task and run it manually to make sure everything works.

phpBB Mail Gateway

I have put together a mod for phpBB3 that allows one to bring web forums and mailing lists together.  Mostly.  There are two parts: the Mail-To-Forum gateway and the Forum-To-Mail gateway.  Each forum is configured separately and there is an assumption that each forum equates to a separate mailing list.  You could configure one gateway without the other but generally you’ll want both.

Mail-To-Forum

The mod can fetch messages for the forum either by polling a drop-path for raw RFC2822 messages or by querying a POP3 mailbox.  In both cases the messages are removed after being retrieved.

Messages are parsed in the hope of converting them to basic BBCode.  If the author is very careful and tidy when it comes to quoting (contiguous lines prefixed with “>”) then the output isn’t too bad.  Signatures are cut off (a blank line followed by two dashes and a space prefix signatures — did you know that?) and line wrapping errors are fixed.

Users are matched based on the e-mail address found in the “From:” header.  Optionally new users can be automatically created with a random password or unrecognised senders can be treated as guests.

Forum-To-Mail

Outgoing messages are converted from BBCode into reasonably tidy plain-text and sent using SMTP.

Automation

The gateway operations need a scheduled task to run periodically.  You (the site operator) need to call /cron_mailgateway.php regularly.  On Windows it is just a matter of scheduling a task and on Unix you need the “at” and “curl” and other things I don’t know about.

Administration page in Forum properties

Download it here: Mail Gateway Module 0.0.3

 

Mailing Lists and Web Forums

My business has used mailing lists for many years.  In fact, one of the first features added to our product was mailing list functionality specifically so we could discuss development and provide technical support.  Many products started out that way but today very few commercial products are supported in this manner.

Why is this?  As with most things, it seems to be a bit of a generational change.  Communicating on a public mailing list actually takes a fair bit of skill.  Those who do it often tend to take pride in their messages and put a great deal of effort into them.  You may be surprised by this but there is quite a big difference between a well structured e-mail and a hard-to-read mess. Much of this is fairly subtle, such as carefully formatting paragraphs so they wrap properly, and some complicated, such as inline quoting with brackets.

The problem is that unless someone has experience they tend either not to know how to do this or simply not to care.  Today when texting, SMS and Twitter are commonplace such textual niceties seem to be going out of fashion.  On a peer support mailing list, however, problem rise up.

In a manner reminiscent of Godwin’s Law, the proportion of traffic dedicated to complaining about post etiquette increases with the number of new members. It isn’t hard to understand both sides though, not that it matters much.  If you put a lot of effort into your posts, as experienced users tend to, it can feel insulting to be mis-attributed, incompetently quoted or just plain ignored by someone who can’t work out how to access an archive.  One undoubtedly becomes snippy when polite requests not to top-post become a daily routine.

One of these posts has a lot of superfluous text, is harder to read and the answer is ambiguous. The other one is better. If you let Outlook teach you how to write in mailing lists, you'll produce messages like the one on the left. Actually I think many of you will prefer the first one because you get to read about electric shocks, but hopefully you get the point.

On the other side, however, it is easy to feel attacked when your first plea for help is answered first with what seems like a personal attack for not being “in” on the system.  This particularly hurts us when such newcomers are often potential customers.  One of their first experiences “with” our product is negative.  Has anybody done anything wrong?  Not really, but said potential customer walks away, never learns from the encounter and the next person comes along.  It doesn’t matter how much one explains all this in introductory posts because… well… people don’t read them and they are all pretty sure they know how to write an e-mail.

As a result, as much as I love mailing lists and appreciate our regulars who put so much time in to helping others, we have had to consider the other side of the community system: forums.

It is here that newcomers are made much happier and regulars are disappointed. Just about everyone knows how to use the forum and so there are few complaints about structure.  New users don’t even necessarily have to post as now they can easily search for the topic that concerns them and avoid that horrible interaction business altogether.  They’re happy.  I know this, because they say so.

We did this by starting an online forum and continuing to run the mailing list at the same time, for those who like it.  It wasn’t long before our regulars and experienced list members were disappointed.  Traffic on the list went down and very few wanted to bother with the forum.  Instead of just looking at the inbox of their favourite mail client, answering questions now involved having to periodically log into a forum.  Their involvement diminished.

This sounds like a problem.  But what actually went wrong here?  We made one group happy at the expense of another.  Did the new group have priority?  In a way, yes.  Not because it is fair or right but because this system is mostly designed to help users and those that needed help were happier.  Unfortunately those that needed help now get less of it as a result.

My solution has been to try and bring the community back together by building a module for phpBB that acts as a gateway between a mailing list and a forum.  The idea is that if you the web interface and search tools, you use it.  If you prefer to never have to log on to a web site and operate entirely in e-mail, you can do just that.  It will be interesting to see how this works out.

The Delphi Tour

Oh no.  This is my first blog post.  Don’t you just hate those? You know what I mean because, well… you’ve probably got a blog too. I think everyone has one and I feel a bit left behind. Now I won’t pretend mine will be extraordinary or such a gripping read you’ll be back five times a day hoping for more. That said, as I’m not particularly fond of reading waffle I’ll try to keep my own as limited as possible. If I still manage to waffle too much, skip over it but if I ever see a TL;DR I’ll shoot you.  Seriously.

The only question left is… where to start?

I’m at the annual Delphi “World Tour” in Montreal.  I’m not inclined to write outside the office but having had such trouble deciding how to begin it seems more than co-incidental that I suddenly find myself at an interesting event and perhaps even something noteworthy. As an aside, I’m using the new Blackberry Bold 9900 and this is probably the first Blackberry I’ve owned where posting a message online isn’t a tremendously frustrating experience. Still I’ll probably have re-written most of this back on the desktop.  “Not entirely frustrating” is far from “actually productive”.

Al Mannarino is warming up the crowd

Rather disappointingly this year there are no t-shirt handouts and I have to admit that has always been quite a big draw for me. Hey Embarcadero: the handouts make the crowd happy! Al asked audience members to raise their hands if they were developers working for companies with more than 5000 employees. A couple went up. How about 3000? Two or so. How about less than 50? All the rest of us. One gentleman sitting in front of me announced he was an ISV who worked from his closet. So come on: we really need these t-shirts.

I don’t know how surprised Al was that the majority of his audience are small ISVs and not big enterprises? I can’t believe this is news to Embarcadero as Delphi has historically been most popular in the small developer community. As much as any business really wants to target other big businesses, who are far more profitable I’m sure, I feel the rest are being forgotten a little.

Following that question about business size, Al asked what new features in Delphi XE 2 people were most in various. I think it is safe to say everybody wanted 64bit. More of a surprise? It sounded like the two discoveries didn’t mesh well. Most are small developers and most want 64bit development. He asked “What’s the point in 64 bit when everybody’s laptop only has 2gb of memory?” Now I’m editing a little (Al was really referring to comments from his seniors) but overall this is deeply frustrating to me. There is this naive presumption that small developers are writing “Angry Birds” and giant enterprises are building massive data clusters. The fact is that small Delphi developers are building all types of applications and some (like mine) are huge.

Later the topic of Interbase was brought up. “Why aren’t you using Interbase?” Someone answered “Firebird”. There was some scorn from Embarcadero there and clearly they view Firebird as an inferior product. Another audience member yelled “Licensing” and that’s the point. We can’t ship Interbase as an embedded component in the small ISV market because the licensing is unmanageable. It just has to be royalty-free or you have to develop a distribution model based entirely around the one component. Firebird is an option available to all of our customers and you can switch to and from it as you need. How do you support that with a per-seat license?

There was talk about all the improvements to Datasnap. I didn’t see that much excitement from the crowd about this and I think I know why. Datasnap is not available in the Professional Edition (and clearly that meant most of us were out of luck). This is unfortunate as Datasnap would be very useful to me but it is without a doubt the only such feature in the Enterprise edition and therefore not worth the expense. That said, as someone pointed out, it is dependent on Indy and that bothers people too. I like Indy and I like the work Chad and Remy have done, but it is an almost abandoned library and bugs keep cropping up. It has also become a paradigm rather than just a thin sockets layer and that doesn’t necessary integrate well. I can’t believe that a serious and by all accounts professional component like DataSnap, which is only available is expensive editions, relies entirely on an open-source third-party API that is famous for compatibility issues, upgrade woes and all manner of little niggles. I like and use Indy and I’ve contributed fixes myself, but if I use two components based on different versions of Indy (one might be DataSnap) I suddenly have a lot of trouble I don’t have time for.

I think I drifted off topic again.

Then we all got caught up in Mobile devices and perhaps that’s the future of Embarcadero — I don’t really have a clear vision after tonight. Many of the audience members, myself included, just want a stable development environment that is better suited to us than Visual Studio and .NET. Windows Application development doesn’t seem like Embarcadero’s priority to me. Firemonkey is all very well (what is “3-Definition” though?) but I’ve asked this question every year at this conference since Vista: where is “Windows Aero” support? It mystifies them and now it is too late; we’ve moved on. What about “Wizards”? Nothing yet. Delphi was once the leader in user interface development and yet it is still practically impossible to build an MMC snapin (ask me if you want to know how). Windows Standards are being ignored in favour of potential cross-platform gains. Is Delphi really the best cross-platform solution?

Briefly we were asked what we wanted and unfortunately there really wasn’t time for many comments. Someone asked for a “Lock Free” memory manager. That is exactly the sort of thing I’d like to see too. Unfortunately I think Al misunderstood as he pointed out AQTime is included with Delphi now. Firstly, AQTime is mostly crippled in the Professional Edition, and secondly what has that to do with the question? Sure you can develop around locking issues and perhaps a profiler can help but that is not the same thing at all. It just kept adding to the sense that we were all being misunderstood. It wasn’t just the French accents.

The last question was about “AppWave” and from my reading of the audience this was the one thing they came in unaware of and the one thing that they left most interested in. AppWave sounds like the “App Store” for iOS and perhaps this isn’t a bad idea. But again it does shoehorn Delphi ISVs in the “Gadget Vendor” category even if we are all apparently quite desperate to find successful sales outlets.

There was no more time for wishful thinking from the audience so I’ll just have to talk to myself again: Hey Embarcadero? Help us build applications. How about built-in stack traces and debugging tools for deployed applications like Windows Services? How about a powerful embedded database engine that is tightly integrated, easy to deploy and royalty free? How about tools to render and process embedded HTML forms? How about getting rid of dependencies on Indy and having instead a simple and reliable built-in socket framework?

And with that the event was over and I headed to Lucien L’Allier to catch the last train. If the trains ran more often, I’d have stayed to talk to the other developers and I now wish I had and settled for the metro as I’d like to know what other people thought. Particularly he who works from the closet. Did he also need a new t-shirt?