Confirmation email problem, & greetings

[choralia]

You're welcome . I hope your ISP has been able to fix this issue.

Max

[vaarky]

Do we know what the cause of the problem was?

[MandyShaw]

Still waiting for my ISP to diagnose the problem, but I don't think it was at the CPDL end, as i have a similar problem with a couple of other websites. I'm therefore very grateful for Max's help.

[vaarky]

If you figure out what the problem was even on the receiving end, I'd be curious to know. For example, if it's being flagged by spam filtering technology, there may be steps we can take to adjust the keywords or formatting of the message to prevent other individuals from potentially being similarly affected. If you are able to find out anything from your ISP, it could be helpful. Thanks!

[vaarky]

P.S. Most useful would be if the ISP can identify what software flagged it (e.g SpamAssassin, Barracuda, Frongbridge, StrongMail etc. or a home-brewed check) and indicate the type of issue (e.g. low score due to missing DKIM record, or SpamAssassin score too high (and info about the criteria it got dinged on).

[MandyShaw]

Absolutely, I'll find out as much as I can & post it here. As we have a public holiday today, I suspect nothing will happen until at least tomorrow though.
Mandy

[MandyShaw]

My ISP, Zen, have now found something:

@cpdl.org
------------------------------------------------------
2009-05-07 09:35:32 H=bastion04.mail.zen.co.uk [212.23.8.64] F=<cpdl@gator575.hostgator.com> rejected RCPT <mandy.shaw@iperimeter.co.uk>: Sender verify failed

They say they reject such emails as an anti-spam measure. I have pointed out that verifying email addresses in this way is an absurd thing for them to be doing (or words to that effect) and asked them to switch this filter off asap for my email address. I await developments.

In the meantime, I am posting this here for the benefit of others with equally officious ISPs.

Mandy

[vaarky]

I notice it says Sender Verify Failed. Is that because our domain does not list a DKIM or SPF record?

[MandyShaw]

I think it is simply that the 'noreply@cpdl.org' email address doesn't match anything in the website SMTP engine's directory (why should it, really). I'm not sufficiently expert on SMTP to understand anything further. If you have detailed questions we could ask the ISP, I'm entirely happy to pass them on.

[choralia]

Mandy, there are couple of strange things about the way your ISP handles this kind of emails:

1) for antispamming reasons the ISP performs a check about the SMTP server sending the email and SMTP server that is supposed to receive emails for the same address;

2) when the above check fails, the email is not delivered at all to the intended recipient, rather than being delivered but tagged as spam.

Item 2 is completely crazy, so I wouldn't comment on it.

In the case of CPDL, check 1) probably fails because we have 3 SMTP servers involved:

a) the SMTP server sending the confirmation email, that is not a CPDL-specific server. By default, the wiki makes use of the php sendmail function, that is based on the "generic" SMTP server of the hosting provider;

b) the main CPDL-specific SMTP server, that runs on a physically different server, for disaster recovery reasons;

c) the back-up CPDL-specific SMTP server, that runs on the same server as the wiki, but it's logically a different server than the SMTP server made available by the hosting provider.

I've now (temporarily) moved the main CPDL-specific SMTP server on the same server running the wiki, and switched off the back-up SMTP server. This may fool your ISP's check. Could you please try again, and see whether this solves the problem with your ISP?

Max

[MandyShaw]

Tried, but nothing arrived ... thanks anyway. To be honest I am going to change my registered email address on CPDL and the other websites where I have the same problem - my hotmail account works fine for these purposes, and (despite the wild imaginings of my ISP's tech support team) I hardly think American Express (for example) are going to reconfigure their SMTP server to meet the requirements of an obscure UK ISP. I would change ISP a) if I didn't have my broadband and 2 domains with them, and b) if, to be fair, this was other than a single blot on an excellent customer service record since 2002. I am extraordinarily grateful to you all for your help with this.

[vaarky]

Other users are very likely seeing this problem. Are there changes we can make to CPDL's configuration to help?

[choralia]

I think that the wiki can be configured to use the CPDL-specific SMTP server rather than the generic SMTP server of the hosting provider. This may remove the mismatch when the SMTP server is checked, and also improve traceability of outgoing email. I'll look into it.

Max

[MandyShaw]

Thanks Max. I am still talking to my ISP about this, having formally complained about it (one of their managers will be ringing me on Monday, I hope), so if there are questions you need asking, please let me know.
All the best
Mandy

[MandyShaw]

I quote...

'... I am glad to advise that our Hosting Team have actually changed the policy on the cPanel server and have switched off the 'Sender Verify' for all customers. The original decision for this to be implemented was as a way to combat spam. As I'm sure you have been made aware the way this works is that the email gets sent to [ISP], and our servers would check the email address that the message claimed to be from. If the address existed, then the mail would be allowed through. If the address did not exist, the message would be black-holed and no bounce back message would be sent etc.

'What they have done now, is reduced the level of the sender check so that our servers will simply check to see if the domain that the email is from has a corresponding MX (mail) record set up. If it does, the message will now be delivered. This means that legitimate emails ... should now be allowed through.

'Please accept my apologies that this could not be actioned straight away for you following your emails on Friday - the issue was actually escalated to the senior developers for our cPanel who looked into this and came to the conclusion that the Sender Verify was causing extra load on our servers, and stripping it back would actually make the service more efficient.'

I'm amazed and delighted.