Problem with SMTP...

[bluesin]

Hi I'm having an intermittent problem with one of my accounts mail smtp server.

I've restarted the mta agent and the mda agent and even rebooted the server but this one account cannot use the smtp server to send mail.

Don't see any problems in any of the logs, tried stopping the firewall, all to no avail?

Where do I need to look next?

[R-n-R]

You might want to go as far as doing a complete deletion of the site off the server and reinstalling it.

[IWorx-Socheat]

bluesin,

Can you give us a few more details about the problem. So, you can receive email on this domain, but you can't send email from the domain?

Have you tried sending mail from the webmail client, or a different desktop email client? If you can send from webmail, that would mean something is wrong with your email client, or something is wrong with the communication between the server and your email client.

Socheat

[bluesin]

Yes, mail can be sent from webmail, my client first reported it and he is using outlook. When he first reported it I set up a test email account and tried it using Opera and had no problems, making me suspicious of things on his end. He reported earlier this week that the problem had cleard up, but then reported again this morning that the problem had resurfaced. I tried it again using the test email and am having the problem on my end with just this particular domain. All other domains work fine. The mail server is pingable (mail.weststarchurch.org) receiving mail works just fine also.

I have looked at the mail logs and don't see anything in the logs I've looked at.

As a side note, I deleted the test user from the domain and tried sending an email using the id expecting to get an immediate authentication failure, but it took nearly 15 minutes for the server to respond with the authentication prompt...

That is making me think somewhere there is a problem with authentication, but that's just a guess...

[IWorx-Chris]

Is your client's ISP blocking outbound port 25 connections bluesin? Can he telnet to port 25 on the box and see the standard smtp greeting?

Chris

[bluesin]

Not sure Chris, but as I'm having the same problem with the test email, I'm fairly certain its not on his end at this time.

As I said, this is the only domain out of the 30 or so that is having a problem, I tried telnet to several of them with "telnet mail.domainname.com 25" for several of them and I don't get connected to any of them via that method, however as I said, all of them work except this one when sending mail...

You want me to open a ticket with all the info?

[jimp]

I am going to jump in on this one. For approximately two months now, we have been having the same problem. POP3 and IMAP work perfectly, but SMTP connections may or may not work. I thought it was a problem with Mozilla Thunderbird, because everyone in our office uses it and no one else was complaining, but an Apple Mail user reported the problem last Friday. :(

The problem is difficult to reproduce, but it is effortless to cause. You simply...

Compose an email
Click "Send"
1 out of ~15 emails will block with an error message (attached to this post)
Trying again (immediately) will succeed without a problem

My personal testing found that SMTP w/o TLS almost never produces the error.

More evidence

We have written a photography sales system (Mozart Photo System (http://mozartcart.info)) that sends emails to customers who have requested a reminder before photos are taken offline. I noticed that (rough guess) 10-20% of a store owner's nightly emails would fail, although the script (called by cron) is only trying to connect to the local SMTP server. (I think that rules out the possibility of faulty data center connection.)

While trying to make Mozart's emailer more resilient, I found some interesting properties surrounding the problem. Note that the problem can be reproduced by "hammering" the server, but I'm not talking about "attacking" the server. I just mean a script that needs to send 10+ emails in a row is very likely to surface the bug.

My test setup was trying to send an email 10 times to the local server and local email address. When the script failed to send an email, the script will enter to a recovery loop that would try 2 more times. Here is what I discovered:

Trying 3 times would fail all 3 times.
Trying 3 times, and sleeping 10s between each attempt, would fail all 3 times.
Trying 3 times, but first establishing a new connection, recovered everytime on the 2nd attempt. NO sleep was required.


So Mozart's emailer is now working fine, but the mail server's strange behavior is very similar to Mozilla Thunderbird's. Thunderbird would fail once, but try again (make a fresh connection), and the mail would deliver just fine.

I am fairly sure this problem is very recent (within 3 months). Would an upgrade or downgrade of the mail server, and/or its dependencies be a good idea? It is possible the mail server's memory limits are being hit?

[R-n-R]

It wasnt until jimp's message I realized this VERY thing has happened to me a few times. On the first attempt on sending to try to send (using Outlook Express) , I would get some kind of error, and then I retry again right after, and the mail will go through.

It only happens once or twice a month and I just assumed some internet glitch, BUT since I see that others are having the same problem, I have to say HUMMM

[IWorx-Socheat]

Jimp,

Is that a screenshot from Thunderbird, Outlook Express, or some other mail client? It may be useful to have a screenshot of each (one from Thunderbird, one from Outlook Express).

Regardless, to those who are affected, we are looking into the problem. I believe Bluesin has a ticket open right now, and we're working on diagnosing and consistently reproducing the problem.

Socheat

[jimp]

The screen shot is from the latest Thunderbird.

While I was fixing the Mozart built-in emailer to be more fault tolerant, these are the error messages I could produce consistently. I think the two notices followed the "Failed to set sender" warning everytime.
Warning: Failed to connect to mail.basicmatrix.com:25 [SMTP: Invalid response code received from server (code: 421, response: unable to read controls (#4.3.0))] in ${PATH}/${FILE} on line 162

Warning: Failed to set sender: ${EMAIL_ADDR} [SMTP: Invalid response code received from server (code: -1, response: )] in ${PATH}/${FILE} on line 170

Notice: fwrite(): send of 37 bytes failed with errno=32 Broken pipe in /usr/lib/php/Net/Socket.php on line 294

Notice: fwrite(): send of 6 bytes failed with errno=32 Broken pipe in /usr/lib/php/Net/Socket.php on line 294 Also note that Mozart's emailer uses PEAR_Mail to actually handle the delivery.

[R-n-R]

In my case I was using Outlook Express. The next time I have the problem I will post a screen shoot here of the error. However I have not had the problem in a while.

[R-n-R]

OK, the SMTP error happened again (good timing). Right after this error message I sent the email again just a few seconds later and it sent successfully.

[bluesin]

Ok, this appears to be a problem with bellsouth blocking port 25 for new domains.

http://home.bellsouth.net/csbellsouth/s/editorial.dll?fromspage=all/home.htm&categoryid=&bfromind=62&eeid=3784168&eetype=article&render=y&ck=

There is some other notes on it about migration and such, but I suspect that they are allowing older domains to transfer through port 25 and blocking newer ones.

I set up a new account this week and I cannot use the smtp server on it either.

Many ISP's are doing this, i've seen references to setting up alternative ports, can someone provide instructions for doing so?

[IWorx-Chris]

bluesin,

We'll have a point release out this week that will allow you to easily setup the alternate port (defaulting to 587). I'd recommend waiting for it but if you need it ASAP we can get you a pre-release version of the software.

Chris

[bluesin]

Thanks Chris, no problem on the wait...

[bluesin]

Oh and Chris, do you know if it will be set up to listen on both ports or does it have to be just the alternate?

[IWorx-Chris]

It has to work on 25 bluesin or else smtp itself will break :). So the 587 is just an alternate in addition to 25.

Chris

[bluesin]

That's what I thought, thanks Dude!

[jimp]

Has anyone made any progress on this one?

[IWorx-Tim]

Has anyone made any progress on this one?

The feature is now available inthe 2.1.2 version of InterWorx.

To enalbe it


Log into NodeWOrx
Click on System Services
Click on Mail Server
Ckick on MTA Settings
Under Alternate Inbound SMTP Server, ckick Start The default port is 587, but you can change it if you want.

[R-n-R]

OK I am trying to use Port 587 with Outlook Express, I did the following
First I did what IWorx-Tim said to do above in his post, port 587 on the server is showing to be active, running.

Then I opened up Outlook Express Accounts > selected properties then on the advanced Tab change Outgoing SMTP to port 587 clicked on Apply and then OK and tried to send email and got this error:

The connection to the server has failed. Account: 'XXXXXXX', Server: 'mail.XXXXXXXX.com', Protocol: SMTP, Port: 587, Secure(SSL): No, Socket Error: 10060, Error Number: 0x800CCC0E

IS there something else I need to do?

[IWorx-Chris]

Make sure if you're using a firewall and it's on that port 587 / tcp is open.

Chris

[R-n-R]

Ah, yes thanks Chris

[bluesin]

Chris, I added this port to the firewall, but for some reason the service is listed as "submission", is there anyway to set the service name to smtp?

[IWorx-Paul]

"Submission" is actually technically the correct term for the 587 smtp port service. It's meant for "submission" of e-mail from the end user, as opposed to communication between mail servers, which is what port 25 is for (in theory).

See: http://www.google.com/search?hl=en&lr=&safe=off&c2coff=1&q=smtp+submission+587

If you really wanted to change the name, you could probably edit the /etc/services file. Just be careful :)


Paul

[bluesin]

Thanks, i'm actually going to use 1025 for it, I also have some other ports open for some other servers that I'd like to be able to have a service name associated with them in the firewall...

[IWorx-Tim]

Make sure if you're using a firewall and it's on that port 587 / tcp is open.

Chris

Whoops, forgot to mention that . . . my bad :-)

[jimp]

The feature is now available inthe 2.1.2 version of InterWorx.

To enalbe it
Log into NodeWOrx
Click on System Services
Click on Mail Server
Ckick on MTA Settings
Under Alternate Inbound SMTP Server, ckick StartThe default port is 587, but you can change it if you want. Sorry, I was actually wondering if anyone has made any progress on the original purpose of this thread (SMTP connection issues). :) See post #7 (http://www.interworx.com/forums/showpost.php?p=6493&postcount=7).

[R-n-R]

jimp I was having the same trouble from time to time. And I activated port 587 in the latest version of Interworx (2.1.2). I then set up my email manager to use SMTP PORT 587 and its been working perfect ever since. With the exception of a small problem at first with this, but it was easly fixed by opening PORT 587 in my servers firewall. Other than that works fine, solved my problem,....so far.

[IWorx-Paul]

I am going to jump in on this one. For approximately two months now, we have been having the same problem. POP3 and IMAP work perfectly, but SMTP connections may or may not work. I thought it was a problem with Mozilla Thunderbird, because everyone in our office uses it and no one else was complaining, but an Apple Mail user reported the problem last Friday. :(

The problem is difficult to reproduce, but it is effortless to cause. You simply...

Compose an email
Click "Send"
1 out of ~15 emails will block with an error message (attached to this post)
Trying again (immediately) will succeed without a problem

My personal testing found that SMTP w/o TLS almost never produces the error.

More evidence

We have written a photography sales system (Mozart Photo System (http://mozartcart.info)) that sends emails to customers who have requested a reminder before photos are taken offline. I noticed that (rough guess) 10-20% of a store owner's nightly emails would fail, although the script (called by cron) is only trying to connect to the local SMTP server. (I think that rules out the possibility of faulty data center connection.)

While trying to make Mozart's emailer more resilient, I found some interesting properties surrounding the problem. Note that the problem can be reproduced by "hammering" the server, but I'm not talking about "attacking" the server. I just mean a script that needs to send 10+ emails in a row is very likely to surface the bug.

My test setup was trying to send an email 10 times to the local server and local email address. When the script failed to send an email, the script will enter to a recovery loop that would try 2 more times. Here is what I discovered:

Trying 3 times would fail all 3 times.
Trying 3 times, and sleeping 10s between each attempt, would fail all 3 times.
Trying 3 times, but first establishing a new connection, recovered everytime on the 2nd attempt. NO sleep was required.


So Mozart's emailer is now working fine, but the mail server's strange behavior is very similar to Mozilla Thunderbird's. Thunderbird would fail once, but try again (make a fresh connection), and the mail would deliver just fine.

I am fairly sure this problem is very recent (within 3 months). Would an upgrade or downgrade of the mail server, and/or its dependencies be a good idea? It is possible the mail server's memory limits are being hit?

A lot of different topics ended up being discussed in this thread, so I quoted jimp's post to make it clear what I'm responding to here - and that is, the "intermittent smtp connection failing" problem.

After quite a bit of investigation, it looks like jimp's final suggestion is the problem. The smtp server's memory limit appears to be getting it "sometimes". It isn't exactly clear what's causes it to happen, the memory limit was indeed getting hit occasionally, causing smtp connections to fail.

For anyone having this problem, specificially anyone that ever received the message
"421 unable to read controls (#4.3.0)"
when trying to send an e-mail message, here's what you can do to fix it.

1) As root on your server, edit the /service/smtp/run
2) change the line

exec /usr/bin/softlimit -m 6400000 \
to
exec /usr/bin/softlimit -m 12800000 \

effectively doubling the memory limit
3) restart the smtp server: svc -t /service/smtp

Paul

[jimp]

After quite a bit of investigation, it looks like jimp's final suggestion is the problem. The smtp server's memory limit appears to be getting it "sometimes". It isn't exactly clear what's causes it to happen, the memory limit was indeed getting hit occasionally, causing smtp connections to fail.I am very curious to hear what you did to pinpoint the solution. :) My entire office, and one annoyed customer who eventually started using their ISP's SMTP server instead, will all be very happy to hear the good news.

A while back, I also had to increase our IMAP service's memory limit from 6400000 to 10200000, to correct a similar annoying error message about the IMAP connection being reset. I think a member of the InterWorx team helped me with that memory limit problem, too. No one in our office uses POP3. Could it be limiting out, like IMAP and SMTP were? (It is capped at the default 6400000.) Many customers just accept error messages without ever telling us, so I admit it could be happening and we would never know it.

[Zeth]

I've been getting the same error once in a while with my server. What I dont understand is why it is now using more memory than it was using 2 months ago, since that is about the time the error started.