EPSV and PASV Errors from Dreamhost

[Jofell Gallardo]

Some of our FTP client customers have this error whenever they upload stuff
to dreamhost:

Bad PASV/EPSV response: 229

Searching the net and you mailing list gives us the reason that it might be
Dreamhost that's causing this. Will disabling EPSV remove this error (people
keep uploading the same file over and over again) or is it caused by a list
of reasons?

Please tell me if you understand my question. BTW, I just do simple FTP
downloads through curl. Nothing fancy.

[Daniel Stenberg]

Post by Jofell Gallardo
Bad PASV/EPSV response: 229
Searching the net and you mailing list gives us the reason that it might be
Dreamhost that's causing this. Will disabling EPSV remove this error (people
keep uploading the same file over and over again) or is it caused by a list
of reasons?
Please tell me if you understand my question. BTW, I just do simple FTP
downloads through curl. Nothing fancy.

I understand your question. Recent libcurl should try PASV if EPSV fails so
disabling EPSV _should_ not change anything, although of course there are
weird servers everywhere so you should probably try that out.

[Jofell Gallardo]

Hi Daniel.

I'm actually a fan of yours when it comes to support... you're like superman
for mailing lists...

Anyway... as I was saying, I gathered the following logs from his
application (with his permission of course). We already disabled EPSV so we
figured it could've solved the problem. Sadly, it didn't. Lemme show you the
bad logs (BTW, we reupload the same thing if we don't get the curlcode
CURLE_OK):

* Remembering we are in dir /music//BWFTP/766WD-WCANY1634154/583/
* FTP response timeout
* control connection looks dead
* Connection #0 to host crackerjack.dreamhost.com left intact
2007-07-16 16:45:35.204 Bandwagon FTP[16846] Error: FTP response timeout
2007-07-16 16:45:35.204 Bandwagon FTP[16846] Headers {"550 583" = "No such
file or directory"; } and Error FTP response timeout
2007-07-16 16:45:35.204 Bandwagon FTP[16846] Uploaded size: 0
* Examining connection #0 for reuse
* Re-using existing connection! (#0) with host crackerjack.dreamhost.com
* Connected to crackerjack.dreamhost.com (64.111.115.19) port 21 (#0)
* Request has same path as previous transfer

PASV

* Connect data stream passively
< 226 Transfer complete
* Bad PASV/EPSV response: 226
* Remembering we are in dir /music//BWFTP/766WD-WCANY1634154/583/766WD-
WCANY16341541184618583.mp3
* Uploaded unaligned file size (0 out of 5903550 bytes)
* Connection #0 to host crackerjack.dreamhost.com left intact
2007-07-16 16:45:36.518 Bandwagon FTP[16846] Error: Bad PASV/EPSV response:
226
2007-07-16 16:45:36.518 Bandwagon FTP[16846] Headers {} and Error Bad
PASV/EPSV response: 226
2007-07-16 16:45:36.911 Bandwagon FTP[16846] Uploaded size: 0
* Examining connection #0 for reuse
* Connection #0 seems to be dead!
* Closing connection #0
* About to connect() to crackerjack.dreamhost.com port 21 (#0)
* Trying 64.111.115.19... * connected
* Connected to crackerjack.dreamhost.com (64.111.115.19) port 21 (#0)
< 220 ProFTPD 1.3.1rc2 Server (DreamHost FTP) [64.111.115.19]

USER hifilyrics

< 331 Password required for hifilyrics

PASS ******

< 230 User hifilyrics logged in

PWD

< 257 "/" is the current directory
* Entry path is '/'

CWD /music

< 250 CWD command successful

CWD BWFTP

< 250 CWD command successful

CWD 766WD-WCANY1634154

< 250 CWD command successful

CWD 583

< 250 CWD command successful

PASV

* Connect data stream passively
< 227 Entering Passive Mode (64,111,115,19,213,24).
* Trying 64.111.115.19... * connected
* Connecting to 64.111.115.19 (64.111.115.19) port 54552

TYPE I

< 200 Type set to I

STOR 766WD-WCANY16341541184618583.mp3

< 150 Opening BINARY mode data connection for
766WD-WCANY16341541184618583.mp3

BTW, as you may see, they're using dreamhost as servers. The logs you've
seen are Cocoa logs, but they are just coded the same way we see it in your
examples. Were the logs clear enough? Should you ask anything about how we
handle curl handles, what we could say is we've done the basics, so nothing
special about coding FTP...

What could we be doing wrong? BTW, this is libcurl v7.16.1, and I dunno if
this has been resolved on the current releases, but I didn't saw anything
related to this after 7.16.1 in your changelogs (at least on the ones I
could understand).

Thanks for the fast response.

Post by Jofell Gallardo
Bad PASV/EPSV response: 229
Searching the net and you mailing list gives us the reason that it might

be

Post by Jofell Gallardo
Dreamhost that's causing this. Will disabling EPSV remove this error

(people

Post by Jofell Gallardo
keep uploading the same file over and over again) or is it caused by a

list

Post by Jofell Gallardo
of reasons?
Please tell me if you understand my question. BTW, I just do simple FTP
downloads through curl. Nothing fancy.

I understand your question. Recent libcurl should try PASV if EPSV fails so
disabling EPSV _should_ not change anything, although of course there are
weird servers everywhere so you should probably try that out.
--
Commercial curl and libcurl Technical Support: http://haxx.se/curl.html

[Dan Fandrich]

Post by Jofell Gallardo
* Remembering we are in dir /music//BWFTP/766WD-WCANY1634154/583/
* FTP response timeout
* control connection looks dead
* Connection #0 to host crackerjack.dreamhost.com left intact
2007-07-16 16:45:35.204 Bandwagon FTP[16846] Error: FTP response timeout
2007-07-16 16:45:35.204 Bandwagon FTP[16846] Headers {"550 583" = "No such file
or directory"; } and Error FTP response timeout
2007-07-16 16:45:35.204 Bandwagon FTP[16846] Uploaded size: 0

What is the sequence that led to this state? That's where the
roots of the 226 bad response lie. Can you post a complete log?

Post by Jofell Gallardo

Dan

[Jofell Gallardo]

Post by Dan Fandrich
What is the sequence that led to this state? That's where the
roots of the 226 bad response lie. Can you post a complete log?

I kinda don't understand your question. But that is the full log. It just
cycles and cycles and re-uploads the item over and over again.
It's not that I don't want to give the source code, but it's just too
stereotype... I just followed the instructions in your manual.

Do you think reusing the same handle has caused this? Sorry but I just used
hardcore libcurl for like 4 months now... I used to live with CurlHandle
before.

[Dan Fandrich]

Post by Jofell Gallardo
I kinda don't understand your question. But that is the full log. It just
cycles and cycles and re-uploads the item over and over again.

Then you're admitting it's not the full log! :^) Actually, the part you
posted started with "Remembering we are in dir..." which is definitely not
the first thing that would appear in a curl log. I'd like to see the log
starting from the beginning, up to the first time where you experience a
problem. Is the upload URL available to the public?

Post by Jofell Gallardo

Dan

[Daniel Stenberg]

Post by Jofell Gallardo
I'm actually a fan of yours when it comes to support... you're like superman
for mailing lists...

Thanks!

Post by Jofell Gallardo
* control connection looks dead
* Connection #0 to host crackerjack.dreamhost.com left intact

While I share Dan F's views on this not being the full log, I believe the
above two lines reveal a bug! When the control connection "seems dead", we
should not re-use that in a subsequent request!

I'll commit the change below, that may make your case behave differently:

--- lib/ftp.c 13 Jul 2007 20:04:53 -0000 1.419
+++ lib/ftp.c 17 Jul 2007 20:27:36 -0000
@@ -3137,7 +3137,7 @@
if(!nread && (CURLE_OPERATION_TIMEDOUT == result)) {
failf(data, "control connection looks dead");
ftpc->ctl_valid = FALSE; /* mark control connection as bad */
- return result;
+ conn->bits.close = TRUE; /* mark for closure */
}

if(result)

[Jofell Gallardo]

Post by Jofell Gallardo

Post by Jofell Gallardo
I'm actually a fan of yours when it comes to support... you're like

superman

Post by Jofell Gallardo
for mailing lists...

Thanks!

Post by Jofell Gallardo
* control connection looks dead
* Connection #0 to host crackerjack.dreamhost.com left intact

While I share Dan F's views on this not being the full log, I believe the
above two lines reveal a bug! When the control connection "seems dead", we
should not re-use that in a subsequent request!

I'm really sorry if I sounded stubborn or something. What I meant by "full
log" was that was the only
logs my customer gave me. He told me that was the start of the verbose part
of the curl (I was quite
startled he knows stuff like this). Anyway, my apologies, and next time,
I'll be giving them all.

Post by Jofell Gallardo
--- lib/ftp.c 13 Jul 2007 20:04:53 -0000 1.419
+++ lib/ftp.c 17 Jul 2007 20:27:36 -0000
@@ -3137,7 +3137,7 @@
if(!nread && (CURLE_OPERATION_TIMEDOUT == result)) {
failf(data, "control connection looks dead");
ftpc->ctl_valid = FALSE; /* mark control connection as bad */
- return result;
+ conn->bits.close = TRUE; /* mark for closure */
}
if(result)

Way too cool! Can't wait to use this patch. Umm... one question though. Was
this fix already on
versions > 7.16.1? Or is this the real deal fix (like nothing committed yet
to stable)? Nothing biggie...
just ecstatic you guys are doing great work!

[Daniel Cater]

Post by Jofell Gallardo
Way too cool! Can't wait to use this patch. Umm... one question though. Was
this fix already on
versions > 7.16.1? Or is this the real deal fix (like nothing committed yet
to stable)? Nothing biggie...
just ecstatic you guys are doing great work!

The patch was committed yesterday (see
http://curl.haxx.se/auto/cvshistory.html) which means that it will be
in the next release (7.16.5). If you would like to test the patch, it
is in today's daily snapshot (see http://cool.haxx.se/curl-daily/). If
you could get one of your customers who are experiencing this problem
to see if today's snapshot fixes the problem that would probably be
appreciated.

Note the top of that page though: "WARNING, these packages are built
daily and automatically straight from CVS. They may be broken,
crashing, working or whatever. Don't assume anything else!"

Dan.

[Jofell Gallardo]

Hi guys. The patch seems to work.

You guys are great! Thanks a bunch for the help. :)

Post by Daniel Cater
The patch was committed yesterday (see
http://curl.haxx.se/auto/cvshistory.html) which means that it will be
in the next release (7.16.5). If you would like to test the patch, it
is in today's daily snapshot (see http://cool.haxx.se/curl-daily/). If
you could get one of your customers who are experiencing this problem
to see if today's snapshot fixes the problem that would probably be
appreciated.
Note the top of that page though: "WARNING, these packages are built
daily and automatically straight from CVS. They may be broken,
crashing, working or whatever. Don't assume anything else!"
Dan.