Discussion:
multithreading problem with libcurl
Konstantin Schauwecker
2003-06-01 09:41:22 UTC
Permalink
Hello everybody

I have some strange problems when using libcurl in multiple threads. I've
spend days of debugging and modifying my source, and now I'm completely
helpless :-(
I have an application which uses 10 threads (I had the problem with lower
counts of threads too). Each thread calls a curl_easy_init to get a curl
handle. All threads are doing constantly HTTP requests.

Now my Problems:

On Linux:
The program runs fine for a few minutes, but then it crashes with a
segmentation fault. Valgrind told me, that the reason for crashing is inside
of libcurl. Here the output of valgrind:

==1487== valgrind's libpthread.so: KLUDGED call to: siglongjmp (cleanup
handlers are ignored)
==1487== Invalid read of size 4
==1487== at 0x40235303: Curl_resolv (hostip.c:242)
==1487== Address 0x41B0C3B0 is on thread 1's stack
==1487==
==1487== Invalid read of size 4
==1487== at 0x4023B01B: Curl_failf (sendf.c:154)
==1487== Address 0x10 is not stack'd, malloc'd or free'd

I thougth this might be a problem of setting the wrong options. So I tried to
remove the calls of curl_easy_setopt to a minimum (I already tried reducing
all code to a minimum *g*), but it didn't help.

On Windows:
On Windows the program doesn't crash (I even used a memory debugger too), but
it shows some other strange behaviour: First everything works fine too, but
then the CPU load jumps to 100% and won't come down until I terminate all
threads. As far as I could figure out, the calls (or at least one call) of
curl_easy_perform must produce the high CPU load. But none of them hangs up
(just getting extremely slow).

All these problems are gone, if I reduce the number of threads to 1. I hope
somebody of you has an idea what I could do.

Konstantin Schauwecker


-------------------------------------------------------
This SF.net email is sponsored by: eBay
Get office equipment for less on eBay!
http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5
Jean-Philippe Barrette-LaPierre
2003-06-01 17:40:16 UTC
Permalink
Post by Konstantin Schauwecker
Hello everybody
I have some strange problems when using libcurl in multiple threads. I've
spend days of debugging and modifying my source, and now I'm completely
helpless :-(
I have an application which uses 10 threads (I had the problem with lower
counts of threads too). Each thread calls a curl_easy_init to get a curl
handle. All threads are doing constantly HTTP requests.
The program runs fine for a few minutes, but then it crashes with a
segmentation fault. Valgrind told me, that the reason for crashing is
which version of curl are you using?
Post by Konstantin Schauwecker
==1487== valgrind's libpthread.so: KLUDGED call to: siglongjmp (cleanup
handlers are ignored)
==1487== Invalid read of size 4
==1487== at 0x40235303: Curl_resolv (hostip.c:242)
==1487== Address 0x41B0C3B0 is on thread 1's stack
==1487==
==1487== Invalid read of size 4
==1487== at 0x4023B01B: Curl_failf (sendf.c:154)
==1487== Address 0x10 is not stack'd, malloc'd or free'd
I thougth this might be a problem of setting the wrong options. So I tried
to remove the calls of curl_easy_setopt to a minimum (I already tried
reducing all code to a minimum *g*), but it didn't help.
On Windows the program doesn't crash (I even used a memory debugger too),
but it shows some other strange behaviour: First everything works fine too,
but then the CPU load jumps to 100% and won't come down until I terminate
all threads. As far as I could figure out, the calls (or at least one call)
of curl_easy_perform must produce the high CPU load. But none of them hangs
up (just getting extremely slow).
All these problems are gone, if I reduce the number of threads to 1. I hope
somebody of you has an idea what I could do.
Konstantin Schauwecker
-------------------------------------------------------
This SF.net email is sponsored by: eBay
Get office equipment for less on eBay!
http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5
-------------------------------------------------------
This SF.net email is sponsored by: eBay
Get office equipment for less on eBay!
http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5
Konstantin Schauwecker
2003-06-01 18:20:44 UTC
Permalink
Post by Jean-Philippe Barrette-LaPierre
which version of curl are you using?
I am using Version 7.10.5 on both systems.


-------------------------------------------------------
This SF.net email is sponsored by: eBay
Get office equipment for less on eBay!
http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5
Bo Lorentsen
2003-06-02 06:38:02 UTC
Permalink
Post by Konstantin Schauwecker
I have an application which uses 10 threads (I had the problem with lower
counts of threads too). Each thread calls a curl_easy_init to get a curl
handle. All threads are doing constantly HTTP requests.
I have a simular setup (only 5 threads), but not the problems you
descripe here, not that it will help you :-)
Post by Konstantin Schauwecker
The program runs fine for a few minutes, but then it crashes with a
segmentation fault. Valgrind told me, that the reason for crashing is inside
One silly question, do you handle timeouts ?, and have you then disabled
the signal handler (CURLOPT_NOSIGNAL) ?

Would it be possible for you to post both some sample code, and a small
Makefile that generated the crash ?

/BL



-------------------------------------------------------
This SF.net email is sponsored by: eBay
Get office equipment for less on eBay!
http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5
Konstantin Schauwecker
2003-06-03 09:30:25 UTC
Permalink
Post by Bo Lorentsen
One silly question, do you handle timeouts ?, and have you then disabled
the signal handler (CURLOPT_NOSIGNAL) ?
Oh yes, that solved the Linux problem :-)
Many thanks. Now I have the same high CPU load problem like on Windows. Maybe
a problem with my code. At least it seems to work, if I reduce the thread
priority of each thread.

Konstantin Schauwecker


-------------------------------------------------------
This SF.net email is sponsored by: eBay
Get office equipment for less on eBay!
http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5
Bo Lorentsen
2003-06-03 09:43:45 UTC
Permalink
Post by Konstantin Schauwecker
Oh yes, that solved the Linux problem :-)
Nice to be of any help.

/BL



-------------------------------------------------------
This SF.net email is sponsored by: eBay
Get office equipment for less on eBay!
http://adfarm.mediaplex.com/ad/ck/711-11697-6916-5

Loading...