Threaded resolver still blocked in Curl_resolver_cancel?

[Mac Wang]

Hi,

I am working on Android and have successfully built libcurl 7.25.0 with
threaded resolver support. I learnt through reading the source code that it
seems to only work with the multi interface, and switched to that.
In my test, I intentionally set DNS server to some stranger IP so DNS would
never work. I actually got timeout error from curl_multi_info_read within
the given time limit, but the call to curl_multi_remove_handle hung for a
few minutes before it came back.
It seems 'remove handle' actually calls Curl_done -> Curl_resolver_cancel,
and finally trying to 'join' that resolver thread.
Without curl_multi_remove_handle, I wouldn't be able to reuse that easy
handle. May I ask how threaded resolver is intended to be used?

Thanks!

My code looks like this:

initialization code:
curl_easy_init <http://curl.haxx.se/libcurl/c/curl_easy_init.html>
curl_multi_init <http://curl.haxx.se/libcurl/c/curl_multi_init.html>


begin actual transfer (this is called multiple times for different purposes)
curl_easy_reset
curl_easy_setopt <http://curl.haxx.se/libcurl/c/curl_easy_setopt.html>
curl_multi_add_handle<http://curl.haxx.se/libcurl/c/curl_multi_add_handle.html>

loop:
curl_multi_perform <http://curl.haxx.se/libcurl/c/curl_multi_perform.html>
curl_multi_fdset <http://curl.haxx.se/libcurl/c/curl_multi_fdset.html>
select
end loop

curl_multi_info_read<http://curl.haxx.se/libcurl/c/curl_multi_info_read.html>
curl_multi_remove_handle<http://curl.haxx.se/libcurl/c/curl_multi_remove_handle.html>
end actual transfer

finalization code
curl_easy_cleanup <http://curl.haxx.se/libcurl/c/curl_easy_cleanup.html>
curl_multi_cleanup <http://curl.haxx.se/libcurl/c/curl_multi_cleanup.html>

[Daniel Stenberg]

Post by Mac Wang
I am working on Android and have successfully built libcurl 7.25.0 with
threaded resolver support. I learnt through reading the source code that it
seems to only work with the multi interface

That is not true. I would expect us to have gotten (many more) bug reports if
it just didn't work with the easy interface.

Post by Mac Wang
In my test, I intentionally set DNS server to some stranger IP so DNS would
never work. I actually got timeout error from curl_multi_info_read within
the given time limit, but the call to curl_multi_remove_handle hung for a
few minutes before it came back. It seems 'remove handle' actually calls
Curl_done -> Curl_resolver_cancel, and finally trying to 'join' that
resolver thread. Without curl_multi_remove_handle, I wouldn't be able to
reuse that easy handle. May I ask how threaded resolver is intended to be
used?

The resolver is not directly used at all by an application. The application
uses libcurl's API as documented and libcurl will do the proper name resolves
in the backend. The API is independent of what resolver backend that is used.

Now, is curl_multi_remove_handle() can hang due to this, it is either a bug
that can be fixed, or it is due to a more fundamental problem in that
something needs to wait for the thread to die and the name resolve thread
cannot easily be terminated without risking to lose resources.

I haven't really investigated this, but please do and we can get a closer look
on if we can improve this!

[Mac Wang]

Post by Daniel Stenberg
That is not true. I would expect us to have gotten (many more) bug reports if
it just didn't work with the easy interface.

Inside this function, Curl_resolver_is_resolved, after we determined
the resolving hasn't finished, and calculated the timeout value, we
call

if(done) {
}
else {
...
Curl_expire(conn->data, td->poll_interval);
}

There is this check in the beginning of Curl_expire:

if(!multi)
return;

So I guess Curl_expire doesn't do anything if not used in multi interface (?)

Post by Daniel Stenberg
The resolver is not directly used at all by an application. The application
uses libcurl's API as documented and libcurl will do the proper name resolves
in the backend. The API is independent of what resolver backend that is used.

This is exactly what I expected, but it turns out
curl_multi_remove_handle() calls

Curl_done(&easy->easy_conn, easy->result, premature);

which, in turn, calls

Curl_resolver_cancel(conn);

and then

destroy_async_data(&conn->async);

and finally blocks at here:

Curl_thread_join(&td->thread_hnd);

Post by Daniel Stenberg
Now, is curl_multi_remove_handle() can hang due to this, it is either a bug
that can be fixed, or it is due to a more fundamental problem in that
something needs to wait for the thread to die and the name resolve thread
cannot easily be terminated without risking to lose resources.

I guess it's totally understandable that a 'join' would block here,
for the whole need for a threaded resolver is because the system
resolver blocks. If we can have only one name resolver thread for each
connection (is that so?), we're bound to wait for that thread to
finish before we can use name resolver again. If we can have multiple
threads and 'detach' them, we can potentially have multiple (blocked)
resolver threads running while the main system continues.

Regards,
Mac

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Et

[Daniel Stenberg]

Post by Mac Wang
So I guess Curl_expire doesn't do anything if not used in multi interface (?)

That is correct. But that expire time is not used for anything with the easy
interface so it follows very clear logic. The threaded resolver still works!

Post by Mac Wang
Curl_thread_join(&td->thread_hnd);

...

Post by Mac Wang
I guess it's totally understandable that a 'join' would block here, for the
whole need for a threaded resolver is because the system resolver blocks. If
we can have only one name resolver thread for each connection (is that so?),

Well, perhaps not. First, I could imagine to pass over ownership of the
resolver to the multi handle so that a single handle can still terminate while
the resolver thread hasn't yet died. Secondly, there doesn't *have to* be a
strict 1:1 relationship between easy handles and their name resolver threads
but changing this will require proper concentration and design so that it
still works correctly. After all, the resolver thread uses the stock resolver
functions and they are blocking and they may take a very long time so just
blindly re-using another thread for resolving will introduce other problems.
But of course, perhaps we can allow an easy handle to start a new resolver
thread before the previous one has completed...

Possibly we would even figure out that not using a dedicated thread for each
handle isn't really possibly while still maintaining the same level of
functionality.

Post by Mac Wang
If we can have multiple threads and 'detach' them, we can potentially have
multiple (blocked) resolver threads running while the main system continues.

Yes, I believe it would be possible. Don't count on *me* working on that very
much though. In my high volume use cases I've used the c-ares backend instead
to avoid those pesky threading quirks... I'll of course still help out with
design and review of any work anyone would be intersted in pushing forward
within this area.

[Mac Wang]

If we can have multiple threads and 'detach' them, we can potentially have multiple (blocked) resolver threads running while the main system continues.

Yes, I believe it would be possible. Don't count on *me* working on that very much though.
In my high volume use cases I've used the c-ares backend instead to avoid those pesky threading quirks...
I'll of course still help out with design and review of any work anyone would be intersted in pushing forward within this area.

Sure. I'd trace the threaded resolver more and see if I can find a
work around. Thanks!
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html