Connection timeouts?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Connection timeouts?

Bill Moseley
Any suggestions how to set a connection timeout connections?

I'm using the Replicated storage.   If there's a single slave configured and it's not running (listening on a socket) DBIC will return quickly and fall back to using the master.

But, if the slave is listening but not responding then DBIC will hang waiting on a response.   This can be demonstrated by using netcat to listen to port and configure the slave to connect there.



--
Bill Moseley
[hidden email]

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|

Re: Connection timeouts?

Ben Tilly
The obvious thing to try is to use the alarm() function to send
yourself a signal and set a signal handler that will just die to
escape.  But this is unlikely to work due to safe signals.  Search for
"Safe Signals" in perldoc perlipc for details.  It will also show you
how to use the POSIX module to set a signal handler that can work
despite that.

On Thu, Oct 9, 2014 at 4:33 PM, Bill Moseley <[hidden email]> wrote:

> Any suggestions how to set a connection timeout connections?
>
> I'm using the Replicated storage.   If there's a single slave configured and
> it's not running (listening on a socket) DBIC will return quickly and fall
> back to using the master.
>
> But, if the slave is listening but not responding then DBIC will hang
> waiting on a response.   This can be demonstrated by using netcat to listen
> to port and configure the slave to connect there.
>
>
>
> --
> Bill Moseley
> [hidden email]
>
> _______________________________________________
> List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
> IRC: irc.perl.org#dbix-class
> SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
> Searchable Archive:
> http://www.grokbase.com/group/dbix-class@...

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|

Re: Connection timeouts?

Bill Moseley
On Thu, Oct 9, 2014 at 6:32 PM, Ben Tilly <[hidden email]> wrote:
The obvious thing to try is to use the alarm() function to send
yourself a signal and set a signal handler that will just die to
escape.  But this is unlikely to work due to safe signals.  Search for
"Safe Signals" in perldoc perlipc for details.  It will also show you
how to use the POSIX module to set a signal handler that can work
despite that.

But where?   Set storage_type and override/wrap the private method _connect or _populate_dbh?  I'm wondering what's a good place to hook into.

 


 

On Thu, Oct 9, 2014 at 4:33 PM, Bill Moseley <[hidden email]> wrote:
> Any suggestions how to set a connection timeout connections?
>
> I'm using the Replicated storage.   If there's a single slave configured and
> it's not running (listening on a socket) DBIC will return quickly and fall
> back to using the master.
>
> But, if the slave is listening but not responding then DBIC will hang
> waiting on a response.   This can be demonstrated by using netcat to listen
> to port and configure the slave to connect there.
>
>
>
> --
> Bill Moseley
> [hidden email]
>
> _______________________________________________
> List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
> IRC: irc.perl.org#dbix-class
> SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
> Searchable Archive:
> http://www.grokbase.com/group/dbix-class@...

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...



--
Bill Moseley
[hidden email]

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|

Re: Connection timeouts?

Bill Moseley


On Sat, Dec 20, 2014 at 5:34 AM, Peter Rabbitson <[hidden email]> wrote:
On 10/10/2014 06:37 AM, Bill Moseley wrote:
On Thu, Oct 9, 2014 at 6:32 PM, Ben Tilly <[hidden email]
<mailto:[hidden email]>> wrote:

    The obvious thing to try is to use the alarm() function to send
    yourself a signal and set a signal handler that will just die to
    escape.  But this is unlikely to work due to safe signals.  Search for
    "Safe Signals" in perldoc perlipc for details.  It will also show you
    how to use the POSIX module to set a signal handler that can work
    despite that.


But where?   Set storage_type and override/wrap the private method
_connect or _populate_dbh?  I'm wondering what's a good place to hook into.


Hi Bill

I am going through my backlog, wondering if you still need help with this or you figured out a way forward?


Hi Peter,

No, I never followed up and made any code changes.  

I think when I posted this we had just moved to configuring a single slave in DBIC which talked to HA Proxy configured to load balance a number of pg_bouncer instances in front of Postgresql databases.    We likely had some config hiccup where HA Proxy accepted the connection yet there was no pg_bouncer (or Postgresql?) connection available and it just blocked.

Have not had problems like that since then, but obviously having a database connection block for a long time brings a busy site to its knees.

Of course, timing out and falling back to the master may not result in better results if the master cannot support the load alone.   But, better to return an error quickly than blocking.


I'm not sure it makes sense to have the timeout in DBIC -- if it was then maybe just in the _ping method.   But, in general how long to wait depends on what is expected from the request which is only known at the app level.



--
Bill Moseley
[hidden email]

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...