Review needed for proposed Pg replication enhancement

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Review needed for proposed Pg replication enhancement

Peter Rabbitson-2
Greetings!

I am sending this to the list, but I specifically want to hear from Bill
Moseley, as I know they (used to) run an involved Pg replicated setup.

There is a proposal to "bless" the Pg 9.0 hot-standby replication into
the core DBIC::Storage::DBI::Pg driver:
https://github.com/dbsrgits/dbix-class/pull/103/files

The patch itself is small and straightforward. But given this is a
one-way change (we can't un-bless it once it ships) I want to ask the
thoughts of any existing users.

This "request for comments" will be open for about a week, so please do
speak up sooner rather than later.

Cheers

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Review needed for proposed Pg replication enhancement

Dave Howorth
On 2016-04-22 10:13, Peter Rabbitson wrote:

> Greetings!
>
> I am sending this to the list, but I specifically want to hear from Bill
> Moseley, as I know they (used to) run an involved Pg replicated setup.
>
> There is a proposal to "bless" the Pg 9.0 hot-standby replication into
> the core DBIC::Storage::DBI::Pg driver:
> https://github.com/dbsrgits/dbix-class/pull/103/files
>
> The patch itself is small and straightforward. But given this is a
> one-way change (we can't un-bless it once it ships) I want to ask the
> thoughts of any existing users.

I know very little about this, so apologies if I am speaking out of
turn. Is the code safe in the sense it will work if applied to an older
PG that does not know about replication?

The patch itself would be a little larger if it included some tests and
documentation.

Cheers, Dave

> This "request for comments" will be open for about a week, so please do
> speak up sooner rather than later.
>
> Cheers

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Review needed for proposed Pg replication enhancement

Peter Rabbitson-2
On 04/22/2016 12:13 PM, Dave Howorth wrote:
>
> I know very little about this, so apologies if I am speaking out of
> turn. Is the code safe in the sense it will work if applied to an older
> PG that does not know about replication?

None of this is referenced in any way unless one is using
::Storage::Replicated in the first place. This question is specifically
for such users.

>
> The patch itself would be a little larger if it included some tests and
> documentation.
>

The parts are documented, they just live in a different spot:
https://metacpan.org/pod/DBIx::Class::Storage::DBI#is_replicating.
Specific storages can override them, which is what this patch does.

As far as testing - it would be nice of course, but it would require an
actual replicated setup, which is fiddly business. If someone is
interested to implement this as part of the CI here[1] and here[2] -
feel free to take a stab at it.

[1]
https://github.com/dbsrgits/dbix-class/blob/master/maint/travis-ci_scripts/10_before_install.bash#L132-L137
[2]
https://github.com/dbsrgits/dbix-class/blob/master/t/storage/replicated.t#L214-L233

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Review needed for proposed Pg replication enhancement

Bill Moseley
In reply to this post by Peter Rabbitson-2


On Fri, Apr 22, 2016 at 2:13 AM, Peter Rabbitson <[hidden email]> wrote:
Greetings!

I am sending this to the list, but I specifically want to hear from Bill Moseley, as I know they (used to) run an involved Pg replicated setup.

I'll take a closer look tomorrow, but our Pg replication setup has changed quite a bit.

We were using Slony and had added code to check replicant (slave) lag and when exceed a threshold would stick to the master. We did experience lags often.

We would also stick to the master (for a given user) after an update for about the current replicant lag.

Me moved to using Postgresql native replication support.   But, we removed the replicant lag checks because we now do that via HAProxy -- and HAProxy falls back to the master if needed.  We use PgBouncer for connection pooling.

I know it doesn't sound like it, but it simplifies things a bit.   The apps are just configured for a master and a single replicant using the existing DBIC replication code and the slave should always be available (even if all real replicants have failed because will ultimately fall back to the master).

Also, this means DBAs can add or move replicants (or promote a replicant to a master) w/o making any application changes.



 

There is a proposal to "bless" the Pg 9.0 hot-standby replication into the core DBIC::Storage::DBI::Pg driver: https://github.com/dbsrgits/dbix-class/pull/103/files

The patch itself is small and straightforward. But given this is a one-way change (we can't un-bless it once it ships) I want to ask the thoughts of any existing users.

This "request for comments" will be open for about a week, so please do speak up sooner rather than later.

Cheers



--
Bill Moseley
[hidden email]

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Loading...