REPOST: DBIx::Class and caching

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

REPOST: DBIx::Class and caching

Oleg Pronin


---------- Пересланное сообщение ----------
От: Oleg Pronin <[hidden email]>
Дата: 25 июня 2008 г. 12:17
Тема: Re: [Dbix-class] DBIx::Class and caching
Кому: [hidden email], "DBIx::Class user and developer list" <[hidden email]>


This is how i cache resultset search results in memcache:

There 2 modules in attach.

1) To use memcaching for some object you need to set ResultSetMemcached as its resultset class (or resultset which derives from it).
2) Code in ResultSource.pm must be in your base resultsource class.
3) You have to set up on start:
     ResultSetMemcached->memd (а memd object)
     ResultSource->schema_by_class({
         'MyApp::Schema' => $myappSchemaObject,
         'MyApp::OtherSchema' => $myappOtherSchemaObject,
     });
     This is needed for correct restoring from cache;

Usage:

$someRS->search(
    {ANYCOND},
    {ATTRS,
      cache_for => 30
    },
);
or $someRS->search(...)->search(...)->cache_for(30)->search(...)->all;
or $someRS->single({id => $id}, {cache_for => 120});

The key for memcache is generated automatically.
You can generate it manually
$someRS->search({id => $id}, {
    cache_for => 30,
    cache_key => 'user_'.$id,
});

Some code can be executed before object is cached (from DB):
$someRS->search({id => $id}, {
    cache_for => 30,
    cache_key => 'user_'.$id,
    cache_after => sub { $_[0]->set_some_difficult_virtual_data(....) },
});

2008/6/24 Andreas Pronakis <[hidden email]>:

Forgive me if this question sounds like asking for baby step instructions  but I hope it will benefit other DBIx::Class starters.

We have a lot of procedural code that we are looking to convert to DOM (Domain Object Modeling) and one of the tools we are lokoing to use is DBIx::Class (instead of writing our own Model which could be fun but a maintenance nightmare).

Anyway, the set up, we are look is mod_perl, DBIx::Class, MySQL backend, and some sort of MVC concept but we will not (for now) be using Catalyst (don't ask why).

Based on the above information my questions are:
1. ResultSet has a couple of methods related to caching, but going by the example given (http://search.cpan.org/~ash/DBIx-Class-0.08010/lib/DBIx/Class/ResultSet.pm#cache)  it seems like you have to turn caching on a per search()/find() call, is there some way of turning it on for the whole class (even better all classes) in one place (i.e. Parent class that all other classes inherit from)?
1.1 Is overriding/subclass resultset/ResultSet to capture search/find request to add the cache parameter an options to achieve the above?
2. Is there a different way to cache objects and/or ResultSet objects?
3. Can we cache SQL natively in DBIx::Class i.e. the SQL executed by a find, search or manual SELECT - I guess if 1 and 1.1 are supported SQL caching might be redundant except from manual SELECTs?
4. If there isn't native support for the above, can someone give me any advice and/or examples regarding using DBIx::Class and MemCache to achieve the same result?
5. I came across the module DBIx::Class::Cursor::Cached but since the documentation is a bit on the thin side, I was wondering if someone can explain it in a bit more detail.  For example is it possible to use it to achieve points 1/1.1 and also set a default cache_for periodf for a whole class, rather than individual search requests?

Thanks in advance for your patience and help

Andreas Pronakis
DBIx::Class newbie but believer




_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...



_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...

ResultSetMemcached.pm (3K) Download Attachment
ResultSource.pm (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: REPOST: DBIx::Class and caching

Alexander Hartmaier

This looks promising to me (but I’m sure mst will find an argument against it ;-).

If not it would be a great  addition to dbic or a nice standalone module!

 

-Alex

 

From: Oleg Pronin [mailto:[hidden email]]
Sent: Thursday, July 03, 2008 4:19 PM
To: [hidden email]
Subject: REPOST: [Dbix-class] DBIx::Class and caching

 

 

---------- Пересланное сообщение ----------
От: Oleg Pronin <[hidden email]>
Дата: 25 июня 2008 г. 12:17
Тема: Re: [Dbix-class] DBIx::Class and caching
Кому: [hidden email], "DBIx::Class user and developer list" <[hidden email]>


This is how i cache resultset search results in memcache:

There 2 modules in attach.

1) To use memcaching for some object you need to set ResultSetMemcached as its resultset class (or resultset which derives from it).
2) Code in ResultSource.pm must be in your base resultsource class.
3) You have to set up on start:
     ResultSetMemcached->memd (а memd object)
     ResultSource->schema_by_class({
         'MyApp::Schema' => $myappSchemaObject,
         'MyApp::OtherSchema' => $myappOtherSchemaObject,
     });
     This is needed for correct restoring from cache;

Usage:

$someRS->search(
    {ANYCOND},
    {ATTRS,
      cache_for => 30
    },
);
or $someRS->search(...)->search(...)->cache_for(30)->search(...)->all;
or $someRS->single({id => $id}, {cache_for => 120});

The key for memcache is generated automatically.
You can generate it manually
$someRS->search({id => $id}, {
    cache_for => 30,
    cache_key => 'user_'.$id,
});

Some code can be executed before object is cached (from DB):
$someRS->search({id => $id}, {
    cache_for => 30,
    cache_key => 'user_'.$id,
    cache_after => sub { $_[0]->set_some_difficult_virtual_data(....) },
});

2008/6/24 Andreas Pronakis <[hidden email]>:

 

Forgive me if this question sounds like asking for baby step instructions  but I hope it will benefit other DBIx::Class starters.

We have a lot of procedural code that we are looking to convert to DOM (Domain Object Modeling) and one of the tools we are lokoing to use is DBIx::Class (instead of writing our own Model which could be fun but a maintenance nightmare).

Anyway, the set up, we are look is mod_perl, DBIx::Class, MySQL backend, and some sort of MVC concept but we will not (for now) be using Catalyst (don't ask why).

Based on the above information my questions are:
1. ResultSet has a couple of methods related to caching, but going by the example given (http://search.cpan.org/~ash/DBIx-Class-0.08010/lib/DBIx/Class/ResultSet.pm#cache)  it seems like you have to turn caching on a per search()/find() call, is there some way of turning it on for the whole class (even better all classes) in one place (i.e. Parent class that all other classes inherit from)?
1.1 Is overriding/subclass resultset/ResultSet to capture search/find request to add the cache parameter an options to achieve the above?
2. Is there a different way to cache objects and/or ResultSet objects?
3. Can we cache SQL natively in DBIx::Class i.e. the SQL executed by a find, search or manual SELECT - I guess if 1 and 1.1 are supported SQL caching might be redundant except from manual SELECTs?
4. If there isn't native support for the above, can someone give me any advice and/or examples regarding using DBIx::Class and MemCache to achieve the same result?
5. I came across the module DBIx::Class::Cursor::Cached but since the documentation is a bit on the thin side, I was wondering if someone can explain it in a bit more detail.  For example is it possible to use it to achieve points 1/1.1 and also set a default cache_for periodf for a whole class, rather than individual search requests?

Thanks in advance for your patience and help

Andreas Pronakis
DBIx::Class newbie but believer




_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...

 

 


*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
Handelsgericht Wien, FN 79340b
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
Notice: This e-mail contains information that is confidential and may be privileged.
If you are not the intended recipient, please notify the sender and then
delete this e-mail immediately.
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: REPOST: DBIx::Class and caching

Oleg Pronin


2008/7/16 Hartmaier Alexander <[hidden email]>:

This looks promising to me (but I'm sure mst will find an argument against it ;-).


The main argument is perfomance for this thing (it does matter in huge projects).
 

If not it would be a great  addition to dbic or a nice standalone module!


maybe, (if i will understand how to make it more convenient for configuring)
 

 

-Alex

 

From: Oleg Pronin [mailto:[hidden email]]
Sent: Thursday, July 03, 2008 4:19 PM
To: [hidden email]
Subject: REPOST: [Dbix-class] DBIx::Class and caching

 

 

---------- Пересланное сообщение ----------
От: Oleg Pronin <[hidden email]>
Дата: 25 июня 2008 г. 12:17
Тема: Re: [Dbix-class] DBIx::Class and caching
Кому: [hidden email], "DBIx::Class user and developer list" <[hidden email]>


This is how i cache resultset search results in memcache:

There 2 modules in attach.

1) To use memcaching for some object you need to set ResultSetMemcached as its resultset class (or resultset which derives from it).
2) Code in ResultSource.pm must be in your base resultsource class.
3) You have to set up on start:
     ResultSetMemcached->memd (а memd object)
     ResultSource->schema_by_class({
         'MyApp::Schema' => $myappSchemaObject,
         'MyApp::OtherSchema' => $myappOtherSchemaObject,
     });
     This is needed for correct restoring from cache;

Usage:

$someRS->search(
    {ANYCOND},
    {ATTRS,
      cache_for => 30
    },
);
or $someRS->search(...)->search(...)->cache_for(30)->search(...)->all;
or $someRS->single({id => $id}, {cache_for => 120});

The key for memcache is generated automatically.
You can generate it manually
$someRS->search({id => $id}, {
    cache_for => 30,
    cache_key => 'user_'.$id,
});

Some code can be executed before object is cached (from DB):
$someRS->search({id => $id}, {
    cache_for => 30,
    cache_key => 'user_'.$id,
    cache_after => sub { $_[0]->set_some_difficult_virtual_data(....) },
});

2008/6/24 Andreas Pronakis <[hidden email]>:

 

Forgive me if this question sounds like asking for baby step instructions  but I hope it will benefit other DBIx::Class starters.

We have a lot of procedural code that we are looking to convert to DOM (Domain Object Modeling) and one of the tools we are lokoing to use is DBIx::Class (instead of writing our own Model which could be fun but a maintenance nightmare).

Anyway, the set up, we are look is mod_perl, DBIx::Class, MySQL backend, and some sort of MVC concept but we will not (for now) be using Catalyst (don't ask why).

Based on the above information my questions are:
1. ResultSet has a couple of methods related to caching, but going by the example given (http://search.cpan.org/~ash/DBIx-Class-0.08010/lib/DBIx/Class/ResultSet.pm#cache)  it seems like you have to turn caching on a per search()/find() call, is there some way of turning it on for the whole class (even better all classes) in one place (i.e. Parent class that all other classes inherit from)?
1.1 Is overriding/subclass resultset/ResultSet to capture search/find request to add the cache parameter an options to achieve the above?
2. Is there a different way to cache objects and/or ResultSet objects?
3. Can we cache SQL natively in DBIx::Class i.e. the SQL executed by a find, search or manual SELECT - I guess if 1 and 1.1 are supported SQL caching might be redundant except from manual SELECTs?
4. If there isn't native support for the above, can someone give me any advice and/or examples regarding using DBIx::Class and MemCache to achieve the same result?
5. I came across the module DBIx::Class::Cursor::Cached but since the documentation is a bit on the thin side, I was wondering if someone can explain it in a bit more detail.  For example is it possible to use it to achieve points 1/1.1 and also set a default cache_for periodf for a whole class, rather than individual search requests?

Thanks in advance for your patience and help

Andreas Pronakis
DBIx::Class newbie but believer




_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...

 

 


*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
T-Systems Austria GesmbH Rennweg 97-99, 1030 Wien
Handelsgericht Wien, FN 79340b
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
Notice: This e-mail contains information that is confidential and may be privileged.
If you are not the intended recipient, please notify the sender and then
delete this e-mail immediately.
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...


_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: REPOST: DBIx::Class and caching

Matt S Trout
On Thu, Jul 17, 2008 at 07:18:08PM +0400, Oleg Pronin wrote:

> 2008/7/16 Hartmaier Alexander <[hidden email]>:
>
> >  This looks promising to me (but I'm sure mst will find an argument
> > against it ;-).
> >
>
> The main argument is perfomance for this thing (it does matter in huge
> projects).
>
>
> > If not it would be a great  addition to dbic or a nice standalone module!
> >
>
> maybe, (if i will understand how to make it more convenient for configuring)

Any reason why you can't do it via resultset attributes like Cursor::Cached
does?

--
      Matt S Trout       Need help with your Catalyst or DBIx::Class project?
   Technical Director                    http://www.shadowcat.co.uk/catalyst/
 Shadowcat Systems Ltd.  Want a managed development or deployment platform?
http://chainsawblues.vox.com/            http://www.shadowcat.co.uk/servers/

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: REPOST: DBIx::Class and caching

Oleg Pronin
First that comes in my mind is that 'single' does not accept attributes. But caching in 'single' is often needed.
Doing $rs->search(...)->single() is not pretty

2008/7/20 Matt S Trout <[hidden email]>:
On Thu, Jul 17, 2008 at 07:18:08PM +0400, Oleg Pronin wrote:
> 2008/7/16 Hartmaier Alexander <[hidden email]>:
>
> >  This looks promising to me (but I'm sure mst will find an argument
> > against it ;-).
> >
>
> The main argument is perfomance for this thing (it does matter in huge
> projects).
>
>
> > If not it would be a great  addition to dbic or a nice standalone module!
> >
>
> maybe, (if i will understand how to make it more convenient for configuring)

Any reason why you can't do it via resultset attributes like Cursor::Cached
does?

--
     Matt S Trout       Need help with your Catalyst or DBIx::Class project?
  Technical Director                    http://www.shadowcat.co.uk/catalyst/
 Shadowcat Systems Ltd.  Want a managed development or deployment platform?
http://chainsawblues.vox.com/            http://www.shadowcat.co.uk/servers/


_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: REPOST: DBIx::Class and caching

Matt S Trout
On Sun, Jul 20, 2008 at 06:11:00PM +0400, Oleg Pronin wrote:
> First that comes in my mind is that 'single' does not accept attributes. But
> caching in 'single' is often needed.
> Doing $rs->search(...)->single() is not pretty

Right, but single doesn't accept attributes specifically to avoid burning
CPU constructing an extra rs. I guess we could make single() support it
and have it do the search() under the hood? No objections here to a patch
for that.

--
      Matt S Trout       Need help with your Catalyst or DBIx::Class project?
   Technical Director                    http://www.shadowcat.co.uk/catalyst/
 Shadowcat Systems Ltd.  Want a managed development or deployment platform?
http://chainsawblues.vox.com/            http://www.shadowcat.co.uk/servers/

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Loading...