Microsoft SQL Server Primary Key Auto-Increment Woes

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Microsoft SQL Server Primary Key Auto-Increment Woes

John Myles White
I've recently been tasked at work with porting a large in-house web  
app that uses DBIx::Class from MySQL to Microsoft SQL Server. In the  
process, I've come across a problem where the auto-incrementing  
primary key column of the first row created using a DBIx::Class schema  
object is given an undef value rather than its true numeric value. All  
subsequent calls to create() against the same schema object give the  
proper numeric value of the primary key. Also, this problem does not  
come up if any other method such as find() or search() has been  
previously called against the schema object. Only a create() that is  
the very first method called on a given schema fails.

I unfortunately do not know enough about the architecture of  
DBix::Class to solve this by myself, but I am inclined to assume based  
on my initial poking around that the schema is not aware that it needs  
to use MSSQL specific workarounds for last_insert_id until after the  
first query is made to the database.

The following code snippet exhibits the problem consistently while  
using DBIx::Class 0.08009 with ActiveState's Perl build 822 and  
Microsoft SQL Server 2005.

my $schema = Schema->connect($dsn, $user, $pwd);

my $first_object = $schema->resultset('Object')->create({other_column  
=> 13});

if (not defined $first_object->object_id)
{
     print "No ID on first_object\n";
}

my $second_object = $schema->resultset('Object')->create({other_column  
=> 13});

if (not defined $second_object->object_id)
{
     print "No ID on second_object\n";
}

The next bit of code does not have any problems because a search() is  
called on the schema before the first create().

my $schema = Schema->connect($dsn, $user, $pwd);

my @objects = $schema->resultset('Object')->search();

my $first_object = $schema->resultset('Object')->create({other_column  
=> 13});

if (not defined $first_object->object_id)
{
     print "No ID on first_object\n";
}

my $second_object = $schema->resultset('Object')->create({other_column  
=> 13});

if (not defined $second_object->object_id)
{
     print "No ID on second_object\n";
}

If anyone wants more thorough code to test this, I can provide a  
complete tarball by e-mail to those interested.

My hope in writing to the mailing list is that someone can point me in  
the right direction to work on solving this at the level of the  
DBIx::Class source itself (assuming I'm not making an ass of myself by  
thinking that the problem is with DBIx::Class rather than with my use  
of it). With a little guidance, I would be happy to come up with a  
patch myself. At present I have a means of working around the problem  
temporarily by simply making useless queries in advance of any calls  
to create().

Thanks in advance for any comments anyone has to offer.

  -- John

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Microsoft SQL Server Primary Key Auto-Increment Woes

Michael Higgins
On Thu, 10 Jul 2008 10:27:56 -0400
John Myles White <[hidden email]> wrote:

> I've recently been tasked at work with porting a large in-house web  
> app that uses DBIx::Class from MySQL to Microsoft SQL Server. In the  
> process, I've come across a problem where the auto-incrementing  
> primary key column of the first row created using a DBIx::Class
> schema object is given an undef value rather than its true numeric
> value. All subsequent calls to create() against the same schema
> object give the proper numeric value of the primary key. Also, this
> problem does not come up if any other method such as find() or
> search() has been previously called against the schema object. Only a
> create() that is the very first method called on a given schema fails.
>
> I unfortunately do not know enough about the architecture of  
> DBix::Class to solve this by myself, but I am inclined to assume
> based on my initial poking around that the schema is not aware that
> it needs to use MSSQL specific workarounds for last_insert_id until
> after the first query is made to the database.
>
> The following code snippet exhibits the problem consistently while  
> using DBIx::Class 0.08009 with ActiveState's Perl build 822 and  
> Microsoft SQL Server 2005.
>

[snip]

Linux 2.6.24 (Gentoo) & this is perl, v5.8.8 built for i686-linux

Microsoft SQL Server 2000 DBIx::Class 0.08010

> (assuming I'm not making an ass of myself
> by thinking that the problem is with DBIx::Class rather than with my
> use of it). With a little guidance, I would be happy to come up with
> a patch myself. At present I have a means of working around the
> problem temporarily by simply making useless queries in advance of
> any calls to create().
>
> Thanks in advance for any comments anyone has to offer.
>


John --

No help, but I can confirm that I see the same behaviour. Occasionally
I put up a set of records with only one record. This inevitably throws a
warning.

I'll see about starting my script with a useless query to see if the
warnings go away. However, I do think the issue is specific to
DBIx::Class.

Cheers,

--
 |\  /|        |   |          ~ ~  
 | \/ |        |---|          `|` ?
 |    |ichael  |   |iggins    \^ /
 michael.higgins[at]evolone[dot]org

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Microsoft SQL Server Primary Key Auto-Increment Woes

Matt S Trout
On Fri, Jul 11, 2008 at 01:41:40PM -0700, Michael Higgins wrote:

> On Thu, 10 Jul 2008 10:27:56 -0400
> John Myles White <[hidden email]> wrote:
>
> > I've recently been tasked at work with porting a large in-house web  
> > app that uses DBIx::Class from MySQL to Microsoft SQL Server. In the  
> > process, I've come across a problem where the auto-incrementing  
> > primary key column of the first row created using a DBIx::Class
> > schema object is given an undef value rather than its true numeric
> > value. All subsequent calls to create() against the same schema
> > object give the proper numeric value of the primary key. Also, this
> > problem does not come up if any other method such as find() or
> > search() has been previously called against the schema object. Only a
> > create() that is the very first method called on a given schema fails.
> >
> > I unfortunately do not know enough about the architecture of  
> > DBix::Class to solve this by myself, but I am inclined to assume
> > based on my initial poking around that the schema is not aware that
> > it needs to use MSSQL specific workarounds for last_insert_id until
> > after the first query is made to the database.
> >
> > The following code snippet exhibits the problem consistently while  
> > using DBIx::Class 0.08009 with ActiveState's Perl build 822 and  
> > Microsoft SQL Server 2005.
> >
>
> [snip]
>
> Linux 2.6.24 (Gentoo) & this is perl, v5.8.8 built for i686-linux
>
> Microsoft SQL Server 2000 DBIx::Class 0.08010
>
> > (assuming I'm not making an ass of myself
> > by thinking that the problem is with DBIx::Class rather than with my
> > use of it). With a little guidance, I would be happy to come up with
> > a patch myself. At present I have a means of working around the
> > problem temporarily by simply making useless queries in advance of
> > any calls to create().
> >
> > Thanks in advance for any comments anyone has to offer.
> >
>
>
> John --
>
> No help, but I can confirm that I see the same behaviour. Occasionally
> I put up a set of records with only one record. This inevitably throws a
> warning.
>
> I'll see about starting my script with a useless query to see if the
> warnings go away. However, I do think the issue is specific to
> DBIx::Class.

Are you guys using MSSQL via ODBC?

I think because that tries to wrap $storage->insert it won't work unless
the $storage is already connected.

$schema->storage->ensure_connected;

should work as a query-free workaround if so.

Assuming that's true, can Marc Mims (the author) or anybody else who's
interested start looking at a fix?

--
      Matt S Trout       Need help with your Catalyst or DBIx::Class project?
   Technical Director                    http://www.shadowcat.co.uk/catalyst/
 Shadowcat Systems Ltd.  Want a managed development or deployment platform?
http://chainsawblues.vox.com/            http://www.shadowcat.co.uk/servers/

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Microsoft SQL Server Primary Key Auto-Increment Woes

Marc Mims
* Matt S Trout <[hidden email]> [080719 18:30]:

> Are you guys using MSSQL via ODBC?
>
> I think because that tries to wrap $storage->insert it won't work unless
> the $storage is already connected.
>
> $schema->storage->ensure_connected;
>
> should work as a query-free workaround if so.
>
> Assuming that's true, can Marc Mims (the author) or anybody else who's
> interested start looking at a fix?

It appears to me all that is required is $self->ensure_connected in the
overridden insert.  I'd be happy to make that change if one of the MSSQL
users will test it.

Although I wrote the module, I did so without the benefit of access to
MSSQL.  If the change proves to be more involved, I'd appreciate one of
the MSSQL users giving me some limited server access so I can test the
code.

        -Marc

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Microsoft SQL Server Primary Key Auto-Increment Woes

Michael Higgins
On Mon, 21 Jul 2008 08:56:17 -0700
Marc Mims <[hidden email]> wrote:

> * Matt S Trout <[hidden email]> [080719 18:30]:
> > Are you guys using MSSQL via ODBC?
> >
> > I think because that tries to wrap $storage->insert it won't work
> > unless the $storage is already connected.
> >
> > $schema->storage->ensure_connected;
> >
> > should work as a query-free workaround if so.
> >
> > Assuming that's true, can Marc Mims (the author) or anybody else
> > who's interested start looking at a fix?
>
> It appears to me all that is required is $self->ensure_connected in
> the overridden insert.  I'd be happy to make that change if one of
> the MSSQL users will test it.

Can do.

>
> Although I wrote the module, I did so without the benefit of access to
> MSSQL.  If the change proves to be more involved, I'd appreciate one
> of the MSSQL users giving me some limited server access so I can test
> the code.
>

Contact me off-list for the needed info for this, then. I can do that
too.

Cheers,


--
 |\  /|        |   |          ~ ~  
 | \/ |        |---|          `|` ?
 |    |ichael  |   |iggins    \^ /
 michael.higgins[at]evolone[dot]org


_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Microsoft SQL Server Primary Key Auto-Increment Woes

Matt S Trout
In reply to this post by Marc Mims
On Mon, Jul 21, 2008 at 08:56:17AM -0700, Marc Mims wrote:

> * Matt S Trout <[hidden email]> [080719 18:30]:
> > Are you guys using MSSQL via ODBC?
> >
> > I think because that tries to wrap $storage->insert it won't work unless
> > the $storage is already connected.
> >
> > $schema->storage->ensure_connected;
> >
> > should work as a query-free workaround if so.
> >
> > Assuming that's true, can Marc Mims (the author) or anybody else who's
> > interested start looking at a fix?
>
> It appears to me all that is required is $self->ensure_connected in the
> overridden insert.  I'd be happy to make that change if one of the MSSQL
> users will test it.

Not going to work - insert() is called too early.

I think you need to try and wrap _execute() instead and probably add
ensure_connected to insert() and friends.

I think the real solution is to not use reblessing for this stuff, but
that's not really something we can address until 09, I think
 

> Although I wrote the module, I did so without the benefit of access to
> MSSQL.  If the change proves to be more involved, I'd appreciate one of
> the MSSQL users giving me some limited server access so I can test the
> code.
>
> -Marc
>
> _______________________________________________
> List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
> IRC: irc.perl.org#dbix-class
> SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
> Searchable Archive: http://www.grokbase.com/group/dbix-class@...

--
      Matt S Trout       Need help with your Catalyst or DBIx::Class project?
   Technical Director                    http://www.shadowcat.co.uk/catalyst/
 Shadowcat Systems Ltd.  Want a managed development or deployment platform?
http://chainsawblues.vox.com/            http://www.shadowcat.co.uk/servers/

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Microsoft SQL Server Primary Key Auto-Increment Woes

Marc Mims
* Matt S Trout <[hidden email]> [080721 14:40]:

> I think you need to try and wrap _execute() instead and probably add
> ensure_connected to insert() and friends.
>
> I think the real solution is to not use reblessing for this stuff, but
> that's not really something we can address until 09, I think
>  
> > Although I wrote the module, I did so without the benefit of access to
> > MSSQL.  If the change proves to be more involved, I'd appreciate one of
> > the MSSQL users giving me some limited server access so I can test the
> > code.

Michael Higgins has given me access to an MSSQL server.  I'll go to work
on this.

        -Marc

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Microsoft SQL Server Primary Key Auto-Increment Woes

Marc Mims
In reply to this post by John Myles White
* John Myles White <[hidden email]> [080710 07:40]:

> I've recently been tasked at work with porting a large in-house web app
> that uses DBIx::Class from MySQL to Microsoft SQL Server. In the  
> process, I've come across a problem where the auto-incrementing primary
> key column of the first row created using a DBIx::Class schema object is
> given an undef value rather than its true numeric value. All subsequent
> calls to create() against the same schema object give the proper numeric
> value of the primary key. Also, this problem does not come up if any
> other method such as find() or search() has been previously called
> against the schema object. Only a create() that is the very first method
> called on a given schema fails.
> [snip]

I committed a fix for this.
http://dev.catalyst.perl.org/svnweb/bast/revision?rev=4750

Please retest with trunk and let me know if you have any problems.

A big thanks to Michael Higgins for getting me access to a MSSQL server
for debugging and testing.

        -Marc

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Microsoft SQL Server Primary Key Auto-Increment Woes

John Myles White
My apologies for the delay in testing your patch, Marc. What you've  
written completely resolves the problem.

Thank you so much for your help.

  -- John

On Aug 12, 2008, at 1:47 PM, Marc Mims wrote:

> * John Myles White <[hidden email]> [080710 07:40]:
>> I've recently been tasked at work with porting a large in-house web  
>> app
>> that uses DBIx::Class from MySQL to Microsoft SQL Server. In the
>> process, I've come across a problem where the auto-incrementing  
>> primary
>> key column of the first row created using a DBIx::Class schema  
>> object is
>> given an undef value rather than its true numeric value. All  
>> subsequent
>> calls to create() against the same schema object give the proper  
>> numeric
>> value of the primary key. Also, this problem does not come up if any
>> other method such as find() or search() has been previously called
>> against the schema object. Only a create() that is the very first  
>> method
>> called on a given schema fails.
>> [snip]
>
> I committed a fix for this.
> http://dev.catalyst.perl.org/svnweb/bast/revision?rev=4750
>
> Please retest with trunk and let me know if you have any problems.
>
> A big thanks to Michael Higgins for getting me access to a MSSQL  
> server
> for debugging and testing.
>
> -Marc


_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Microsoft SQL Server Primary Key Auto-Increment Woes

Matt S Trout
On Sat, Aug 30, 2008 at 09:46:45AM -0400, John Myles White wrote:
> My apologies for the delay in testing your patch, Marc. What you've  
> written completely resolves the problem.

Top and large.

Thanks everybody who pitched in on this.

--
      Matt S Trout       Need help with your Catalyst or DBIx::Class project?
   Technical Director                    http://www.shadowcat.co.uk/catalyst/
 Shadowcat Systems Ltd.  Want a managed development or deployment platform?
http://chainsawblues.vox.com/            http://www.shadowcat.co.uk/servers/

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Loading...