rubyonrailsin

A Ruby and Rails talk

Thursday, April 1, 2010


Re: [Rails] Re: how to insert a batch of records into db more efficientl

by rubyonrailsin 0 comments

Tag


Share this post:
Design Float
StumbleUpon
Reddit

On Thu, Apr 1, 2010 at 8:50 AM, Fitz Fitz <lists@ruby-forum.com> wrote:

I have been facing this problem for a while now.  ar-extensions is
really good but it doesn't preserve the links between models like
one-to-one many-to-many etc.

Have you tried doing batches via transactions? By default activerecord will make every save operation it's own transaction which adds overhead to the DB in many setups. See ActiveRecord::Transactions::ClassMethods, and put ten or a hundred records at a time in a transaction wrapper and see what you gain. The improvement you see from this depends on the DB engine in use as well as how much the DB is actually the bottleneck versus the processing you're doing application-side per record. I suspect your problem is actually a combination of both DB overhead and unoptimized application logic. You might want to try some profiling of your application to see where the bottleneck really is - at the scale of 1M operations per day it's worth doing some of that.

That's my 2 cents, the disclaimer is that I am by no means a rails expert. Just trying to help out where I can.

jsw

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To post to this group, send email to rubyonrails-talk@googlegroups.com.
To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.

No comments:

Post a Comment

Subscribe feeds via e-mail

Blog Archive