Ruby Import from CSV into Spree, Half The Rails Way and Half The Un-Rails Way
Posted by admin
There are several useful aspects to this script for anyone who is using ruby for CSV import, especially if using ActiveRecord in Rails, and definitely if importing into Spree.
Big thanks to Chadwick Wood for the imagemagick part, which I borrowed from his post: Batch Image Resizing with Ruby and ImageMagick. Ran this locally on OS X and remotely on FreeBSD 4.x/5.x without any hiccups.
Here is the first line of the CSV file (product_data.csv):
"original_id","title","subtitle","description","price","sku","weight","category","category_group","category_name","image_attachments"Here is the CSV import script (import_from_csv.rb):
require 'rubygems'
require 'csv'
require 'csv-mapper'
require 'active_record'
require 'active_support'
require 'paperclip'
require 'spree'
require 'action_controller'
require 'action_controller/test_process'
require 'ftools'
require 'fileutils'
require 'open3'
# This is how you access your ActiveRecord objects in rails (referencing your database.yml), specifying which database, and logging all the database activity to a file:
ActiveRecord::Base.logger = Logger.new(File.dirname(__FILE__) + '/debug.log')
ActiveRecord::Base.configurations = YAML::load(IO.read(File.dirname(__FILE__) + '/config/database.yml'))
ActiveRecord::Base.establish_connection('development')
# It turned out I didn't need this, but you might if you are going to attach on-the-fly with Paperclip:
#Paperclip.options[:command_path] = '/opt/local/bin/'
# Initialize ActiveRecord objects:
class Product < ActiveRecord::Base; end
class Variant < ActiveRecord::Base; end
class Taxon < ActiveRecord::Base; end
class Asset < ActiveRecord::Base; end
class Image < Asset; end
# Do not reinvent wheel, instead install the gem (http://csv-mapper.rubyforge.org/) and include this if you know what's good for you:
include CsvMapper
# Read in the CSV file (name your attributes in the first line of the CSV file, which is a normal thing to do):
results = import('./product_data.csv') do
read_attributes_from_file
end
# FYI this destroy part destroys everything (drops all values in the tables `products`, `variants`, and `assets`):
Product.all.each do |j| j.destroy end
Variant.all.each do |k| k.destroy end
Asset.all.each do |a| a.destroy end
# Enter the loop where we iterate the records:
results.each do |item|
new_product = Product.create(
:name => item.title,
:meta_description => item.subtitle,
:description => item.description,
:permalink => item.title.downcase.gsub(/[\/.]/, " ").split(" ").join("-"),
:available_on => Time.now
)
new_variant = Variant.create(
:product_id => new_product.id,
:is_master => 1,
:sku => item.sku,
:price => item.price,
:weight => item.weight
)
# Doesn't work:
#new_product.taxons << Taxon.find_by_id(item.category_group.to_i + 999)
# For taxon assignment, insert directly to the database:
taxon_id = item.category_group.to_i + 1000
sql_query_for_taxon_assignment = "INSERT INTO `products_taxons` (product_id, taxon_id) VALUES (#{new_product.id}, #{taxon_id});"
ActiveRecord::Base.connection.execute(sql_query_for_taxon_assignment)
# Now we get into imagemagick and Paperclip and creating Asset objects of type Image
# For some reason, the previous developer put all the image filenames in rows of the category database as strings delimited by pipes ("|"). Don't do that. But if you have to deal with it:
pics = item.image_attachments.split("|")
# Loop through the elements of the pics array and add image attachments
pics.each do |image_basename_part|
next if image_basename_part =~ /^\s*$/ # skip blank lines
# get the image filenames
image_file = `find public/php/images/products/* | grep -i "/#{image_basename_part}[.]"`.chomp
puts "Using file: #{image_file} (found by grepping for #{image_basename_part})"
next if image_file.blank?
mime = "image/" + image_file.match(/\w+$/).to_s
image_file_basename = File.basename(image_file, ".jpg")
# get the image dimensions
geometry = Paperclip::Geometry.from_file(image_file)
# add a record to the `assets` table
new_asset = Asset.create(
:viewable_id => new_product.id,
:viewable_type => "Product",
:attachment_content_type => "image/jpeg",
:attachment_file_name => image_file_basename+'.jpg',
:type => "Image",
:attachment_updated_at => Time.now,
:attachment_width => geometry.width,
:attachment_height => geometry.height
)
# Resize each image and place in a new directory under assets in the structure and dimensions that spree expects
sizes = ["mini", "small", "product", "medium", "large", "original"]
@mini_dimension = '77x77'
@small_dimension = '175x175'
@product_dimension = '265x265'
@medium_dimension = '425x425'
@large_dimension = '870x870'
sizes.each do |size|
case size
when "mini"
this_dimension = @mini_dimension
when "small"
this_dimension = @small_dimension
when "product"
this_dimension = @product_dimension
when "medium"
this_dimension = @medium_dimension
when "large"
this_dimension = @large_dimension
when "original"
this_dimension = geometry.to_s
end
new_image_file = 'public/assets/products/'+new_asset.id.to_s+'/'+size+'/'+image_file_basename+'.jpg'
# make the new path
FileUtils.mkdir_p 'public/assets/products/'+new_asset.id.to_s+'/'+size+'/'
@cmd = "convert \"#{image_file}\" -resize #{this_dimension} -quality 90 #{new_image_file}"
stdin, stdout, stderr = Open3.popen3(@cmd)
puts l while l = stdout.gets
puts l while l = stderr.gets
end
end
end
# End of the record-iterating loop