Twitter4R Announcements

Written on 8:46:00 PM by S. Potter

First off I have been meaning to post an entry letting people know of a recently released Rails application written by Sergio Santos called TwitterNotes that is using the Twitter4R gem. Thanks Sergio (and anyone else that worked on the project - sorry if I left you off). Also Sergio emailed yesterday requesting message paging support with a suggested patch (thanks for contributing back - muchas gracias). This evening I released Twitter4R version 0.2.5 that incorporates the necessary code changes for this support. What does this mean? Previously you would have only been able to get the last 20 sent/received direct messages from the Twitter4R bindings doing something like the following:


messages = client.messages(:sent)
Now you can request specific pages like so:

messages = client.messages(:sent, :page => 2)
The same is true for the :received case. To install the Twitter4R RubyGem simply run the following in your terminal/console:
$ sudo gem install twitter4r
Give the Rubyforge mirrors a few hours to sync if you are only getting Twitter4R v0.2.4. Thanks and enjoy!

Custom Processors for ActiveWarehouse ETL

Written on 12:45:00 AM by S. Potter

For anyone interested in extending ActiveWarehouse ETL's features with custom pre/post processors, I thought I would share this piece of code that I wrote in August for a personal project I am working on. The example should provide you with enough details for you to create your own custom processors.


# Written by Susan Potter  under open source MIT license.
# August 12, 2007.

require 'net/ftp'

module ETL
  module Processor
    # Custom pre-processor to download files via FTP before beginning control process.
    class FtpDownloaderProcessor < ETL::Processor::Processor
      attr_reader :host
      attr_reader :port
      attr_reader :remote_dir
      attr_reader :files
      attr_reader :username
      attr_reader :local_dir
      
      # configuration options include:
      # * host - hostname or IP address of FTP server (required)
      # * port - port number for FTP server (default: 21)
      # * remote_dir - remote path on FTP server (default: /)
      # * files - list of files to download from FTP server (default: [])
      # * username - username for FTP server authentication (default: anonymous)
      # * password - password for FTP server authentication (default: nil)
      # * local_dir - local output directory to save downloaded files (default: '')
      # 
      # As an example you might write something like the following in your control process file:
      #  pre_process :ftp_downloader, {
      #    :host => 'ftp.sec.gov',
      #    :path => 'edgar/Feed/2007/QTR2',
      #    :files => ['20070402.nc.tar.gz', '20070403.nc.tar.gz', '20070404.nc.tar.gz', 
      #               '20070405.nc.tar.gz', '20070406.nc.tar.gz'],
      #    :local_dir => '/data/sec/2007/04',
      #  }
      # The above example will anonymously download via FTP the first week's worth of SEC filing feed data
      # from the second quarter of 2007 and download the files to the local directory +/data/sec/2007/04+.
      def initialize(control, configuration)
        @host = configuration[:host]
        @port = configuration[:port] || 21
        @remote_dir = configuration[:remote_dir] || '/'
        @files = configuration[:files] || []
        @username = configuration[:username] || 'anonymous'
        @password = configuration[:password]
        @local_dir = configuration[:local_dir] || ''
      end
      
      def process
        Net::FTP.open(@host) do |conn|
          conn.connect(@host, @port)
          conn.login(@username, @password)
          remote_files = conn.chdir(@remote_dir)
          @files.each do |f|
            conn.gettextfile(remote_file(f), local_file(f))
          end
        end
      end
      
      private
      attr_accessor :password
      
      def local_file(name)
        File.join(@local_dir, name)
      end
      
      def remote_file(name)
        File.join(@remote_dir, name)
      end
    end
  end
end
The key things to note from this is that you are at present required to:
  • define all your custom processors with in the ETL::Processor module
  • name your custom processor class in the form XXXXProcessor
  • need to extend (or really just adhere to the message interface of) ETL::Processor::Processor class defined in ActiveWarehouse ETL
  • define initialize taking two arguments (look above for guidance)
  • define a process method to do what you need do before or after the control process runs (for pre and post processors respectively)
Hope this helps someone customize ActiveWarehouse more easily, since the only bad thing I have found with ActiveWarehouse is lack of documentation.

Separating Rails Layout Associations

Written on 9:16:00 PM by S. Potter

I just realized I hadn't shared this code yet on this blog. I will also be including it in one of the metafusion subprojects (coming soon). I've been using it when needed in my Rails projects on and off for the last several months.

The Problem

You have some controllers you include into your application from plugins or engines and you want to associate a particular layout to the plugin/engine controller from within your application (without changing anything in the plugin or engine - of course!). There is not good way of doing this nicely in Rails presently as each controller usually defines controller-wide layouts within its definition.

The Solution

[finsignia/paths.rb]

# See:
# * Finsignia::Paths

# Contains helper methods related to paths and resolving modules and 
# classes from paths.  Provides for helpful mixin for various applications.
module Finsignia::Paths

  def self.included(base)
    base.extend ClassMethods
  end
  
  # Contains class methods for Finsignia::Paths mixin
  module ClassMethods
    @@element_postfixes = {:model => '', :controller => 'Controller'}

    def resolve_model(path)
      resolve_element :model, path
    end
  
    def resolve_controller(path)
      resolve_element :controller, path
    end
    
    def normalize_module_name(path)
      list = path.split('_')
      list.collect do |item|
        item.capitalize
      end.join
    end
    
    # Resolves path to a type of Rails element 
    # (e.g. model, controller, etc.).
    # 
    # Path refers to 'internal' path, NOT require path:
    #  'users/users' #=> internal path
    #  'users/users_controller #=> require path
    def resolve_element(type, path)
      first, rest = path.split('/')
      mod = ObjectSpace.const_get(normalize_module_name(first))
      while rest
        first, rest = rest.split('/')

        unless rest
          mod = mod.const_get(normalize_module_name("#{first}_#{@@element_postfixes[type].downcase}"))
        else
          mod = mod.const_get(normalize_module_name(first))
        end
        # I doubt this will ever get this far as per expeceted behavior of ObjectSpace and Module...so commented it out following unreachable line.
        #raise NameError.new("#{type} constant not found for internal path #{path}") unless mod
        return mod unless rest
      end      
    end
    
    # Separating this so we can stub this method out in specifications.
    def require_controller(path)
      require("#{path}_controller") unless "test" == ENV["RAILS_ENV"]
    end
    
    # Separating this so we can stub this method out in specifications.
    def require_model(path)
      require(path) unless "test" == ENV["RAILS_ENV"]
    end
  end
end

[finsignia/layouts.rb]

# See:
# * Finsignia::Layouts
# * Finsignia::LayoutsError

# Raised when an exceptional condition arises in the Layouts 
# mapping process.
class Finsignia::LayoutsError < Exception; end

# Provides a closure and declarative way to define layout
# mappings for an application that utilize controllers, views, 
# etc. from plugins.
#
# The closure approach simulates the Rails Routing approach 
# closely, like:
#  require 'application'
#  
#  Finsignia::Layouts.map do |map|
#    map.connect 'users/sessions', 'layout_name'
#  end
#
# The declarative approach looks like the following:
#  require 'application'
#  include Finsignia
#  Layouts.map 'users/sessions', 'layout_name'
# 
module Finsignia::Layouts
  class << self
    def connect(controller_path, layout)
      require_controller(controller_path)
      # resolve controller class and call .layout(layout) on it.
      controller = resolve_controller(controller_path)
      controller.layout(layout)
    end
    
    def map(&block)
      yield self if block_given?
    end
  end

#  private
    include Finsignia::Paths    
end

So basically in a file like config/layouts.rb of our application we have something like:

Finsignia::Layouts.map do |map|
  map.connect 'users/sessions', 'session'
  map.connect 'users/users', 'users'
  map.connect 'accounts/transaction', 'account'
end

Then include the config/layouts.rb file in config/environment.rb and you have separated your concerns relatively nicely and easily. When I release metafusion-rails my plan is that instead of installing the finsignia/paths.rb and finsignia/layouts.rb in the lib directory you would be able to do something like the following at the end of your config/environment.rb file:

gem('metafusion-rails', '=MFR_VERSION')
require 'metafusion/rails'

Finsignia::Layouts.map do |map|
  map.connect 'namespace/resource', 'layout'
  # etc....
end
Alternatively you could still keep the layouts.rb file if your environment.rb is getting large and keep the layouts.rb include in your environment.rb.

Model-Conductor-Controller fix

Written on 11:42:00 AM by S. Potter

The New Bamboo blog talked about Presenters & Conductors last month. The demonstration Rails application was broken, so the enclosed link to a zip file contains the fixes I made to get the application running as expected. The offending code was in lib/action_conductor/lib/action_conductor/errors.rb:


module ActiveRecord
  class Errors
   def add_conductor_errors_for(record, mapping)
    return if record.valid?
    record.errors.each do |attribute, message|
      self.add(mapping[attribute.to_sym], message)
    end
   end
  end
end

The fixed code changed one line as so:

module ActiveRecord
  class Errors
   def add_conductor_errors_for(record, mapping)
    return if record.valid? or record.new_record?
    record.errors.each do |attribute, message|
      self.add(mapping[attribute.to_sym], message)
    end
   end
  end
end

The fixed zip file can be found at my 4shared account and also fixes the minor view errors.