Rails before_validation strip whitespace best practices

Question

I would like my User model to sanitize some input before before save. For now some simple whitespace stripping will do. So to avoid people registering with "Harry " and pretend to be "Harry", for example.

I assume it is a good idea to do this stripping before validation, so that the validates_uniqueness_of can avoid accidental duplicates.

class User < ActiveRecord::Base
  has_many :open_ids

  validates_presence_of :name
  validates_presence_of :email
  validates_uniqueness_of :name
  validates_uniqueness_of :email
  validates_format_of :email, :with => /\A([^@\s]+)@((?:[-a-z0-9]+\.)+[a-z]{2,})\Z/i

  before_validation :strip_whitespace, :only => [:name, :email, :nick]

  private
  def strip_whitespace(value)
    value.responds_to?('strip') ? value.strip : value
  end
end

However, this code comes with an error ArgumentError: wrong number of arguments (0 for 1). I assumed the callback would be passed the values.

Also: is this stripping actually a good idea? Or should I rather validate on space and tell the user that "Harry " contains invalid spacess (I want to allow "Harry Potter" but not "Harry\s\sPotter").

Edit: As pointed out in a comment, my code is wrong (which is why I was asking the question a.o.). Please make sure you read the accepted answer in addition to my question for the correct code and to avoid the same mistakes I made.

Answer 1

I don't believe before_validation works like that. You probably want to write your method like this instead:

def strip_whitespace
  self.name = self.name.strip unless self.name.nil?
  self.email = self.email.strip unless self.email.nil?
  self.nick = self.nick.strip unless self.nick.nil?
end

You could make it more dynamic if you want using something like self.columns, but that's the gist of it.

Answer 2

There are several gems to do this automatically. Those gems work in the similar way of creating callback in before_validation. One good gem is at https://github.com/holli/auto_strip_attributes

gem "auto_strip_attributes", "~> 2.2"

class User < ActiveRecord::Base
  auto_strip_attributes :name, :nick, nullify: false, squish: true
  auto_strip_attributes :email
end

Stripping is often a good idea. Especially for leading and trailing whitespaces. User often creates trailing spaces when copy/pasting value to a form. With names and other identifying strings you also might want squish the string. So that "Harry    Potter" will become "Harry Potter" (squish option in the gem).

Answer 3

Charlie's answer is good, but there's a little verbosity. Here's a tighter version:

def clean_data
  # trim whitespace from beginning and end of string attributes
  attribute_names.each do |name|
    if send(name).respond_to?(:strip)
      send("#{name}=", send(name).strip)
    end
  end
end

The reason we use

self.foo = "bar"

instead of

foo = "bar"

in the context of ActiveRecord objects is that Ruby interprets the latter as a local variable assignment. It will just set the foo variable in your method scope, instead of calling the "foo=" method of your object.

But if you are calling a method, there is no ambiguity. The interpreter knows you're not referring to a local variable called foo because there is none. So for example with:

self.foo = foo + 1

you need to use "self" for the assignment, but not to read the current value.

Answer 4

I'd like to add one pitfall that you might experience with the "before_validations" solutions above. Take this example:

u = User.new(name: " lala")
u.name # => " lala"
u.save
u.name # => "lala"

This means you have an inconsistent behavior based on whether your object was saved or not. If you want to address this, I suggest another solution to your problem: overwriting the corresponding setter methods.

class User < ActiveRecord::Base
  def name=(name)
    write_attribute(:name, name.try(:strip))
  end
end

I also like this approach because it does not force you to enable stripping for all attributes that support it - unlike the attribute_names.each mentioned earlier. Also, no callbacks required.

Answer 5

Instead we can write a better method more generic regardless whatever may be the type of attributes with the object(might have 3 string type fields, few booleans, few numeric)

before_validation :strip_input_fields


def strip_input_fields
  self.attributes.each do |key, value|
    self[key] = value.strip if value.respond_to?("strip")
  end
end

Hope that will helps someone!

Answer 6

StripAttributes Gem

I used strip_attributes. It's really awesome and easy to implement.

Default Behavior

class DrunkPokerPlayer < ActiveRecord::Base
  strip_attributes
end

By default, this will only strip the leading and trailing whitespaces and will act on all attributes of the model. This is ideal because it's not destructive and doesn't require you to specify which attributes need to be striped.

Using except

# all attributes will be stripped except :boxers
class SoberPokerPlayer < ActiveRecord::Base
  strip_attributes :except => :boxers
end

Using only

# only :shoe, :sock, and :glove attributes will be stripped
class ConservativePokerPlayer < ActiveRecord::Base
  strip_attributes :only => [:shoe, :sock, :glove]
end

Using allow_empty

# Empty attributes will not be converted to nil
class BrokePokerPlayer < ActiveRecord::Base
  strip_attributes :allow_empty => true
end

Using collapse_spaces

# Sequential spaces in attributes will be collapsed to one space
class EloquentPokerPlayer < ActiveRecord::Base
  strip_attributes :collapse_spaces => true
end

Using regex

class User < ActiveRecord::Base
  # Strip off characters defined by RegEx
  strip_attributes :only => [:first_name, :last_name], :regex => /[^[:alpha:]\s]/
  # Strip off non-integers
  strip_attributes :only => [:phone], :regex => /[^0-9]/
end

Answer 7

Starting with Ruby 2.3.0 you can use the Safe Navigation Operator(&.)

before_validation :strip_whitespace

def strip_whitespace
  self.name&.strip!
  self.email&.strip!
  self.nick&.strip!
end

GEMS:
https://github.com/rmm5t/strip_attributes/
https://github.com/holli/auto_strip_attributes/

Answer 8

I like Karl's answer, but is there a way to do it without referencing each of the attributes by name? That is, is there a way to just run through the model attributes and call strip on each one (if it responds to that method)?

This would be desirable so I don't have to update the remove_whitespace method whenever I change the model.

UPDATE

I see that Karl implied that you might want to do this sort of thing. I didn't immediately know how it could be done, but here's something that works for me as described above. There' probably a better way to do it, but this works:

def clean_data
  # trim whitespace from beginning and end of string attributes
  attribute_names().each do |name|
  if self.send(name.to_sym).respond_to?(:strip)
    self.send("#{name}=".to_sym, self.send(name).strip)
  end
end

end

Answer 9

If you have access to ActiveSupport, use squish instead of strip.

http://api.rubyonrails.org/classes/String.html#method-i-squish

Answer 10

Ruby on Rails 7.1 introduced ActiveRecord::Base::normalizes which allows defining data sanitization callbacks like this:

normalizes :name, :email, :nick, with: ->(value) { value.strip }

This method is able to handle nil values and will per default not call the normalization lambda when the value is nil. That means no extra check for nil required.

When the normalization should be called for nil values too, then you can set apply_to_nil: true, to do stuff like this

normalize :name, apply_to_nil: true, with: ->(name) { name&.strip || 'NN' }

which would normalize the name attribute by stripping whitespace if a name is set, otherwise it would set the name to the string "NN"

Answer 11

Overriding the attribute write methods is another good way. For example:

class MyModel
  def email=(value)
    super(value.try(:strip))
  end
end

Then any part of the application that sets the value will have it stripped, including assign_attributes and so on.

Answer 12

Here's an alternative approach, if you are mostly concerned with users mis-entering data in your front-end forms...

# app/assets/javascripts/trim.inputs.js.coffee
$(document).on "change", "input", ->
  $(this).val $(this).val().trim()

Then include the file in your application.js if you aren't already including the whole tree.

This will ensure that every input gets leading & trailing whitespace removed before it is submitted to be saved by Rails. It's bound on document, and delegated to inputs, so any inputs added to the page later will be processed as well.

Pros:

• Does not require listing individual attributes by name
• Does not require any metaprogramming
• Does not require external library dependencies

Cons:

• Data submitted any other way than the forms (eg, via API) will not be trimmed
• Does not have advanced features like squish (but you could add that yourself)
• As mentioned in comments, does not work if JS is disabled (but who codes for that?)

Answer 13

Since I can't comment yet, I'll have to ask here: which method is giving the ArgumentError? strip, or responds_to?

Also, .strip removes only leading and trailing whitespace. If you want "Harry Potter" with two spaces to not be accepted, you would either have to use a regex or, more simply, you could call .split, which removes spaces, and re-concatenate the string with a single space.

As far as if stripping is a good idea, I don't see a problem when it is just the leading/trailing whitespace. If there are multiple spaces in between words though, I would notify the user instead of automatically removing the extra spaces and giving the user a login that is not what they submitted.

Answer 14

Another gem option is attribute_normalizer:

# By default it will strip leading and trailing whitespace
# and set to nil if blank.
normalize_attributes :author, :publisher

:strip Will strip leading and trailing whitespace.

normalize_attribute  :author, :with => :strip

Answer 15

A better alternative is to overwrite the setter method and use value.squish. Its cleaner and you don't have to use before_validation:

class User < ActiveRecord::Base
  def name=(value)
    super(value.squish)
  end  
end

Answer 16

Rails 7.1 introduced ActiveRecord::Base::normalizes

class User < ApplicationRecord
  normalizes :email, with: ->(email) { email.strip.downcase }
end

It invokes before validation

It is safe because doesn't (but optionally can) apply to nil

User.create(email: " [email protected] \n")
# => #<User email: "[email protected]">

User.find_by(email: "\[email protected] \t")
# => #<User email: "[email protected]">

If you have legacy non-normalized record, you can normalize it with ActiveRecord::Base#normalize_attribute

user.email # => "[email protected]  "
user.normalize_attribute(:email)
user.email # => "[email protected]"
user.save