Ruby gems — what, why and how

by on December 14, 2010

Ruby gems — what, why and how

This guest post is by Gonçalo Silva, who is a full-time Ruby on Rails developer at escolinhas.pt and has participated in the Ruby Summer of Code 2010. He loves and contributes to many open-source projects, being a fan of Linux, Ruby and Android. He likes to call himself a hacker, but that’s just an excuse for being in front of the computer all the time. Oh, and he tweets at @goncalossilva.

What is a gem

Gonçalo Silva At its most basic form, a Ruby gem is a package. It has the necessary files and information for being installed on the system. Quoting RubyGems: «A gem is a packaged Ruby application or library. It has a name (e.g. rake) and a version (e.g. 0.4.16)».

Being very powerful, gems are of great importance in the Rubyland. They can easily be used to extend or change functionality within Ruby applications.

Structure

Every gem is different, but most follow a basic structure:

gem/
|-- lib/
|   |-- gem.rb
|-- test/
|-- README
|-- Rakefile
|-- gem.gemspec

Your gem’s code is located under lib/ which typically holds a Ruby file with the name of the gem. You can choose to have all the magic happening in this file, but you can also use it to load some other Ruby files also located under lib/, typically inside a folder with the gem’s name. Confused? Have a look:

your_gem/
|-- lib/
|   |-- your_gem.rb
|   |-- your_gem/
|   |   |-- source1.rb
|   |   |-- source2.rb
|-- ...

The test folder’s name is not necessarily named test/. When you’re working with RSpec, for instance, its name is usually spec/. As you’ve probably guessed, this folder holds tests for your gem.

After the README file, which hopefully doesn’t need any introduction, comes the Rakefile. In a gem’s context, the Rakefile is extremely useful. It can hold various tasks to help building, testing and debugging your gem, among all other things that you might find useful.

The gemspec—as the name implies—contains your gem’s specification by defining several attributes. An example gemspec file could be:

Gem::Specification.new do |s|
  s.name              = "gem"
  s.version           = "0.0.1"
  s.platform          = Gem::Platform::RUBY
  s.authors           = ["Gonçalo Silva"]
  s.email             = ["goncalossilva@gmail.com"]
  s.homepage          = "http://github.com/goncalossilva/gem_template"
  s.summary           = "Sample gem"
  s.description       = "A gem template"
  s.rubyforge_project = s.name

  s.required_rubygems_version = ">= 1.3.6"

  # If you have runtime dependencies, add them here
  # s.add_runtime_dependency "other", "~> 1.2"

  # If you have development dependencies, add them here
  # s.add_development_dependency "another", "= 0.9"

  # The list of files to be contained in the gem
  s.files         = `git ls-files`.split("\n")
  # s.executables   = `git ls-files`.split("\n").map{|f| f =~ /^bin\/(.*)/ ? $1 : nil}.compact
  # s.extensions    = `git ls-files ext/extconf.rb`.split("\n")

  s.require_path = 'lib'

  # For C extensions
  # s.extensions = "ext/extconf.rb"
end

Some attributes like the name, version, platform and summary are required others are optional. If you use git with your project, you can use the nifty trick shown above to list the project’s files, executables and extensions. If you don’t, you can simply fall back to using pure Ruby code like:

s.files = Dir["{lib}/**/*.rb", "{lib}/**/*.rake", "{lib}/**/*.yml", "LICENSE", "*.md"]

The dummy gem is very simple. Because of this, it perfectly illustrates some of the ideas explained above. Some interesting bits are shown below:

dummy/
|-- lib/
|   |-- dummy/
|   |   |-- core_ext/
|   |   |   |-- array.rb
|   |   |   |-- string.rb
|   |   |-- address.rb
|   |   |-- company.rb
|   |   |-- ...
|   |-- dummy.rb

This gem is organized into several source files inside lib/. The dummy.rb implements the top-level module and loads all functionality from the Ruby files inside lib/dummy/. It also includes some core extensions, namely to the Array and String classes (which are part of Ruby’s core).

RubyGems

Finally, RubyGems. It is a package manager which became part of the standard library in Ruby 1.9. It allows developers to search, install and build gems, among other features. All of this is done by using the gem command-line utility. You can find its website at rubygems.org.

Why is this useful

Gems are very useful for not reinventing the wheel and avoiding duplication. That’s basically it. Many Ruby developers create and publish awesome gems which address specific requirements, solve specific problems or add specific functionality. Anyone who comes across similar requirements or problems can use them and eventually improve them. That’s the joint awesomeness of Ruby’s strong open-source foundation and extreme flexibility. Anyway, you’re reading this article… so you’ve probably understood the concept and grasped its usefulness long before reading this paragraph.

How to make your own

Making your own gem is nothing more than packaging your library or application according to the structure stated above. Put all your code under lib/, all your tests under test/ or spec/, your gem specification under your_gem.gemspec and you’re good to go. Of course, a few other files might come in handy, namely a Rakefile, a README and a LICENSE. A CHANGELOG, sometimes, might be useful as well.

Ruby Idioms

When developing a gem, you are probably creating, extending or overriding functionality. You might want people to include your module in their classes, or perhaps you just want to extend a given class with your module—it’s your choice. What you shouldn’t really do, however, is reinventing Ruby’s module system. There is an excellent blog post on this which can help if you—like many gem authors I’ve seen—start overriding include to behave like extend. It’s very important to understand the difference between the two and, fortunately, there are great resources about this out there.

Developing with Bundler

Using Bundler to manage your gem’s dependencies is also pretty easy. Just create a Gemfile and add:

gemspec

After this, fire Bundler:

bundle install

And yes, you got it right. After adding gemspec to your Gemfile, Bundler can scan your gemspec, find your runtime and development dependencies and install them for you.

While not being mandatory, I strongly recommend you to consider using Bundler to manage your gem’s dependencies. If used correctly, it can probably be a time saver.

Testing

When it comes to testing, you’ve got plenty of good options. Some people rely on test-unit (or minitest in 1.9), others prefer RSpec. It’s really up to you. The only bad choice you can possibly make is opting to not testing your gems at all.

Once again, I’m going to use dummy’s simplicity to explain this a bit further. All tests were built on test-unit and are organized as follows:

dummy/
|-- test/
|   |-- address_test.rb
|   |-- company_test.rb
|   |-- ...
|   |-- test_helper.rb

As you’ve seen, tests are structured similarly to dummy itself. The test_helper is in charge of loading the necessary libraries and setting up any variables or methods used across most (if not all) tests. All tests are organized into files which target specific functionality in dummy. The tests contained in address_test.rb run against address.rb and so on.

Publishing

After everything is coded and tested, all you got left to do is packaging and publishing. The previously mentioned gem utility makes it all very simple. Just run gem build your_gem.gemspec and you should see something along these lines:

Successfully built RubyGem
Name: your_gem
Version: 0.0.1
File: your_gem-0.0.1.gem

Pushing your gem to RubyGems is as easy as it is to build it. Just gem push your_gem-0.0.1.gem and soon it’ll be published. Be aware that the first time you issue this command you’ll be prompted to login with a RubyGems.org account.

Concerning this, I like keeping these simple tasks in my Rakefile:

desc "Validate the gemspec"
task :gemspec do
  gemspec.validate
end

desc "Build gem locally"
task :build => :gemspec do
  system "gem build #{gemspec.name}.gemspec"
  FileUtils.mkdir_p "pkg"
  FileUtils.mv "#{gemspec.name}-#{gemspec.version}.gem", "pkg"
end

desc "Install gem locally"
task :install => :build do
  system "gem install pkg/#{gemspec.name}-#{gemspec.version}"
end

These help me build and install my gems. They also aid at keeping all packages in the pkg/ folder, which is useful for keeping the root directory clean and tidy.

Gems for building gems

There are a few gems which were specifically created to help developers build their own gems. Among them are the renowned jeweler, hoe and echoe. I can’t go into detail in any of these since I’ve never really used them – I started building my gem skeleton from scratch right at the beginning. However, some of these tools are very helpful so you should really take a look and see if any fits your needs.

Gem template

As I mentioned, I’ve been using a gem skeleton for some time now, which you can find at GitHub. Every gem I’ve built started with that template, which I kept trying to improve over time.

You can start your gems from scratch, but that’s just nonsense. You should create your own skeleton, use one made by someone else or use a third-party gem to help creating your gem.

Legen—wait for it—dary

Ruby gems are filled with awesomeness. Hop in and start making your own!

Feel free to ask questions and give feedback in the comments section of this post. Thanks and Good Luck!

Technorati Tags: , ,

Posted by Gonçalo Silva

Follow me on Twitter to communicate and stay connected

{ 27 comments… read them below or add one }

Samnang Chhun December 14, 2010 at 11:30 am

Great post! Please fix two invalid links in Ruby Idioms session.

Reply

Satish Talim December 14, 2010 at 12:50 pm

Thanks Samnang for pointing that out. The links are now fixed.

Reply

José Mota December 14, 2010 at 2:34 pm

Finally a Portuguese Rubyist out there. And finally something I could read on creating a gem.

This will do wonders when I start reading José Valim’s eBook. Parabéns, Gonçalo!

Reply

Gonçalo Silva December 15, 2010 at 7:47 pm

Thanks, José :-)

Let me know what you think about the article when you read it throughly!

Reply

postmodern December 14, 2010 at 3:21 pm

Great post, but maybe a tad too much gemspec-boilerplate code :)

I wrote many gems first with Hoe, then with Jeweler. These days, I prefer using Ore (http://github.com/ruby-ore/ore#readme) with ore-tasks (http://github.com/ruby-ore/ore-tasks#readme) to build and release RubyGems. Ore organizes all of the project information into a single YAML file (ex: https://github.com/postmodern/pullr/blob/master/gemspec.yml#readme), and can easily generate new projects (“$ mine my_project –bundler –ore-tasks”).

Reply

Glenn Gillen December 14, 2010 at 3:40 pm

Great post. The only thing I’d challenge is under the “gems that build gems” section:

“some of these tools are very helpful so you should really take a look ”

Don’t! Building a gem skeleton is so simple (as you’ve detailed) that using these tools just adds additional dependencies and complexities for virtually no gain. If creating the base spec file from scratch is too difficult, use gem-this (https://github.com/lazyatom/gem-this).

Reply

Ryan Davis December 14, 2010 at 4:38 pm

Virtually no gain? I challenge you to prove that. I’m up to 88 gems and 434 releases. Poking around I see that you’re at authoring 1 gem released once 3 years ago and contributing to scrubyt that’s at 11 releases. Two other projects on rubyforge look like stillbirths. I may be missing some, but it certainly looks like you’ve hardly had to maintain project version skew across your projects. You’ve hardly had to deal with propagating a new rake task across them, let alone merging different rake tasks across them. Hell, you’ve hardly had to deal with releasing at all…

You can hardly say “virtually no gain” with any level of confidence as you don’t have enough experience to back that up. These tools exist for good reason. Maybe YOU don’t have a good reason to use them, but many many others obviously do.

Reply

postmodern December 14, 2010 at 4:56 pm

@RyanDavis Not sure how many releases per year I’m averaging, but now I’m up to 60 gems to maintain, so I guess I’m catching up :).

@everyone Hoe and Jeweler (and now Ore) have definitely helped me. Ignoring these tools based only on comments left on blog-posts would be a mistake, at least give these tools a try first. Maybe you might like them as well.

Reply

Gonçalo Silva December 14, 2010 at 7:36 pm

«Ignoring these tools based only on comments left on blog-posts would be a mistake, at least give these tools a try first. Maybe you might like them as well.»

That is exactly what I encourage people to do. I tried some of them at the beginning but ended up not feeling comfortable about the additional dependencies, so I created my own template. But that’s *my* opinion… there are a lot of great Rubyists out there using a gem to help them out.

Anyway, it’s really up to the developer.

Reply

postmodern December 15, 2010 at 8:57 pm

Have you tried using Ore? If so, could you send me some user feedback. Thanks.

Glenn Gillen December 14, 2010 at 5:11 pm

If you want to create a gem, writing a gemspec from scratch is as Goçalo pointed out so simple I don’t need to go install a gem just to create it. I also don’t need a rake task to update the versions, it’s easier to update the file than remember what the command is. Releasing is part of gem/gemcutter, likewise I don’t need *another* command to do the same thing. When I release something, I prefer take the time to craft a detailed announcement and explain any changed rather than have it automatically generated for a history.txt.

I wouldn’t be so quick to confuse to the high usage of these tools as an endorsement of the value they’ve added to the process for each developer. Most ruby devs I’ve worked with have assumed they exist because creating a gem is hard, and so just go with the collective group-think without investigating an approach like Goçalo’s.

Kudos for hoe, it was a required install on my machine for a long time whether I wanted it or not.

Reply

Ryan Davis December 15, 2010 at 4:05 am

Glenn,

“Virtually no gain” were your words and I challenged you to prove that. You haven’t done that in my opinion. All of your refutation was based on “When I release something” and as I pointed out, you have (virtually) nothing to show for that. Since you didn’t bother to correct any of my points about your very limited numbers in this area, I’m forced to assume that you concede to my summary.

Not only that, but most of the points you try to make are either wrong or invalid. You no more “need” to use any of the many tools these libraries provide than you “need” to use ruby at all… You might as well make the argument that you should be writing everything in assembler, but even that is probably too much of a crutch of you.

I wouldn’t be so quick to confuse your limited anecdotal evidence with actual data. I’ll let real numbers do that job.

Reply

Glenn Gillen December 15, 2010 at 9:48 pm

Let me rephrase as in hindsight I was a little broad in my painting and more than a little glib. Jeweler creates a gemspec template and excludes the files in .gitignore for your glob/manifest. But you have to learn a new DSL/syntax and include a new dependency to be able to use it. Given the syntax is *almost* identical to the gemspec syntax, I’ll call that “virtually no gain”. Hoe solves a lot of problems with managing workflow, particularly across multiple projects. So my general wariness in suggesting it as a first port of call for someone writing their first gem is that it adds additional cognitive load while solving a problem they don’t have yet. It’s wrong to assume that is never going to be a problem for people, and for that I apologise.

That being said, I still have my reservations:

* As a consumer of a gem, why should I be forced to install a dependency that is only relevant for releasing a gem?
* As a developer who wants to create and test a patch, why should I be forced to install a dependency that is only relevant for releasing a gem?
* As a developer who wants to privately build a fork of your gem, why should I have to install a dependency when my system already has everything it needs?

Releasing is *your* problem, but you’ve the forced the solution upon every user who wishes to use your gem(s).

Contrary to your assumptions, very few of the gems I’ve written have been pushed to rubygems but that doesn’t make the management of them for my clients any more trivial. I pointed out gem-this (https://github.com/lazyatom/gem-this) as it solves many of the same problems by including your own custom rake tasks. Failing that, there is sake if you need system wide rake tasks. git submodules, svnexternals, symlinks, a separate gem installing appropriate commands in /bin, extending rubygems by registering a new gem command, a series of shell scripts, etc…. there are a multitude of ways of solving the problem in a reusable fashion that has zero impact on end users of the gem.

“maintain project version skew across your projects… deal with propagating a new rake task across them”

This is a system-wide problem, not an application one. To me it seems like it should be solved at a system level. Extract the hoe tasks out to a higher level (or extend sow?) and I’d be happy.

postmodern December 15, 2010 at 9:03 pm

Ore does not modify any bit of the projects source code, it still expects the developer to update the version and the Changelog. You can also build and release Ore projects using `gem build` and `gem push`; since Ore generates a dummy *.gempsec file which loads the gempsec.yml data.

Reply

Robert Wahler December 14, 2010 at 6:31 pm

I agree that rolling your own template is a reasonable way to maintain gems. You can easily jump start new gems. It is also gives you more control over maintaining an existing fleet of gems. I use my own template for common gem functionality and use Git to pull updates from the template into existing production gems. My template is located here: https://github.com/robertwahler/basic_gem

Reply

Millisami December 14, 2010 at 7:48 pm

Good read!
But this is also a great place to build a gem with much ease and functionality like command-line gems too. https://github.com/radar/guides/blob/master/gem-development.md

Reply

Gonçalo Silva December 15, 2010 at 7:45 pm

That looks really good!

Reply

Peter Schröder December 15, 2010 at 12:01 am

WOW why did I wrote my blogpost then?

http://blog.nofail.de/2010/11/gem-best-practice/

Reply

Gonçalo Silva December 15, 2010 at 7:50 pm

Nice article!

More information is never too much. When I started making gems I felt that there wasn’t enough information. Most guides were outdated back then. Great thing it’s not the case anymore.

Reply

Ryan Davis December 15, 2010 at 3:38 am

Gonçalo,

I should add that this was a great post. It would be greatly welcomed if you wanted to contribute towards the rubygems manuals @ http://docs.rubygems.org/

Reply

Gonçalo Silva December 15, 2010 at 7:53 pm

Thanks! Is there anything in particular that is missing or lacking attention? I think that the guide Millisami’s linked is filled with useful information which might come in handy to improve the rubygems manual.

Reply

Ryan Davis December 15, 2010 at 9:18 pm

Your information is better imo, as you focus on rubygems. Bundler != rubygems. If you removed your small bundler section, I could see your page going in the rubygems doco as-is.

Reply

Gonçalo Silva December 16, 2010 at 6:06 pm

Nice! The Bundler section can be removed, it doesn’t affect the flow of the article.

Reply

Martin Gross December 15, 2010 at 5:49 pm

I just learnt yesterday that bundler is a great tool to create your own gems. Most rubyists will have it installed anyway.

For example if you want to generate a gem project directory with all required files, just type:
bundle gem your_gem_name

Unfortunately I couldn’t find more information at the Bundler website (http://gembundler.com/) but only here: http://ruby.about.com/od/advancedruby/ss/Creating-And-Distributing-Gems-With-Bundler.htm

Reply

Gonçalo Silva December 15, 2010 at 7:44 pm

Yes, Bundler is a great tool to create gems as well… it sets everything up! But just like every other tool, the developer can choose not to depend on it. Having a gem template is a valid alternative to using Bundler or Hoe.

Reply

Perry January 12, 2011 at 4:35 am
Sven Fuchs February 21, 2011 at 2:57 pm

Plug: for scaffolding/building/releasing/tagging gems also check out https://github.com/svenfuchs/gem-release

Reply

Leave a Comment

{ 86 trackbacks }

Previous post:

Next post: