Send to KindleRuby gems — what, why and how
This guest post is by Gonçalo Silva, who is a full-time Ruby on Rails developer at escolinhas.pt and has participated in the Ruby Summer of Code 2010. He loves and contributes to many open-source projects, being a fan of Linux, Ruby and Android. He likes to call himself a hacker, but that’s just an excuse for being in front of the computer all the time. Oh, and he tweets at @goncalossilva.
What is a gem
At its most basic form, a Ruby gem is a package. It has the necessary files and information for being installed on the system. Quoting RubyGems: «A gem is a packaged Ruby application or library. It has a name (e.g. rake) and a version (e.g. 0.4.16)».
Being very powerful, gems are of great importance in the Rubyland. They can easily be used to extend or change functionality within Ruby applications.
Structure
Every gem is different, but most follow a basic structure:
gem/ |-- lib/ | |-- gem.rb |-- test/ |-- README |-- Rakefile |-- gem.gemspec
Your gem’s code is located under lib/ which typically holds a Ruby file with the name of the gem. You can choose to have all the magic happening in this file, but you can also use it to load some other Ruby files also located under lib/, typically inside a folder with the gem’s name. Confused? Have a look:
your_gem/ |-- lib/ | |-- your_gem.rb | |-- your_gem/ | | |-- source1.rb | | |-- source2.rb |-- ...
The test folder’s name is not necessarily named test/. When you’re working with RSpec, for instance, its name is usually spec/. As you’ve probably guessed, this folder holds tests for your gem.
After the README file, which hopefully doesn’t need any introduction, comes the Rakefile. In a gem’s context, the Rakefile is extremely useful. It can hold various tasks to help building, testing and debugging your gem, among all other things that you might find useful.
The gemspec—as the name implies—contains your gem’s specification by defining several attributes. An example gemspec file could be:
Gem::Specification.new do |s|
s.name = "gem"
s.version = "0.0.1"
s.platform = Gem::Platform::RUBY
s.authors = ["Gonçalo Silva"]
s.email = ["goncalossilva@gmail.com"]
s.homepage = "http://github.com/goncalossilva/gem_template"
s.summary = "Sample gem"
s.description = "A gem template"
s.rubyforge_project = s.name
s.required_rubygems_version = ">= 1.3.6"
# If you have runtime dependencies, add them here
# s.add_runtime_dependency "other", "~> 1.2"
# If you have development dependencies, add them here
# s.add_development_dependency "another", "= 0.9"
# The list of files to be contained in the gem
s.files = `git ls-files`.split("\n")
# s.executables = `git ls-files`.split("\n").map{|f| f =~ /^bin\/(.*)/ ? $1 : nil}.compact
# s.extensions = `git ls-files ext/extconf.rb`.split("\n")
s.require_path = 'lib'
# For C extensions
# s.extensions = "ext/extconf.rb"
end
Some attributes like the name, version, platform and summary are required others are optional. If you use git with your project, you can use the nifty trick shown above to list the project’s files, executables and extensions. If you don’t, you can simply fall back to using pure Ruby code like:
s.files = Dir["{lib}/**/*.rb", "{lib}/**/*.rake", "{lib}/**/*.yml", "LICENSE", "*.md"]
The dummy gem is very simple. Because of this, it perfectly illustrates some of the ideas explained above. Some interesting bits are shown below:
dummy/ |-- lib/ | |-- dummy/ | | |-- core_ext/ | | | |-- array.rb | | | |-- string.rb | | |-- address.rb | | |-- company.rb | | |-- ... | |-- dummy.rb
This gem is organized into several source files inside lib/. The dummy.rb implements the top-level module and loads all functionality from the Ruby files inside lib/dummy/. It also includes some core extensions, namely to the Array and String classes (which are part of Ruby’s core).
RubyGems
Finally, RubyGems. It is a package manager which became part of the standard library in Ruby 1.9. It allows developers to search, install and build gems, among other features. All of this is done by using the gem command-line utility. You can find its website at rubygems.org.
Why is this useful
Gems are very useful for not reinventing the wheel and avoiding duplication. That’s basically it. Many Ruby developers create and publish awesome gems which address specific requirements, solve specific problems or add specific functionality. Anyone who comes across similar requirements or problems can use them and eventually improve them. That’s the joint awesomeness of Ruby’s strong open-source foundation and extreme flexibility. Anyway, you’re reading this article… so you’ve probably understood the concept and grasped its usefulness long before reading this paragraph.
How to make your own
Making your own gem is nothing more than packaging your library or application according to the structure stated above. Put all your code under lib/, all your tests under test/ or spec/, your gem specification under your_gem.gemspec and you’re good to go. Of course, a few other files might come in handy, namely a Rakefile, a README and a LICENSE. A CHANGELOG, sometimes, might be useful as well.
Ruby Idioms
When developing a gem, you are probably creating, extending or overriding functionality. You might want people to include your module in their classes, or perhaps you just want to extend a given class with your module—it’s your choice. What you shouldn’t really do, however, is reinventing Ruby’s module system. There is an excellent blog post on this which can help if you—like many gem authors I’ve seen—start overriding include to behave like extend. It’s very important to understand the difference between the two and, fortunately, there are great resources about this out there.
Developing with Bundler
Using Bundler to manage your gem’s dependencies is also pretty easy. Just create a Gemfile and add:
gemspec
After this, fire Bundler:
bundle install
And yes, you got it right. After adding gemspec to your Gemfile, Bundler can scan your gemspec, find your runtime and development dependencies and install them for you.
While not being mandatory, I strongly recommend you to consider using Bundler to manage your gem’s dependencies. If used correctly, it can probably be a time saver.
Testing
When it comes to testing, you’ve got plenty of good options. Some people rely on test-unit (or minitest in 1.9), others prefer RSpec. It’s really up to you. The only bad choice you can possibly make is opting to not testing your gems at all.
Once again, I’m going to use dummy’s simplicity to explain this a bit further. All tests were built on test-unit and are organized as follows:
dummy/ |-- test/ | |-- address_test.rb | |-- company_test.rb | |-- ... | |-- test_helper.rb
As you’ve seen, tests are structured similarly to dummy itself. The test_helper is in charge of loading the necessary libraries and setting up any variables or methods used across most (if not all) tests. All tests are organized into files which target specific functionality in dummy. The tests contained in address_test.rb run against address.rb and so on.
Publishing
After everything is coded and tested, all you got left to do is packaging and publishing. The previously mentioned gem utility makes it all very simple. Just run gem build your_gem.gemspec and you should see something along these lines:
Successfully built RubyGem Name: your_gem Version: 0.0.1 File: your_gem-0.0.1.gem
Pushing your gem to RubyGems is as easy as it is to build it. Just gem push your_gem-0.0.1.gem and soon it’ll be published. Be aware that the first time you issue this command you’ll be prompted to login with a RubyGems.org account.
Concerning this, I like keeping these simple tasks in my Rakefile:
desc "Validate the gemspec"
task :gemspec do
gemspec.validate
end
desc "Build gem locally"
task :build => :gemspec do
system "gem build #{gemspec.name}.gemspec"
FileUtils.mkdir_p "pkg"
FileUtils.mv "#{gemspec.name}-#{gemspec.version}.gem", "pkg"
end
desc "Install gem locally"
task :install => :build do
system "gem install pkg/#{gemspec.name}-#{gemspec.version}"
end
These help me build and install my gems. They also aid at keeping all packages in the pkg/ folder, which is useful for keeping the root directory clean and tidy.
Gems for building gems
There are a few gems which were specifically created to help developers build their own gems. Among them are the renowned jeweler, hoe and echoe. I can’t go into detail in any of these since I’ve never really used them – I started building my gem skeleton from scratch right at the beginning. However, some of these tools are very helpful so you should really take a look and see if any fits your needs.
Gem template
As I mentioned, I’ve been using a gem skeleton for some time now, which you can find at GitHub. Every gem I’ve built started with that template, which I kept trying to improve over time.
You can start your gems from scratch, but that’s just nonsense. You should create your own skeleton, use one made by someone else or use a third-party gem to help creating your gem.
Legen—wait for it—dary
Ruby gems are filled with awesomeness. Hop in and start making your own!
Feel free to ask questions and give feedback in the comments section of this post. Thanks and Good Luck!
Technorati Tags: Programming, Ruby programming, Ruby Gems
Posted by Gonçalo Silva
{ 27 comments… read them below or add one }
← Previous Comments
Great post! Please fix two invalid links in Ruby Idioms session.
Thanks Samnang for pointing that out. The links are now fixed.
Finally a Portuguese Rubyist out there. And finally something I could read on creating a gem.
This will do wonders when I start reading José Valim’s eBook. Parabéns, Gonçalo!
Thanks, José
Let me know what you think about the article when you read it throughly!
Great post, but maybe a tad too much gemspec-boilerplate code
I wrote many gems first with Hoe, then with Jeweler. These days, I prefer using Ore (http://github.com/ruby-ore/ore#readme) with ore-tasks (http://github.com/ruby-ore/ore-tasks#readme) to build and release RubyGems. Ore organizes all of the project information into a single YAML file (ex: https://github.com/postmodern/pullr/blob/master/gemspec.yml#readme), and can easily generate new projects (“$ mine my_project –bundler –ore-tasks”).
Great post. The only thing I’d challenge is under the “gems that build gems” section:
“some of these tools are very helpful so you should really take a look ”
Don’t! Building a gem skeleton is so simple (as you’ve detailed) that using these tools just adds additional dependencies and complexities for virtually no gain. If creating the base spec file from scratch is too difficult, use gem-this (https://github.com/lazyatom/gem-this).
Virtually no gain? I challenge you to prove that. I’m up to 88 gems and 434 releases. Poking around I see that you’re at authoring 1 gem released once 3 years ago and contributing to scrubyt that’s at 11 releases. Two other projects on rubyforge look like stillbirths. I may be missing some, but it certainly looks like you’ve hardly had to maintain project version skew across your projects. You’ve hardly had to deal with propagating a new rake task across them, let alone merging different rake tasks across them. Hell, you’ve hardly had to deal with releasing at all…
You can hardly say “virtually no gain” with any level of confidence as you don’t have enough experience to back that up. These tools exist for good reason. Maybe YOU don’t have a good reason to use them, but many many others obviously do.
@RyanDavis Not sure how many releases per year I’m averaging, but now I’m up to 60 gems to maintain, so I guess I’m catching up
.
@everyone Hoe and Jeweler (and now Ore) have definitely helped me. Ignoring these tools based only on comments left on blog-posts would be a mistake, at least give these tools a try first. Maybe you might like them as well.
«Ignoring these tools based only on comments left on blog-posts would be a mistake, at least give these tools a try first. Maybe you might like them as well.»
That is exactly what I encourage people to do. I tried some of them at the beginning but ended up not feeling comfortable about the additional dependencies, so I created my own template. But that’s *my* opinion… there are a lot of great Rubyists out there using a gem to help them out.
Anyway, it’s really up to the developer.
Have you tried using Ore? If so, could you send me some user feedback. Thanks.
If you want to create a gem, writing a gemspec from scratch is as Goçalo pointed out so simple I don’t need to go install a gem just to create it. I also don’t need a rake task to update the versions, it’s easier to update the file than remember what the command is. Releasing is part of gem/gemcutter, likewise I don’t need *another* command to do the same thing. When I release something, I prefer take the time to craft a detailed announcement and explain any changed rather than have it automatically generated for a history.txt.
I wouldn’t be so quick to confuse to the high usage of these tools as an endorsement of the value they’ve added to the process for each developer. Most ruby devs I’ve worked with have assumed they exist because creating a gem is hard, and so just go with the collective group-think without investigating an approach like Goçalo’s.
Kudos for hoe, it was a required install on my machine for a long time whether I wanted it or not.
Glenn,
“Virtually no gain” were your words and I challenged you to prove that. You haven’t done that in my opinion. All of your refutation was based on “When I release something” and as I pointed out, you have (virtually) nothing to show for that. Since you didn’t bother to correct any of my points about your very limited numbers in this area, I’m forced to assume that you concede to my summary.
Not only that, but most of the points you try to make are either wrong or invalid. You no more “need” to use any of the many tools these libraries provide than you “need” to use ruby at all… You might as well make the argument that you should be writing everything in assembler, but even that is probably too much of a crutch of you.
I wouldn’t be so quick to confuse your limited anecdotal evidence with actual data. I’ll let real numbers do that job.
Let me rephrase as in hindsight I was a little broad in my painting and more than a little glib. Jeweler creates a gemspec template and excludes the files in .gitignore for your glob/manifest. But you have to learn a new DSL/syntax and include a new dependency to be able to use it. Given the syntax is *almost* identical to the gemspec syntax, I’ll call that “virtually no gain”. Hoe solves a lot of problems with managing workflow, particularly across multiple projects. So my general wariness in suggesting it as a first port of call for someone writing their first gem is that it adds additional cognitive load while solving a problem they don’t have yet. It’s wrong to assume that is never going to be a problem for people, and for that I apologise.
That being said, I still have my reservations:
* As a consumer of a gem, why should I be forced to install a dependency that is only relevant for releasing a gem?
* As a developer who wants to create and test a patch, why should I be forced to install a dependency that is only relevant for releasing a gem?
* As a developer who wants to privately build a fork of your gem, why should I have to install a dependency when my system already has everything it needs?
Releasing is *your* problem, but you’ve the forced the solution upon every user who wishes to use your gem(s).
Contrary to your assumptions, very few of the gems I’ve written have been pushed to rubygems but that doesn’t make the management of them for my clients any more trivial. I pointed out gem-this (https://github.com/lazyatom/gem-this) as it solves many of the same problems by including your own custom rake tasks. Failing that, there is sake if you need system wide rake tasks. git submodules, svnexternals, symlinks, a separate gem installing appropriate commands in /bin, extending rubygems by registering a new gem command, a series of shell scripts, etc…. there are a multitude of ways of solving the problem in a reusable fashion that has zero impact on end users of the gem.
“maintain project version skew across your projects… deal with propagating a new rake task across them”
This is a system-wide problem, not an application one. To me it seems like it should be solved at a system level. Extract the hoe tasks out to a higher level (or extend sow?) and I’d be happy.
Ore does not modify any bit of the projects source code, it still expects the developer to update the version and the Changelog. You can also build and release Ore projects using `gem build` and `gem push`; since Ore generates a dummy *.gempsec file which loads the gempsec.yml data.
I agree that rolling your own template is a reasonable way to maintain gems. You can easily jump start new gems. It is also gives you more control over maintaining an existing fleet of gems. I use my own template for common gem functionality and use Git to pull updates from the template into existing production gems. My template is located here: https://github.com/robertwahler/basic_gem
Good read!
But this is also a great place to build a gem with much ease and functionality like command-line gems too. https://github.com/radar/guides/blob/master/gem-development.md
That looks really good!
WOW why did I wrote my blogpost then?
http://blog.nofail.de/2010/11/gem-best-practice/
Nice article!
More information is never too much. When I started making gems I felt that there wasn’t enough information. Most guides were outdated back then. Great thing it’s not the case anymore.
Gonçalo,
I should add that this was a great post. It would be greatly welcomed if you wanted to contribute towards the rubygems manuals @ http://docs.rubygems.org/
Thanks! Is there anything in particular that is missing or lacking attention? I think that the guide Millisami’s linked is filled with useful information which might come in handy to improve the rubygems manual.
Your information is better imo, as you focus on rubygems. Bundler != rubygems. If you removed your small bundler section, I could see your page going in the rubygems doco as-is.
Nice! The Bundler section can be removed, it doesn’t affect the flow of the article.
I just learnt yesterday that bundler is a great tool to create your own gems. Most rubyists will have it installed anyway.
For example if you want to generate a gem project directory with all required files, just type:
bundle gem your_gem_name
Unfortunately I couldn’t find more information at the Bundler website (http://gembundler.com/) but only here: http://ruby.about.com/od/advancedruby/ss/Creating-And-Distributing-Gems-With-Bundler.htm
Yes, Bundler is a great tool to create gems as well… it sets everything up! But just like every other tool, the developer can choose not to depend on it. Having a gem template is a valid alternative to using Bundler or Hoe.
here is another one on bundler:
http://spin.atomicobject.com/2011/01/10/using-bundler-for-ruby-gem-development
Plug: for scaffolding/building/releasing/tagging gems also check out https://github.com/svenfuchs/gem-release
{ 84 trackbacks }