Introduction

Herder is a simple Ruby program for managing a small/personal cluster of computers. It provides an interactive Ruby scripting environment, along with a shell-like interaction mode. Using Herder, it is very easy to apply firewall policy/install a software package on multiple machines at the same time, monitor systems performance, or have on-demand access to a specific machine (without having set up a prior directory infrastructure).

I developed Herder out of frustration with all the big cluster toolkits that require hours to install, demand format-the-disk-access, and hide their operations behind a multitude of daemon processes. In contrast to the big toolkits, Herder:

can be run by a regular user, in as little as 3 commands (see below), within minutes
is simple to understand (you run each daemon knowing exactly what it does)
is immediate and interactive (via irb)

Herder should run on all platforms that support Ruby. Specifically, it has been tested on Linux/Bash. MacO X 10.4/Bash, and Windows XP/cmd.exe.

Currently, using Herder requires that you know (or willing to learn) a very small bit of the Ruby language. There is plan to provide a web "control panel" using AJAX in the future. There is also plan to develop "herdlet," which are small agent programs that automatically monitor and manage the machines (e.g. an agent that ensures that all machines have the right firewall settings, or one that keeps the JDK updated on all the machines).

Download

Herder 1.1.1

Updated: Herder 1.2.1 is available, with some fixes and a RC script for running at system startup. (note the documentation on this page was developed with version 1.1.1, which should be mostly compatible with 1.2.x series).

Quick Start

Here is an example of how to use Herder in its default setting. This section assumes that you are using Bash on Linux.

[peter@host herder]$ ruby herder.rb -server localhost:9000 > /dev/null &
[peter@host herder]$ ruby herder.rb -client localhost:9000 > /dev/null &
[peter@host herder]$ ruby herder.rb -console localhost:9000
irb(main):001:0> nodes = connect 'druby://localhost:9000'
irb(main):002:0> nodes.keys
=> ["localhost"]
irb(main):003:0> local = nodes["localhost"]
irb(main):004:0> local.enter 'echo Hello Herder!'
irb(main):005:0> print local.last_out
Hello Herder!
=> nil

You can also run "ruby herder.rb -help" for usage information.

Not So Fine Points

Here are some tidbits of information about running Herder.

Herder runs under whatever user started it, so it has the same permission as that user.
Security, in terms of authentication and encryption, is supported at the connection level, but there is no built-in multi-user support.
If you are using the clear protocol, use "druby://localhost:1234" to connect; if you are using SSL, then use "drbssl://localhost:1234"
If you want to evaluate a Ruby expression remotely, use evaluate(); if you want to emulate a shell-like environment, use enter()/input(), which are the same as entering a line of text or somet bit of text on the remote terminal.
evaluate() is synchronous and input()/enter() are asychronous.
If the process that you invoked from the shell needs to be ended by a EOF, issue the eof() call.
If the process that you invoked from the shell doesn't respond to command, you can issue break() to break Herder from it, and use enter() to enter a kill command against that process.

Detailed Example

Registry Server

First, you need to run a registry on a main server:

[peter@registry ~]$ ruby herder.rb -server localhost:1234
Registry is now running at: druby://localhost:1234

Client

Next, for each computer that you want to manage, run a client node:

[peter@node5 ~]$ ruby herder.rb -client registry.example.net:1234
Attached to: druby://registry.example.net:1234 as node5.example.net

[peter@node8 ~]$ ruby herder.rb -client registry.example.net:1234
Attached to: druby://registry.example.net:1234 as node8.example.net

Console

Once you have all the nodes set up, you can now access them using a console, which is a version of IRB with the SSL settings already preloaded (if you specified them on the command line).

First, let's connect and see what nodes are available:

[peter@laptop ~]$ ruby herder.rb -console
DRb configured and started
You can now create connection using IRB
Example: registry = DRbObject.new(nil, 'drbssl://example.com:1234')
irb(main):001:0> registry = DRbObject.new nil, 'druby://registry.example.net:1234'
irb(main):002:0> registry.keys
=> ["node5.example.net", "node8.example.net"]

Now, let's run the 'uptime' command on the two nodes.

irb(main):006:0> registry.each_pair do |name, node|
irb(main):007:1* node.enter 'uptime'
irb(main):008:1> end

Let's see what the commands printed out:

irb(main):013:0> registry.each_pair do |name, node|
irb(main):014:1*   puts '=== ' + name + ' ==='
irb(main):015:1>   puts node.last_out
irb(main):016:1> end
=== node5.example.net ===
 10:37:11 up 23 days, 34 min,  3 users,  load average: 0.00, 0.00, 0.00
=== node8.example.net ===
 10:37:11 up 22 days, 0 min,  2 users,  load average: 0.40, 0.11, 0.01

We can also run Ruby expression on the nodes directly:

irb(main):025:0> registry.each_pair do |name, node|
irb(main):026:1*   puts name + ": " + node.evaluate("RUBY_VERSION")
irb(main):027:1> end
node5.example.net: 1.8.4
node8.example.net: 1.8.2

That's it, you have now run commands on multiple nodes, all from a central console!

More Ways to Use

Given a node object (as above), there are a few Ruby methods you can call on it to access the node:

node.evaluate(code) - Return the result of a Ruby expression remotely.
node.enter(string) - Enter a string on the command line followed by a newline.
node.input(string) - Same as above, but without the automatic newline at the end.
node.out_len or node.err_len - The current sizes of the stdout or stderr history.
node.out(offset, len) or err(offset, len) - Get n characters from the stdout or stderr history, from the given offset.
node.last_out(n) or node.last_err(n) - Get the last n characters from the stdin or stderr streams.
node.eof - Send an EOF character to the remote stdin. This is useful if you want to close some commands (such as 'cat').
node.break - Use to detach the remote node shell from the program. This is useful if the program frooze and you need to break away from it. You should probably then do a node.enter 'kill -9 ' to kill the runaway process.

SSL

Since the default protocol talks over plain text and has no authentication, you probably want to use SSL to secure the communication. Herder supports SSL options for both encrypting and authenticating connections.

Before you begin, you need to get a hold of SSL certificates for the set up. If you are not sure how, I recommend QuickCert as the easiest way to generate the certificates. Just read the sample qc_config and make sure that you generate a server type certificate for your registry and as many client type certificates as the number of nodes you want to run Herder on.

Briefly, given a CA certificate, and host certificate, and a host private key, you can specify them at the end of the command line. They will be used to authenticate and encrypt connections.

First, run on the registry server:

[peter@registry ~]$ ruby herder.rb -server localhost:1234 ca_cert host_cert host_private_key
Registry is now running at: drbssl://localhost:1234

Next, attach the node using SSL:

[peter@node5 ~]$ ruby herder.rb -client registry.example.net:1234 ca_cert node5_cert node5_private_key
Attached to: drbssl://registry.example.net:1234 as node5.example.net

Don't forget that your console will also need to use SSL:

[peter@laptop ~]$ ruby herder.rb -console ca_cert console_cert console_private_key
...
irb(main):001:0> registry = DRbObject.new nil, 'drbssl://registry.example.net:1234'

Future Plan

Web/AJAX interface for executing commands and evaluating Ruby expression.
Firewall traversal
Built-in management features (e.g. node.reboot, node.add_user...)
Built-in SSL certificates generation (no more messy manual generation of SSL certificates)

Herder