What to expect from the Ruby expect library?
14th Aug 2010
A little background first - expect is a library to interact with programs using ruby. Conceptually it’s based on the original UNIX expect program which is commonly used to automate UNIX administration. Expect library provides an API which can be passed a regex to match the expected output from the program, and optionally an action to take on a match. I have been experimenting with the expect library to automate a gdb session. Expect is an undocumented module and :Ri does not help. So just in case you are not able to get it to work for you, read on.
Table of Contents
Spawning interactive programs
Ruby has several ways to spawn programs. For a nice treatment of the subject check out Avdi’s excellent blog posts on the subject. We’ll use Ruby’s PTY library to spawn the program, and interact with it using expect. Using PTY library prevents IO buffering, which can cause problems during interaction with a spawned process. Let’s see how to make gdb print out a help message, using two methods.
Method 1 - Single interaction
This ruby script will take a string as an argument, and print out gdb help for it. We’ll spawn a gdb process, send it the gdb help command and read the results from its stdout. Listing 1 shows the script. You’d notice that PTY.spawn is passed a Ruby block, which is executed immediately after the gdb process is spawned. When the block ends the gdb process gets terminated. PTY.spawn passes in the input/output File objects for the child gdb process, as well as its pid to the block. The till_prompt() method, reads all output from gdb till the next gdb prompt is seen, and returns the data read as a string. Notice the use of IO.getc() method. We don’t use IO.gets because when gdb prints out its prompt, it waits for input from the use and therefore does not print a newline, after the prompt. If IO.gets is used, it will stall waiting for a newline after the prompt is printed by gdb.
Sample output after executing this script is shown in Listing 2. This is a deliberately minimal program, designed to demonstrate concepts, and does not attempt to do any error handling.
2 require ‘pty‘
3
4 def till_prompt(cout)
5 buffer = ""
6 loop { buffer << cout.getc.chr; break if buffer =~ /\(gdb\)/ }
7 return buffer
8 end
9
10 PTY.spawn("gdb") do |gdb_out, gdb_in, pid|
11 printf till_prompt(gdb_out)
12 gdb_in.printf("help #{ARGV[0]}\n")
13 puts till_prompt(gdb_out)
14 end
Listing 1 - gdb.rb
help break
Set breakpoint at specified line or function.
break [LOCATION] [thread THREADNUM] [if CONDITION]
LOCATION may be a line number, function name, or “*” and an address.
If a line number is specified, break at start of code for that line.
If a function is specified, break at start of code for that function.
If an address is specified, break at that exact address.
With no LOCATION, uses current execution address of selected stack frame.
This is useful for breaking on return to a stack frame.
THREADNUM is the number from “info threads”.
CONDITION is a boolean expression.
Multiple breakpoints at one place are permitted, and useful if conditional.
Do “help breakpoints” for info on other commands dealing with breakpoints.
(gdb)
[sudhanshu@sudhanshu-desktop]$
Listing 2 - gdb.rb output
Notice the one shot usage of this program. It exits immediately and that’s why sending PTY.spawn a block for execution makes sense here. We’ll see why we’d not want to send a block of code to execute, in the next method of spawning interactive processes.
Method 2 - Multiple interactions
This method is useful in those circumstances, where you’d like to save the input, output objects returned by PTY.spawn for later and interact with the process multiple times using these objects. Let’s rewrite the gdb.rb program in Listing 1, to be used as a loadable library in irb, instead of a one-shot program. Listing 3 shows this program. You’d notice the new gdb() method which is provided as an API to run arbitrary gdb commands. Also notice that this time we store a reference to the input/output objects returned by PTY.spawn. Listing 4 shows how this version can be used by the user more interactively.
2 require ‘pty‘
3
4 def till_prompt(cout)
5 buffer = ""
6 loop { buffer << cout.getc.chr; break if buffer =~ /\(gdb\)/ }
7 return buffer
8 end
9
10 def gdb(string)
11 @gdb_in.printf("#{string}\n")
12 puts till_prompt(@gdb_out)
13 end
14
15 @gdb_out, @gdb_in, @pid = PTY.spawn("gdb")
16 printf till_prompt(@gdb_out)
Listing 3 - gdb_irb.rb
>> require ‘gdb_irb.rb’
GNU gdb (GDB) 7.1-ubuntu
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
(gdb)=> true
>> gdb "help break"
help break
Set breakpoint at specified line or function.
break [LOCATION] [thread THREADNUM] [if CONDITION]
LOCATION may be a line number, function name, or "*" and an address.
If a line number is specified, break at start of code for that line.
If a function is specified, break at start of code for that function.
If an address is specified, break at that exact address.
With no LOCATION, uses current execution address of selected stack frame.
This is useful for breaking on return to a stack frame.
THREADNUM is the number from "info threads".
CONDITION is a boolean expression.
Multiple breakpoints at one place are permitted, and useful if conditional.
Do "help breakpoints" for info on other commands dealing with breakpoints.
(gdb)
=> nil
>> gdb "file /bin/date"
file /bin/date
Reading symbols from /bin/date…(no debugging symbols found)…done.
(gdb)
=> nil
>> gdb "r"
r
Starting program: /bin/date
[Thread debugging using libthread_db enabled]
Tue Aug 10 05:37:22 IST 2010
Program exited normally.
(gdb)
=> nil
>> @gdb_in.inspect
=> "#<File:/dev/pts/4>"
>>
?> @gdb_out.inspect
=> "#<File:/dev/pts/4>"
>>
Listing 4 - Interactive GDB in irb
Notice how irb “wraps” around gdb and created a much more powerful debugging environment on top of gdb. We could for instance read large amounts of data from the process being debugged into Ruby variables/classes and analyze it using the far more powerful facilities provided by the Ruby programming enviroment, as compared to gdb macros/scripts.
Expect library
Now that we have seen how to spawn processes in Ruby, and interact with them with custom code, let’s see how to use the expect library to interact with them. The expect library adds an expect() method to the IO class, which is the basis of all input output in Ruby. The expect() method is just a beefed up version of the till_prompt() method that we saw above.
While the till_prompt() method used a fixed pattern to match the next gdb prompt, the expect() method takes a ruby String or a regular expression object of type Regexp as a pattern to match against program output.
The till_prompt() method simply returned the whole buffer after matching the fixed pattern. However, the expect() method can optionally take a Ruby block to execute as soon as the pattern matches. This block is passed in the array containing the result of the match. Alternatively, if a block is not given, it will return the result array containing the buffer against which the pattern was matched, followed by the flattened, MatchData object returned by Regexp#match().
Apart from these the expect() method can optionally take a timeout value in seconds as its second argument. If no match is found within the given time limit, it returns nil. Thus the API can be summarized in the following way:
Note that if a block is passed into expect(), the return value is that returned by the block, which can be anything and not necessarily an array. Now let’s see the implementation of the above two programs using the expect() method.
2 require ‘pty‘
3 require ‘expect‘
4
5 PTY.spawn("gdb") do |gdb_out, gdb_in, pid|
6 gdb_out.expect(/\(gdb\)/) { |r| gdb_in.printf("help #{ARGV[0]}\n") }
7 puts gdb_out.expect(/\(gdb\)/)[0]
8 end
Listing 5 - gdb_expect.rb
2 require ‘pty‘
3 require ‘expect‘
4
5 def gdb(string)
6 @gdb_in.printf("#{string}\n")
7 puts @gdb_out.expect(/\(gdb\)/)[0]
8 end
9
10 @gdb_out, @gdb_in, @pid = PTY.spawn("gdb")
11 puts @gdb_out.expect(/\(gdb\)/)[0]
Listing 6 - gdb_irb_expect.rb
Beginners with the Ruby expect library can get into trouble if they assume that the pattern being passed in to the expect() method will be matched against each line output by the spawned program. This assumption is incorrect, because as we saw the pattern is actually matched against all the characters read into a buffer which includes newline characters. It’s also worth mentioning that the newlines seen by expect() are “\r\n” and not “\n”.
As a debugging mechanism there’s a global variable called $expect_verbose, provided by the expect library. Set this variable to true in your program, and expect() method will print every character read at each intermediate step on stdout. This is an extremely useful tool for debugging expect programs.
Resources
- Source Code: tgz, zip
- Ruby 1.9.2 improves expect
- An alternative implementation of expect
- Ruby and SSH
