While designed mainly for distributed systems programming, Erlang is an expressive programming language in its own right, and thus desirable for the construction of other sorts of programs.
The standard implementation of Erlang, Ericsson's Erlang/OTP, is
naturally optimized for supporting long-running systems. As such, its
support for short-lived programs, such as shell utilities, is somewhat
less than stellar. The erlaunch
project seeks to mitigate
this shortcoming, making Erlang an even more attractive option for
the construction of utility programs.
First, several common practices currently employed for implementing utility programs with Erlang/OTP are explored below. A different mechanism for implementing utility programs is then described, and the two are compared.
A common method for running a program written in Erlang as a shell
utility is to write a wrapper shell script in a scripting language such
as the Bourne Shell or the MS-DOS batch language. This script subsequently
uses erl -run module function args
to invoke the
desired Erlang program.
A variation on this general method is to combine the Erlang program and the shell script into the same source file, as is done in escript (Armstrong and Virding, 2001). The mechanics are otherwise largely the same as the plain shell script method: a BEAM emulator is executed to run the supplied Erlang code.
These two methods suffer the following particular shortcomings:
It can take a substantial amount of time just to bring up a BEAM
emulator from the command line. On a test machine1,
it took almost a full second for the command
erl -noshell -run erlang halt
to finish.
Although erlang:halt/0
can bring an emulator down
quickly, it is only reliable on some platforms. On other platforms,
it can truncate execution, and init:stop/0
is preferred.
The delay introduced by init:stop/0
is configurable, but
by default, erl -noshell -run init stop
took two full
seconds on the test machine.
With this in mind, we would do well to ask ourselves if it would not be better to have a BEAM emulator already running, which we can then request to run an Erlang function on our behalf.
erl_call
Another method of invoking Erlang programs from a shell is to have a distribution of Erlang nodes already running, and to use the erl_call utility (Törnkvist, 1999(?)) to connect to it and cause it to execute a function.
While exempt from the startup and shutdown costs mentioned above, this method can suffer from other idiosyncrasies:
Anecdotal evidence2 suggests that adding a node to a distribution only to remove it very shortly thereafter is not a reliable means of connecting to a running system for the purposes of exchanging a very small amount of data with it. This may be due to the overhead of managing the new node within the distribution.
The Erlang node need not be started by the same user as the caller. It need not be started in the same directory, nor with the same environment variables. While this may be convenient in certain situations, it does not reflect the most common case for utility programs, where a user expects the program to be run in the same environment they are currently working in.
Once a connection to a running Erlang distribution is established, any Erlang function on any node can be invoked. Again, this may be convenient under some circumstances, but is generally not a good idea when it is not called for, as it is perfectly happy to let the user shoot him or herself in the foot.
erlaunch
attempts to build upon the idea of call_erl
.
To minizime the shortcomings of this mechanism, it does not use Erlang's
built-in distribution mechanism, preferring instead discrete connections
on TCP/IP sockets. It also introduces requisite protocols for configuring
which functions may be invoked, and for informing the long-running
emulator of the environment of the caller.
erlaunch
works by initially starting a BEAM emulator and
an associated system shell. The shell is given environment variables
which point to the emulator. Subsequent invokations of erlaunch
in this subshell detect the environment variables and use them to create a
TCP/IP connection to the running emulator. Each time a command is invoked
via erlaunch
, the name of the selected command, the shell's
environment, and all input and output are transferred through this
connection. When the user no longer needs the emulator, they may exit
the shell, which will cause the emulator to exit as well.
erlaunch
consists of two parts: a client, erlaunch
,
written in C language (erlaunch.c
), which is what the user
invokes and interacts with; and a server, erlaunch.beam
, written
in Erlang (erlaunch.erl
), which is generally started when
erlaunch
launches the BEAM emulator.
When the server erlaunch.beam
is started, it reads a text
file called erlaunch.cmds
in its priv
directory.
This file is in file:consult/1
format, and specifies which
commands the client may request for execution, what Erlang functions they
map to, and how the arguments to the function and the return value from
the function are mapped to and from the caller's perspective.
Each function listed in this file can be invoked in one of two styles.
In the simplest style, ``raw
'', the function is assumed to
be written specifically for the erlaunch
interface. It must
have take four arguments: an erlaunch_context()
; a list()
of
string()
s which represent the arguments of the command
(including the "zeroth" argument, which is the name of the command, a
la C' language's argv[0]
); a string()
giving
the user's current working directory; and a dict()
mapping
string()
s to string()
s, representing the user's
environment. The function must return a tuple of the form {exit, integer()}
,
where the integer()
gives the desired exit code of the command,
as interpreted by the shell (that is, an exit code of zero generally indicates
success.) Communication with the user must be peformed by passing the
erlaunch_context()
to I/O functions in erlaunch.beam
.
Functions may also be invoked in ``mapped
'' style. In this
style, the function is not assumed to be written with erlaunch
in mind. It may take any number of arguments, and return any sensible return
value. The instructions in the erlaunch.cmds
file describe how
to map the user's arguments to the function's arguments, and the function's
return value to a usable exit code for the user. While mapped
functions are executed, their current working directory and environment
is set to the same as the user's, and I/O is captured and routed to and
from the user, through the TCP/IP connection.
On the TCP/IP connection, data is sent to and received from the server on a packet basis. All packets in both directions have the same general structure. Each packet consists of a header and a body. The header is an integer encoded as four bytes in network byte order, and describes the length of the body.
The body consists of two parts: a single status byte, and a string of arbitrary length. The string is not null-terminated; since the header specifies the length of the entire body, the string is always that length minus one byte for the status byte.
Lists or arrays of strings are sent as a series of packets containing the individual strings, followed by a packet containing a zero-length string.
Packets sent from the server to the client use the status byte to indicate an exit code. If the status byte for a given packet from the server is non-zero, this indicates that the client should exit to the shell. The exit code returned by the client should be one less than the value of the status byte.
Packets sent from the client to the server use the status byte to indicate an end-of-file condition. When an end-of-file is encountered on the client's standard input, it sends a packet with a non-zero status byte, and zero-length string.
Before erlaunch
initially launches the emulator, it chooses
a TCP/IP port on which communications will take place, and a cookie
(shared secret) which must be given during communications. It passes both
of these values to the launched emulator, so that it can set up a TCP/IP
socket server as needed. It also stores both of these values in the
environment before starting the new shell.
After a subsequent invokation of erlaunch
creates a TCP/IP
connection to the running emulator on the port specified in the environment,
it sends the cookie specified in the environment. If the correct cookie
is not given as the first item in the connection, further access is not
allowed. This restricts access to the running emulator to those who know
the cookie.
Assuming the cookie is correct, erlaunch
then sends the
command the user wishes the running emulator to execute. After receiving
the command, the server sends a packet back to the client. The status byte
of this packet will be non-zero if the server does not support the
specified command.
The client then sends an array of arguments which
accompany that command, a string representing the current working directory
of the user, and an array containing the user's environment
variables as NAME=VALUE
pairs. The current working directory is sent
seperately from the environment, as not all systems necesarily have the
convention of an environment variable called PWD
which contains this
information; further, it may be easily overridden, with confusion resulting.
Assuming all went well, the communication enters the I/O marshalling
phase. In this phase, lines of data read from the client's standard input
is sent via the TCP/IP socket to the server, where it is made available to
the Erlang command's input, either through erlaunch:get_line/2
for raw
functions or the io
module for mapped
functions, as appropriate. Likewise, data sent from
the Erlang command, either by erlaunch:fwrite/3
or by the
io
module, is sent through the socket and made available at
the client's standard output.
This phase lasts until either the server or the client receives a packet with a non-zero status byte, which closes the connection3.
With a working prototype of erlaunch
in place,
some measurements can be taken to see what kind of performance
improvement results can be expected.
For these measurements, a simple test function, erlaunch_eg:ls
,
is used, which simply lists the contents of the current directory. We have set
up three ways to call the function: from a shell script (script/ls_erl
,)
from an escript
script (script/ls_escript
,) and via
erlaunch
. Measurements were not taken for erl_call
,
as it is assumed that they would be comparable to those for erlaunch
,
while not being for the expected "current" directory.
script/ls_erl
# time script/ls_erl "Makefile" "script" "priv" "bin" "src" "doc" "c_src" "ebin" 0.705u 0.348s 0:02.06 50.4% 985+4640k 0+0io 0pf+0w
script/ls_escript
# time script/ls_escript "Makefile" "script" "priv" "bin" "src" "doc" "c_src" "ebin" 0.765u 0.302s 0:01.08 98.1% 963+4491k 0+0io 0pf+0w
erlaunch
# bin/erlaunch start erlaunch v2004.0309 - execute selected Erlang functions from command line Copyright (c)2003-2004 Cat's Eye Technologies. All rights reserved. Emulator started, starting erlaunch subshell. Type ^D to exit. # time bin/erlaunch ls "Makefile" "script" "priv" "bin" "src" "doc" "c_src" "ebin" 0.000u 0.006s 0:00.20 0.0% 0+0k 0+0io 0pf+0w # exit Stopping emulator. Thank you for using erlaunch!
An obvious application for erlaunch
is to improve on the
time taken by the command-line interface to the Erlang compiler. While
use of this utility, erlc
, can be avoided altogether by using
the c
module directly from EShell, it is often not practical
to take this route for some compilations, such as mixed-language project
Makefile
s.
A good test case for this sort of scenario is the Jungerl (Gorrie et al, 2003-). First, we measure the time it takes to build the entire Jungerl4 (command output omitted for the sake of brevity):
# gmake clean ... # time gmake ... 395.176u 176.260s 10:30.59 90.6% 963+5259k 604+1217io 213pf+0w
Next, we can force the Jungerl to use erlaunch_erl
by
making sure that the erlaunch
client is in the search path,
and manually editing support/include.mk
, changing the
ERLC
variable to ERLC := erlaunch erlc
.
We can then build the entire Jungerl again:
# gmake clean ... # time gmake ... 92.597u 40.801s 5:01.23 44.2% 961+4943k 304+397io 111pf+0w
This measurement indicates an impressive reduction of build time to roughly 50% of the original duration.
1. An AMD K6 800 MHz running FreeBSD 4.9 under minimal load.
2. This observation was made by the author in 2001 while attempting to use Erlang as an incoming e-mail processor. It seems to be confirmed by this posting to the erlang-questions mailing list, although the situation may well have changed since that time.