mkRinterp 1.1 Manual mkRinterp 1.1 ManualmkRinterp 1.1 - A remote interpreter for Tcl/Tk 8.4
Introduction
Notes
Commands
Security
Limitations
Examples
Applications
Installation
Changes
Author
mkRinterp is a package centered around a so-called 'remote interpreter', in short 'rinterp'. A remote interpreter is very similar to a standard Tcl interpreter, except that it is running on a remote machine somewhere in the network. The main aspects worth to mention here in the introduction are:
mkRinterp is in use for more than two years now in an international multi-platform project. It is used to install and update software and configuration data on 24/7 production servers. When it comes to copying files or executing commands and programs remotely, mkRinterp is strong. Because of its portability (thanks to Tcl), it replaces a whole range of high-maintenance and platform specific tools like Samba, NFS, rcp, rsh, rexec, FTP, Telnet or rcmd.
As briefly mentioned above, mkRinterp provides a so-called remote interpreter by means of the only new command 'rinterp'. 'rinterp' and its many sub-commands are intentionally very similar to the standard Tcl command 'interp'. It is recommended to read and understand the interp man page first, since it will then be easy to understand concept and usage of 'rinterp'.
Furthermore, mkRinterp is split into a server and a client part, similar to programs like Telnet or FTP: The server must be running on a host in order for clients to connect to it. For a connected client, the server creates a local isolated Tcl interpreter, which the client can use. This interpreter shall be called the sandbox for the rest of this document. The client operates with this remote interpreter very much like with a regular local interpreter. When the client disconnects, the sandbox interpreter is destroyed and hence does not retain state across subsequent connects.
A certain level of security is achieved through a username/password mechanism (see section Security) and the possibility to create sandboxes with a limited command set, based upon safe interpreters and/or the "Safe Tcl" concept, which allows for a mediated access to the server's host.
mkRinterp 1.1 was developed with Tcl 8.4 and makes intense use of the documented features about standard channel setup and redirection. The corresponding man page is Tcl_StandardChannels. It's not quite as easy as one would think... Especially the fact that calls like package require, namespace import, file channels and interp create initialize the standard channels is slightly bothering in a Windows service environment. Also, once closed, the original standard channels cannot be re-obtained, as it seems. But to be fair, redirection of standard channels to TCP socket is a great feature.
An rinterp server sets up a server socket as listener, then closes its standard channels. Hence, the next three incoming TCP clients are made the new standard channels. A controlled connection sequence including authentication ensures that the connection is set up properly. After a successful connect, commands are received through stdin and evaluated in the sandbox interpreter. The result is sent back via stdin, with stdout and stderr working as usual. Since standard channels are shared across interps by default, the sandbox standard channels and the application's are identical. As a result, the application must not do any 'puts ...' or 'gets stdin' etc. on its own, because it would end up across the socket on the client side. And: Since there can be only one set of stdin, stdout and stderr, there can only be one connection at a time.
A connection opened with rinterp create fires 3 sockets to the server, which on the server side are made into stdin, stdout and stderr. After successful authentication, 3 listeners on these 3 sockets are established. The listeners for the stdout and stderr sockets simply forward incoming data to the local stdout or stderr respectively. stdin is used to transfer commands and results back and forth. The client is listening to stdout and stderr as long as the result from a command has not been received on stdin yet. The result is made the local result. Data coming in through stdout and stderr after that stays in the TCP channel buffers. This effect is caused by the fact that sockets are per se independent and not synchronized, and can by mitigated by means of rinterp sync. On slow connections the effect can be observed for commands that put a lot into stdout in a short time (e.g. parray env).
rinterp features an alias option just like interp. This is in so far remarkable, as the aliased command is evaluated locally and can itself contain calls back to the remote side. A typical use case is, for example, to trace/debug messages from inside the remote application. The trace command would be aliased to a local command, which e.g. prints text to stdout, to a file, or into some GUI. If necessary, the local command could retrieve additional information from the server with rinterp eval.
The rinterp command features the sub-commands exec, read, write and copy. They are intended to help with typical remote administration tasks and are purely built upon rinterp's client-side functions. Hence, they cannot overwrite any limitations that are imposed on the sandbox interpreter by means of the security mechanisms. In other words, if the server-side sandbox cannot open a file (e.g. because it is a safe interpreter), the read sub-command does neither so.
This is the only server-side command. It starts a listener on the given port and closes the
standard channels. Then the application should enter the event loop in order for the rinterp
server to accept connections. With Tk loaded, the event loop is usually entered automatically.
In a pure Tclsh, one would have to use the
vwait
command to get into the event loop.
-access spec
Tells the server to use an acess specification. See section authentication
for details. If no access spec if given, the server uses
'allow 127.0.0.1; user guest guest -home . -script {} -safe 1{}'.
This accepts only connects from localhost as user guest in a safe sandbox interpreter.
-myaddr address
Specifies the network interface to use for the listener. This option may be useful if the client
machine has multiple network interfaces. The option is identical to that of the socket command.
-log script
The server can report messages (e.g. incoming connects etc.), if a log script is specified.
The message is appended to the script as one single argument, then the script is evaluated.
Analog to interp create.
Creates a remote interpreter specified by name after successfully
connecting to a server. The connect string must specify the remote host (host name or IP
address) and a port number, optionally preceeded by a username and password.
If user name and password are omitted, guest/guest is assumed.
See section authentication for details.
-stdout script
Data from stdout is normally forwarded to the local stdout channel. With the -stdout option,
a script can be specified instead. Whenever a line is available at stdout, the client would
append that line to the script and evaluate it.
-stderr script
Same like -stdout but for the stderr standard channel.
Analog to interp delete. Deletes a remote interpreter after disconnecting from the server.
Analog to interp eval. This command concatenates all of the arg arguments in the same fashion as the concat command, then evaluates the resulting string as a Tcl script in the remote interpreter's sandbox identified by name. The result of this evaluation (including error information such as the errorInfo and errorCode variables, if an error occurs) is mapped to the invoking local interpreter.
Synchronizes the three sockets of the remote interpreter name. Output coming from the remote stdout or stderr may tail after the result of the originating command has already been received. The sync option waits as long as the remote stdout and stderr is completely drained.
Analog to interp alias srcPath srcCmd targetPath targetCmd ?arg ...?. Creates an aliased command specified by serverCmd on the server side, which maps to localCmd arg... in a local interpreter localInterp on the client side. All arguments appended to serverCmd are appended to the args list of localCmd. The local interpreter must exist. As with the interp command, {} specifies the top-most, global interpreter.
Analog to interp alias srcPath srcCmd {}. Deletes the alias for serverCmd in the specified remote interpreter.
Analog to interp alias srcPath srcCmd. Returns the local command and its arguments associated to the alias specified by serverCmd.
Analog to interp target. Returns the local target interpreter name of the alias specified by alias, which must have been created previously by rinterp alias.
Analog to interp aliases. Returns a list with all aliases defined in the remote interpreter identified by name.
Returns state information about a remote interpreter. option must be one of:
peername
Returns information of the remote stdin channel, as obtained by fconfigure socket -peername.
It returns a list of three elements: address, host name and port to which the peer socket is connected or bound.
sockets
Returns the local socket handles that represent stdin, stdout and stderr on the server side, in that order.
issane
Indicates if the remote interpreter is still sane. Checks if the three sockets are open, and if an empty
command can be evaluated remotely. The result is either 0 or 1.
Analog to interp exists. Indicates, if the remote interpreter specified by name exists. The result is either 0 or 1.
Analog to interp slaves. Returns a list with the names of all existing remote interpreters.
Executes a program on the server side, similar to the standard exec command. It is almost identical with rinterp eval name { exec arg ... }, except that it takes care that stdout and stderr of the called program are redirected to the client's stdout and stderr. The internal implementation uses in fact a pipe, because the easier variant exec ... >@stdout 2>@stderr is not supported on Windows platforms. This causes the effect that any stderr output is sent after any stdout output, and not in the order it was flushed by the called program. See the exec man page for more details on this command.
Reads the contents of a remote file and returns it.
The -mode option determines the translation mode in FTP style.
See the man page for fconfigure channel -translation
for details of line end transformation.
-mode {ascii|binary ?pattern ...?}
Specifies the translation mode of the file in regards to carriage returns and line feeds.
The value ascii always transforms a file to the local platform's line end representation.
The value binary leaves the contents of the file unchanged.
If a list of glob-style patterns is specified, then the mode value only applies to files that
match one of these patterns. If there is no match, the other mode is applied.
Writes data to a file on the server side. This is the counterpart to rinterp read.
The -mode option determines the translation mode in FTP style.
-mode mode
See the comments supplied for this option in the rinterp read section.
Copies files and directories across local and remote platforms.
The remote interpreter name identifies the server where the file(s) shall be copied from.
The remote interpreter targetName specifies the server where the file(s) shall be copied to.
The source argument specifies the source file or directory.
The target argument specifies the target file or directory.
If one of name or targetName are specified by an empty string, then the local
platform is assumed.
-force
Analog to file copy -force. Causes existing files to be overwritten on the target side.
-into
If specified, required the target argument to be a directory. The source files are then copied
into this directory.
-preserve
Attemts to preserve file attributes of the source file(s), such as timestamps and permissions.
This feature has limitations if files are copied across different operating systems.
-command script
When specified, script is called with each copied file. The full pathname of the target file
is hereby appended to script.
-mode mode
See the comments supplied for this option in the rinterp read section.
A client connection is established only if certain conditions are fulfilled: A client must 'fire' exactly three connects to the server-side listener within a short time (as mentioned above, these three sockets are assigned to the server's stdin, stdout and stderr channels). In addition, the server asserts that all three connects come from the same IP address.
Now the server waits a short time for the authentication string (username and password) and checks it against the access specification (see below). It is also checked, if the client's IP address is one of the allowed IP adresses. Finally, the server-side interpreter, the sandbox, in which the client's commands are evaluated, is established, and the startup script is executed.
If any of the above tests fails, or if a timeout occurs, or if the startup script has an error, the connection is closed immediately. Also, if a previously established connection indicates any problem, like one of the sockets being closed unexpectedly, or if transmitted data is garbled and cannot be parsed, the connection is closed immediately.
Authentication of clients is controlled through a so-called 'access specification', which is passed to the server by means of the -access option. An access spec is expected to come as a Tcl script, since the server parses it in an isolated interpreter. You can also put the access spec in a file and use -access {source myfile}.
Clients authenticate through a username and password, which is passed as part of the connect string in the rinterp create command, and transmitted to the server during the connection setup sequence. The server looks up the access spec for the received username, compares the password (optionally hashes it first), and sets up the sandbox interpreter according to directives in the access spec.
An access spec's essential two commands are profile and user:
user username password profilename
Defines a user by means of a (unique) user name, a password and a profile.
The profile must have been defined with the profile keyword before it can be assigned to a user.
The password may be hashed ('encrypted', but that's not quite the same) by means of any
command of your choice, as long as you specify this command in the profile (with the hash
keyword, see below) and also provide the command in your server application, so that it can be called.
profile profilename body
Defines a profile by means of a (unique) profile name. A profile's body consists of a series
of sub-commands, which define how a password is encrypted, what client IP addresses are allowed
to connect from, and how the client's sandbox shall be configured.
The following sub-commands may be used in a profile's body:
hash command
Specifies a command that can digest any string to a "hashed" string.
Such method is typically used to obfuscate passwords and make them unreadable, even if someone can read the access spec file.
The sha1 command is a good example and comes with the
Tcllib and ActiveTcl.
If such a hash command is specified, the password sent by the client is encoded with it first and then compared with
the password in the access spec. The latter must hence be already hashed.
The server always invokes the hash command with exactly one argument, which is the received username and password,
separated by a slash '/' (e.g. 'guest/guest').
allow ?pattern ...?
Specifies which IP addresses the profile's assigned users can connect from.
Pattern is a glob-style pattern that is matched against an incoming connect's IP address (not hostname).
Several patterns may be specified. The allow keyword may be used more than once within a profile's body.
disallow ?pattern ...?
Specifies exceptions for the IP address ranges defined with allow.
An IP address is hence allowed, if it is matched by at least one pattern from the allow keyword,
and not matched by any pattern of the disallow keyword.
The disallow keyword may be used more than once within a profile's body.
The order of allow and disallow keywords in a body is not relevant.
home directory
Specifies the initial directory of the sandbox. This does not prevent the client to 'cd' into other
directories, nor is there a Unix-style concept of a home directory per user.
issafe boolean
Specifies, if the client's sandbox shall be created as a
safe interpreter or not. If not, the sandbox
has full access to the file system and can execute programs just like any regular Tcl shell can do.
Safe interpreters cannot harm the system in such way, because critical commands do not exist in
a safe interpreter. Read the interp man page
for details on safe interpreters.
script code
This keyword allows to specify a Tcl script, which is executed right after the sandbox interpreter has been created.
The script is evaluated in the global context of the server application directly and not inside some isolated interpreter.
The purpose of the script is to modify the sandbox interpreter directly by e.g. hiding or exposing commands,
or by applying the "Safe Tcl" concept.
The name of the sandbox interpreter is simply 'sandbox'.
Not all of the above keywords need to be specified as part of a profile definition, since the following default values are predefined like this:
profile defaultprofile {
hash {} ;# no hash: passwords in clear text
home . ;# home directory is where the application is
allow 127.0.0.1 ;# only allow connects from localhost
disallow {} ;# don't allow nothing
issafe 1 ;# always create a safe interpreter
script {} ;# without special initialization
}
The access spec file should be protected similar to the /etc/password file on Unix systems. It must not be editable by anybody except an administrator account, but it must be readable by the server application. Passwords should be hashed, or an unsafe sandbox could spy on the file and read the clear passwords.
Example of an access specification:
# profile for guests: leave most of the restrictive defaults
profile guestprofile {
allow * ;# allow guests to login from everywhere
script { ::safe::interpInit sandbox } ;# let's use Tcl's safe base concept
}
# profile for the admin: strong authentication, limited hosts
profile adminprofile {
hash ::sha1::sha1 ;# use the sha1 package from tcllib
allow 127.0.0.1 192.167.1.* ;# allow localhost and the test network 1
issafe 0 ;# create an unlimited sandbox
}
# users guest and test, password is no secret
user guest guest guestprofile
user test test guestprofile
# user admin, password encoded with sha1
user admin fe6a6f4dcf706093e8bbb14e696528a5349d910f adminprofile
The password for admin is admin, as an example. It has been created in a regular Tcl shell and then was simply copied into the access specification:
% package require sha1
1.0.3
% ::sha1::sha1 admin/admin
fe6a6f4dcf706093e8bbb14e696528a5349d910f
%
mkRinterp has some limitations and special things to consider:
The following code is all you need to create a server (on port 1234) from inside a regular Tcl shell. Note that you have to append the 'vwait forever' to the 'rinterp server' command, because (as described above), an rinterp server closes the standard channels, and the Tcl shell would terminate upon losing its stdin channel, when it needs it for user input.
% package require rinterp
1.1
% rinterp server 1234; vwait forever
The above server does not contain any special access specification and hence would only allow connects as user guest from localhost into a safe interpreter. Let's create an access specification, which defines a user 'test' with password 'test'. This user shall get a fully functional sandbox, but the password shall be encrypted therefore:
% package require rinterp
1.1
% package require sha1
1.0.3
% ::sha1::sha1 test/test
c2673e92de702654f259bd04bd41443c2a0db7a7
% set sAccess {
profile testprofile { issafe no; hash ::sha1::sha1 }
user test c2673e92de702654f259bd04bd41443c2a0db7a7 testprofile
}
% rinterp server -access $sAccess 1234; vwait forever
This server shall be used for the client-side code examples. If you do not have the sha1 package installed, please get Tcllib or ActiveTcl, or remove the 'hash' command from testprofile and put the password in clear text.
We connect to the above server and check, if our sandbox is a safe interpreter or not.
% package require rinterp
1.1
% rinterp create i test/test@localhost:1234
i
% rinterp eval i interp issafe {}
0
%
The following examples are trivial, because we are on 'localhost'. But it could be any remote machine running a different operation system far away.
% rinterp eval i parray tcl_platform
tcl_platform(byteOrder) = littleEndian
tcl_platform(machine) = intel
tcl_platform(os) = Windows NT
tcl_platform(osVersion) = 5.0
tcl_platform(platform) = windows
tcl_platform(user) = KRAUS
tcl_platform(wordSize) = 4
% rinterp eval i pwd
D:/Tcl/lib/mkRinterp1.1/shells
% rinterp eval i glob *
rconsh.tcl rconsole.tcl rtclsh.tcl
%
The imitation of the 'interp' command lets 'rinterp' appear rather transparent. Here is a direct comparison between a regular local interpreter and a remote one:
% interp create l % rinterp create i test/test@localhost:1234
l i
% interp eval l expr 5 + 3 % rinterp eval i expr 5 + 3
8 8
% interp eval l { expr 5 + 3 } % rinterp eval i { expr 5 + 3 }
8 8
% interp eval l puts "hello world" % rinterp eval i puts "hello world"
can not find channel named "hello" can not find channel named "hello"
% interp eval l { puts "hello world" } % rinterp eval i { puts "hello world" }
hello world hello world
% %
In the local interpreter we define a procedure that multiplies an arbitrary number of values. Then we create an alias on the remote side. If the alias is called in the remote interpreter, execution is in fact performed on the client side in the local interpeter:
% interp eval l {
proc multiply { f1 args } { foreach f2 $args { set f1 [expr $f1*$f2] }; return $f1 }
}
% interp eval l { multiply 3 4 5 6 }
360
% rinterp alias i mult l multiply
mult
% rinterp eval i { mult 3 4 5 6 }
360
%
Server ApplicationThe directory 'demon' contains a ready-to-use and stand-alone server application 'rinterpd.tcl', which runs in various modes on both Windows and Unix platforms:
The demon gets its access specification from the file 'rinterpd.access'. If it does not exist, the demon applies the default as specified above. You can also pass the options -access, -myaddr and -port to specify other values. The options correspond to the parameters of the 'rinterp server' command. The default port is 1234.
Remote Tcl Shell
The directory 'shells' contains a console-based client
application 'rtclsh.tcl', which resembles the behaviour of the standard Tcl shell.
Start modes from a command line:
tclsh rtclsh.tcl
The shell prompts for a connect string and, after a successful connection, goes into interactive mode like a Tcl shell would.
tclsh rtclsh.tcl connectstring
The shell attemts to connect immediately and, if successfully, goes into interactive mode.
tclsh rtclsh.tcl connectstring script ?arg ...?
The shell attemts to connect immediately and executes the specified script with any optional arguments.
This is also analog to a Tcl shell's behaviour. The shell exits when the script has finished.
A connection is exited with 'exit'. The shell is exited with 'exit' as a connect string.
On Windows, it cannot be started with Wish, since the Wish console's stdin channel is not a real stdin.
Remote Tk ConsoleAlso in the 'shells' directory, a metawidget 'rconsole.tcl' can be found, which provides a Tk-based remote shell (resembling the Wish console on Windows platforms).
The rconsole metawidget is used in the minimal sample application 'rconsh.tcl'. See the source code of rconsole.tcl and rconsh.tcl for details and examples.
When you start 'rconsh.tcl', e.g. by double-click or 'wish roncsh.tcl', you will be prompted for a valid connect string, and you can specify the prompt format. After a successful connection, you can issue Tcl commands on the remote side similar to the Remote Tcl Shell described above. You can exit a session with the 'exit' command.
The rconsole metawidget is based on a
text widget and accepts all of its commands and options.
In addition, the following special commands and options apply:
rconsole window ?option value ...?: Create an Rconsole metawidget.
window connect connectstring: Connect to a remote interpreter.
window disconnect: Disconnect from a remote interpreter.
window configure -prompt promptspec: Set the prompt (may contain commands in [...])
window configure -tracecmd command: Specify a command to call for important events.
window cget -prompt: Returns the current prompt.
window cget -tracecmd: Returns the current tracecmd, or {}.
window cget -rinterpname: Returns the name of the active remoter interpreter.
Remote Browser/ShellA more complex yet incomplete and certainly not bug-free example application can be found in the 'apps' directory. The file 'remshell.tcl' is a graphical application that shows a tree browser and a remote shell (implemented with the rconsole metawidget, see above).
After connecting to an mkRinterp server, you should be able to browse the filesystem by means of the treeview on the left side. By clicking on an underlined file you can view the contents of the file in a primitive editor. You can also enter Tcl commands in the shell on the right side of the application window.
If the first connect attempt fails, choose 'Connect...' from the 'File' menu to re-enter the logon dialog. Select 'Disconnect' to terminate an existing connection.
Last but not least, the example applicaton 'filecopy.tcl' allows to copy files and directories across two remote rinterp servers, or from and to the local system. This application is just another example and not bug-free. Be careful not to damage your system when copying files back and forth!
On startup, the application does not connect to a host but shows the local filesystem in two tree browsers. Choose 'Connect Left...' and/or 'Connect Right...' to connect to one or two remote systems. The corresponding tree is rebuilt, and the host name of the remote system is displayed on top of it.
You can copy files and directories from one side to the other, if on the target side a volume or directory is selected as the target location, and on the source side a file, directory or volume that shall be copied. Clicking the corresponding 'Copy' button will copy the selected source item into the directory or volume selected on the target side.
mkRinterp is written in pure Tcl/Tk (version 8.4) and does not contain any other package dependencies.
To install, place the directory "mkRinterp1.1" in one of the directories contained in the global Tcl variable "auto_path". For a standard Tcl/Tk installation, this is often "C:/tcl/lib" (Windows) and "/usr/local/lib" (Unix). The package can then be loaded with "package require rinterp".
Changes from 1.0 to 1.1
Michael Kraus
mailto:michael@kraus5.de
http://mkextensions.sourceforge.net