Scripty: The Ultimate Scripting Language

Introduction:

This site is about the Scripty programming language. The basic idea behind this language is to allow very simple interop between multiple scripting languages in the same program. Within a single file, you can specify snippets of Bash, Perl, Python, Clojure, etc, and with a few simple annotations, get them to seamlessly call each other as subroutines. This includes passing parameters and capturing result output (stdout). In theory, this concept could be extended to include any scripting language, but currently we only support Bash, Python, Perl, and Clojure.

Sitemap:

• File format
• Examples
• Interop
• Compilation
• Downloads
• TODO

File format:

The source for a Scripty program is a .scr file, which is an XML document with the following basic format:

        program := <program main="the id of the main script to start">
                      (bash | perl | clojure | python)*
                   </program>

        bash := <bash id="the id of this script">
                   (text | CDATA | param | run)*
                   <!-- all values here comprise the body of the script program -->
                </bash>
	
        perl := <perl id="the id of this script">
                   (text | CDATA | param | run)*
                   <!-- all values here comprise the body of the script program -->
                </perl>

        python := <python id="the id of this script">
                     (text | CDATA | param | run)*
                     <!-- all values here comprise the body of the script program -->
                  </python>

        clojure := <clojure id="the id of this script">
                      (text | CDATA | param | run)*
                      <!-- all values here comprise the body of the script program -->
                   </clojure>

        param := <param id="the id of a formal parameter of this script"/>

        run := <run id="the id of the script to run">
                  arg*
               </run>

        arg := <arg id="the id of a parameter for this actual parameter value">
                  (text | CDATA | param)*
                  // all values here will be taken as a string expression to be passed
               </arg>
    

Ranting about XML:

Why do we use XML? Because I love it!

• Just try to think of an easier way to parse N different languages out of the same document.
• XML is easy to read and write for both humans and computers.
• XML has its character escaping story figured out. There are like 3 special chars, and you can easily avoid them.
• XML has its own comments, so we can introduce (multi-line!) comments into any of these scripts, and moreover these are comments that do not even participate in the final text of the program.
• Since the script code is text/CDATA inside an element, I can easily embed other elements right in the middle of the script text. These can be replaced by whatever I want at compile time.
• XML can use different character set encodings if you need it to. Suck on that, JSON.

Examples:

• Example 1
• Example 2
• Example 3
• Example 4
• Example 5

Interop:

Programs interact by adding run tags within the program text. A run tag names another script to run, and may include argument values passed as parameters to the script.

The way that a run tag is implemented differs by language, but is essentially just a call to the "system" command, or whichever built-in library command allows the language to execute a shell command. When script A wants to call script B, it builds up a shell command string to execute B, and passes it to the "system" command. Then script A captures the output from B's STDOUT stream and uses the resulting string as the result of the run tag in A. In each language, the result of a run tag will be a (non-literal) string expression, and can be manipulated as such.

The method we use for passing parameters is through the program environment variables. Each script defines the set of required parameters it expects, and each other script that attempts to run it must provide arg values for all of those parameters. Each program's parameters are ordered and numbered, and stored in environment variables with names like "__arg0", "__arg1", etc. These names are designed to be fairly unique so that they will not conflict with existing environment variables. When script A wants to pass a value for "__arg0" to script B, it makes a "system" call similar to the following:

env __arg0=THEARGVALUE __arg1=THEARGVALUE python -c THEPYTHONPROGRAMTEXT

This example would run a python script with 2 parameters. In addition to passing the parameter values through the environment, the text of scripts themselves are all passed through the environment as well. For a script with an id of "MYID", the script body text would be placed in environment variable "__PROGRAM_MYID". Again, this name is munged to be unlikely to conflict with existing variables.

Compilation:

Compilation is achieved by performing a source-to-source translation from a scripty file (*.scr) to a bash script (*.sh), no matter which types of languages are used. The text of each program is stored in a top-level bash variable named "__PROGRAM_xxx" for a given script with id "xxx". To avoid escaping issues between the source language and bash, we wrap the text of the program inside a multi-line string statement, as such:

__PROGRAM_xxx="$(cat<<"__EOF"
   // In here is the entire text of the program.
   // No manner of weird special characters should
   // cause a problem, except for __EOF.
__EOF
)";
      

(little-known bash fact: when you wrap quotes around the starting "__EOF", it prevents any kind of string expansion inside the program text).

The method by which each languages retrieves its parameters is as follows:

• Bash: "$(printenv __arg0)"
• Python: os.environ['__arg0']
• Clojure: (java.lang.System/getenv "__arg0")
• Perl: $ENV{"__arg0"}

The method by which each language makes its system calls is as follows:

• Bash:
  $(env __arg0="$(echo VALUE0)" ... python -c "$(printenv __PROGRAM_xxx)")
• Python:
  subprocess.Popen(['env', '__arg0=' + str(VALUE0), ..., 'bash', '-c', os.environ['__PROGRAM_xxx']],
                     stdout=subprocess.PIPE).communicate()[0]
• Clojure:
  (slurp (.getInputStream (.exec (java.lang.Runtime/getRuntime)
                                   ["env" (str "__arg0=" VALUE0) ... "perl" "-e" (java.lang.System/getenv "__PROGRAM_xxx")]
                                   nil nil)))
• Perl:
     system(("env", "__arg0=" . (VALUE0), ..., "java", "-jar", $ENV{"__CLJJAR"}, "-e", $ENV{"__PROGRAM_xxx"}))

Downloads

• scripty.jar: Contains all Java classes needed to run Scripty
• scripty-src.zip: Contains all class files, source files, Makefile, examples, etc.

TODO

• Support more languages
• Add top-level command line parameters
• Add compiler command-line flags for customization