Programming Languages (4) --- Writing standalone programs using libraries

Enter your name and student ID.

• Name:
• Student ID:

1. Say your language

Say the language you are assigned to. I am working on

BEGIN SOLUTION END SOLUTION Go Julia OCaml Rust

2. Objective

• In this notebook, you are going to learn how to write a standalone program in the language you chose, rather than a program in Jpyter

• Jupyter is a great environment to learn what a particular programming language smells like without getting caught on installations, command lines, options, files, and so on

• In the real world, however, you need to be able to develop a program that "does a job" for you and users

• Also, real programs must use libraries not available by default and have multiple files that call each other, realities difficult to learn by Jupyter alone

• There is also a minimum boilerplate each language imposes, such as the main function of C or a class definition that has public static void main(String[] args) in Java; some languages allow you to write arbitrary expressions in the program toplevel, while others require them to be inside the main function; you may miss such basics if you write your programs only in Jupyter

• To avoid missing such basics for developing real programs, this notebook's goal is to learn how to develop a somewhat useful program with command lines and terminals

• This notebook is going to walk you through skills you need to master, but it only serves as an explanation and a convenient page to save typing commands by bash cells

• You are supposed to do your real work by SSH and command line terminals by yourself. Cells in this page are given to make clear what you need to do (and to help you troubleshoot if anything goes wrong)

• see see How to access Jupyter environment for how to SSH to taulec environment

• if you have difficulty setting it up right away you could use Jupyter terminal for now

• Editors

• emacs, nano, vim are available on taulec; just use them within SSH terminal

• you can run vscode on your computer and use Remote-SSH extension to open files on taulec

3. Roadmap

1. compile and execute a simplest program by command lines
2. use a build system to streamline library installation and dependency management
3. import a library so you can use it in your program

• Along the way, I encourage you to acquire skills to search for libraries that fit whatever tasks you will do in future and know details about a particular library (e.g., functions and types available in them), rather than simply following instructions written here to accomplish this particular task

• You are encouraged to do the following exercise using SSH and command line, rather than merely pressing SHIFT + ENTER

4. Compiling and executing a simplest program from a command line terminal

• Go : go (compiler)
• Julia : julia (compiler and interactive environment)
• OCaml : ocamlc (compiler), ocaml (interactive and interactive environment)
• Rust : rustc (compiler)

4-1. Go

• go is the command that does everything from compilation, execution, installation of the library, with its subcommands
• go build is the subcommand to compile a program

4-1-1. Setting up

• check if you can run go command

go version

• if it raises an error indicating go command is not found, add ~/go/bin to your PATH environment variable

export PATH=~/go/bin:$PATH

• you can put the above in your ~/.profile so it automatically happens when you login with SSH

• make sure you can now run go command and which go shows where it is

which go
go version

4-1-2. Compiling and executing a program

• move to a working directory

cd ~/notebooks/pl04_standalone/00hello/go

• a simplest program

cat hello.go

• compile it

go build hello.go

• execute it

./hello

4-1-3. Interactive environment

• go is basically a batch-compiled language; there is no official interactive environment
• Jupyter uses an external program called gophernotes, which unfortunately does not support the latest version of the go language
• alternatively, you could use The Go Playground to interactively play with Go

4-2. Julia

• julia is the command that serves both as an interactive environment and a file executor
• Julia does not separate compilation and execution; a program (whether it's from an interactive environment or in a file) is always parsed and gets executed immediately; everything is Just-In-Time compiled

4-2-1. Setting up

• check if you can run julia command

julia --version

• if it raises an error indicating julia command is not found, add ~/.juliaup/bin to your PATH environment variable

export PATH=~/.juliaup/bin:$PATH

• you can put the above in your ~/.profile so it automatically happens when you login with SSH

• make sure you can now run julia command and which julia shows where it is

which julia
julia --version

4-2-2. Compiling and executing a program

• move to a working directory

cd ~/notebooks/pl04_standalone/00hello/jl

• a simplest program

cat hello.jl

• execute it

julia hello.jl

4-2-3. Interactive environment

• julia is an interactive environment
• don't execute it within Jupyter
• do it in SSH terminal (recommended) or Jupyter terminal

$ julia
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.10.2 (2024-03-01)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

┌ Warning: Terminal not fully functional
└ @ Base client.jl:422
julia>

4-3. OCaml

• ocaml is an interactive environment to which you input OCaml program interactively, somewhat similar to Jupyter
• ocamlc is a compiler that converts OCaml program files to an executable file

4-3-1. Setting up

• check if you can run ocamlc command

ocamlc --version

• if it raises an error indicating ocamlc command is not found, execute the following in your shell

eval $(opam env)

• you can put the above in your ~/.profile so it automatically happens when you login with SSH

• make sure you can now run ocamlc command and which ocamlc shows where it is

which ocamlc
ocamlc --version

4-3-2. Compiling and executing a program

• move to a working directory

cd ~/notebooks/pl04_standalone/00hello/ml

• a simplest program

cat hello.ml

• compile it to get a standalone executable

ocamlc hello.ml

• execute it

./a.out

4-3-3. Interactive environment

• ocaml is an interactive environment
• don't execute it within Jupyter
• do it in SSH terminal (recommended) or Jupyter terminal

$ ocaml
        OCaml version 4.13.1

#

4-4. Rust

• rustc is the command that compiles a Rust source file

4-4-1. Setting up

• check if you can run rustc command

rustc --version

• if it raises an error indicating rustc command is not found, execute the following in your shell

. ~/.cargo/env

• you can put the above in your ~/.profile so it automatically happens when you login with SSH

• make sure you can now run rustc command and which rustc shows where it is

which rustc
rustc --version

4-4-2. Compiling and executing a program

• move to a working directory

cd ~/notebooks/pl04_standalone/00hello/rs

• a simplest program

cat hello.rs

• compile it

rustc hello.rs

• execute it

./hello

4-4-3. Interactive environment

• rust is basically a batch-compiled language; there is no official interactive environment
• Jupyter uses an external program called evcxr_jupyter
• Alternatively, you could use Rust Playground to interactively play with Rust

5. Build system

• A build system is a program that helps you develop programs using external libraries and/or consisting of multiple files
• OCaml, Go and Rust have their own build system, dune, go, and cargo, respectively
• They have a command to create a project and build it

5-1. go is Go's build system

• go command also serves as go's build system

• spend at least a few minutes to read the first few sections of How to Write Go Code

• go to the directory,

cd ~/notebooks/pl04_standalone/01build_sys/go

• which contains only a file _

ls

• create a project called foo by

mkdir foo
cd foo
go mod init foo

• it just creates go.mod file

ls -lR

• create a file foo.go with the following content (using editor)

package main
func main() {
     println("hello world")
}

• compile it by

go build

• which will create foo; run it by

./foo

5-2. Julia has no particular build system

• Julia has no build system, as it does not require a separate step to compile source files into an executable file
• you always just-in-time compile your source files and run them immediately

5-3. dune: OCaml's build system

• dune is a build system for OCaml

• spend at least a few minutes on its document, including Quickstart section

• install dune for you (do it from a terminal, not by SHIFT-ENTER below). $ below is a command prompt, not a part of your input

$ eval $(opam env)
$ opam install dune

which says

Run eval $(opam env) to update the current shell environment

at the end. Follow the instruction

$ eval $(opam env)

• go to the directory,

cd ~/notebooks/pl04_standalone/01build_sys/ml

• which contains only a file _

ls

• create a project called foo by

dune init proj foo

• let's find files in it

ls -lR

• foo/bin/main.ml is the source file you are going to add your program

cat foo/bin/main.ml

• go to the top level directory of the project

cd foo

• build the executable by

eval $(opam env)
dune build

• which will create ./_build/default/bin/main.exe; run it either by

dune exec foo

or

./_build/default/bin/main.exe

5-4. cargo: Rust's build system

• cargo is the build system of Rust

• spend a few minutes to read Hello, Cargo! section of the Rust book

• go to the directory,

cd ~/notebooks/pl04_standalone/01build_sys/rs

• which contains only a file _

ls

• create a project called foo by

cargo new foo

• which will create a directory foo; go to the directory

cd foo

• let's find files in it

ls -lR

• src/main.rs is the source file you are going to add your program

cat src/main.rs

• build the executable by

cargo build

• which will create target/debug/foo; run it either by

cargo run

or

./target/debug/foo

• we will later explain how to use external libraries from main.rs

6. Using libraries

• Your program rarely consists of only the code you wrote; you rely on many programs written by others, which are generally called "libraries"; they provide your program with functions, variables, classes, types, and so on

• When you need a function that is general enough so you reasonably believe is available as a library and complex enough so you do not want to develop yourself, you search for a library implementing it

• Having a confidence on finding an appropriate library (on the web), knowing what kind of functions/variables/classes it provides and how to use them, and correctly using it from your program is an important step toward mastering a language

• Using a library in general entails

  • searching for a library you want
  • installing it if necessary (e.g., Python's pip command or from a github)
  • telling the compiler/interactive command where the library is, if it is installed at a place they don't know. this is generally done via setting an environment variable (Python's PYTHONPATH), giving a command line, or setting a variable in the interactive environment (Python's sys.path)
  • you write a statement or a command to import it in your program, so that your can mention its function, variables, etc. (Python's import statement)

• Some libraries are builtin, in the sense that they do not require any of the above (e.g., Python's open function)

• Some libraries are not builtin but standard, in the sense that they are already installed along with the language itself; you don't have to install it by yourself but just have to import it (e.g., Python's re module)

• Some languages (OCaml, Go and Rust) have a build system that automatically install external libraries

6-1. Libraries in Go

6-1-1. Builtin libraries

• Go's builtin libraries are called builtin package. See builtin package to find names in them. They are available automatically in any program
• if you find a name in the above page, you can use it by just name in your program
• for example, there is func println(args ...Type) in the above page, so you can just write println(10) in any program

6-1-2. Standard libraries

• Go's Standard library are packages that are installed with the Go system

• your program refers to a name in a package by saying in the beginning of a source file import "module-name" (notice that the name is double-quoted) and with module-name.name

• for example let's say you want to use Abs function in math package. you write

import "math"

in the beginning of a file and

math.Abs(-3)

6-1-3. External libraries

• See Managing dependencies for general concepts
• search pkg.go.dev for a library you want
• once you find the package name, you import its repository name you can find on the right of the package page import "repository-name" in your source file; go build does the rest from downloading the library to compiling it to linking with it
• moreover, the same scheme works for code not in pkg.go.dev
• for example, if you search pkg.go.dev for xmldom, you find it's repository "github.com/subchen/go-xmldom" under Repository section on the right of the go-xmldom. so just change your foo.go file to something like

package main
import "github.com/subchen/go-xmldom"
func main() {
        xmldom.ParseXML("<a>123</a>")
        println("hello world")
}

to your source file.

• install it by

go get github.com/subchen/go-xmldom

and you are done

• go build compile it and link it with your program

6-2. Libraries in Julia

6-2-1. Builtin libraries

• Julia's builtin libraries are called Core and Base modules. See Base tab in Julia documentation page to find names in them. They are available automatically in any program
• if you find Base.name in the above page, you can use it by just name in your program

6-2-2. Standard libraries

• Julia's Standard Library in Julia documentation are modules that are installed with the Julia system

• your program refers to a name in the standard library by saying in the beginning of a source file import module-name and by module-name.name

• alternatively, you can say using module-name in the beginning of your program and name without prefixing it with the module name

• for example let's say you want to use DateTime function in Dates module. you either write

import Dates

in the beginning of a file and

Dates.DateTime(2022)

2. or

using Dates

in the beginning of a file and just

DateTime(2022)

6-2-3. External libraries in Julia

• Julia's general registry; see Julia Packages
• if you find a library you want, either in JuliaHub or Julia Packages, you can install it from julia interactive environment by

$ julia
julia> import Pkg
julia> Pkg.add("name")

• Example: let's say you want to use LightXML package
• install it by

$ julia
julia> import Pkg
julia> Pkg.add("LightXML")

• in your program, write

import LightXML

or

using LightXML

• with the former, you use various functions in the module with LightXML.parse_file, LightXML.root, etc. whereas with the latter simply with parse_file, root, etc.

6-3. Libraries in OCaml

6-3-1. Builtin libraries

• Builtin functions, types, etc. are in Pervasives module. Names in the pervasive module are available automatically in any program

• Example: your program can use sin function just by sin, as it is in the pervasive module

6-3-2. Standard libraries

• OCaml's standard libraries are modules that are installed with the OCaml system.

• Your program doesn't have to do anything special to use entities (functions, types, etc.) in a standard module

• module-name.name refers to name in module module-name

• or, add open module-name in the beginning of your program file and just name refers to name in the module

• Example: you can use a variable argv in Module sys either by

Sys.argv

or by having

open Sys;;

in the beginning of your program and

argv

6-3-3. External libraries

• Find a module you want to use in opam

• If it is found, you can install it by opam command opam install module-name

• Once installed, the simplest way is to use it is dune

• Example: let's use markup module; find its README.md at github and the documentation

• go to the source directory

cd ~/notebooks/pl04_standalone/01build_sys/ml/foo

• modify bin/dune file, which should originally look like

(executable
 (name main)
 (libraries foo))

• confirm it

cat foo/bin/dune

• You must do the following steps in a terminal and a text editor

• add markup to the (libraries foo) clause in the dune file, so it looks like

(executable
 (name main)
 (libraries foo markup))

• change the source file bin/main.ml so it looks like

Markup.parse_xml (Markup.string "<a>123</a>")

• install the library by doing

opam install markup

• build it again with dune

dune build

• and run it, just to make sure it does not raise an error

./_build/default/bin/main.exe

6-4. Libraries in Rust

6-4-1. Builtin libraries

• Rust's builtin libraries are called std::prelude module. From this page, you go to v1 and Module core::prelude::rust_2021 for specific names available in all Rust programs
• if you find a name such as crate::module::module::name in the above pages, you can use it by just name in your program
• for example, there is crate::option::Option::None in v1, so you can write None in any program

6-4-2. Standard libraries

• Rust's Crate std is the set of modules and macros installed with the Rust system
• navigating to a module, you can find entities such as structs, traits, functions of a module
• for example, fs is a module in the Crate std, where you can find entities such as a struct File and a function read
• a module may contain another module (submodule)
• for example, f64 module contains a submodule consts
• you can refer to an entity in a module either by its fully-qualified name starting from the toplevel name (such as std), to a module name (such as fs, f64), all the way down to the name of the entity. there may be multiple module names in between.
• your program refers to an entity by its fully-qualified name, such as std::f64::consts::PI (see std::f64::consts::PI
• or, you can refer to std::f64::consts::PI by saying use std::f64::consts::PI in the beginning of a program and with just PI
• you can go the middle road; for example, you can also say use std::f64 or use std::f64::consts in the beginning of a program and refer to std::f64::consts::PI by f64::PI or consts::f64::PI respectively

6-4-3. External libraries

• crates.io is a repository you can search for a library you want
• the concepts you have just learned above applied to external libraries, too
• an external library is just a module contained in a crate other than std
• once you find a module in crates.io, the simplest way is to let cargo command do the job
• add the name of the crate that contains the module you want to use to Cargo.toml file, under [dependencies] section
• example: let's say you want to use minidom module
• press the copy button on the right to copy a string that looks like

minidom = "0.14.0"

and paste it below [dependencies] in Cargo.toml file

• alternatively you can just write

minidom = "*"

to indicate you want the latest version of the library

• consult the document, by following the link below "Documentation"
• change the source file foo.rs so it looks like

use minidom::Element;

fn main() {
   let s = "<a xmlns=\"https://a.com\">123</a>";
   let _root : Element = s.parse().unwrap();
}

• build it with

cargo build

which does everything from downloading the source to compiling it to linking it with the code you write

• execute it with either by

./target/debug/foo

or

cargo run

Problem 1 : Show command line args (like echo command)

• write a standalone program that prints the first argument given in the command line followed by a newline, just like the echo command (echo hello prints hello)
• it must be written under ~/notebooks/pl04_standalone/p1/<lang> directory, where <lang> is one of ml, jl, go and rs
• always create echo folder under the above directory, either manually or by creating a project with the respective build system
• as a result, the source file name must be
  • Go : ~/notebooks/pl04_standalone/p1/go/echo/echo.go
  • Julia : ~/notebooks/pl04_standalone/p1/jl/echo/echo.jl
  • OCaml : ~/notebooks/pl04_standalone/p1/ml/echo/bin/main.ml
  • Rust : ~/notebooks/pl04_standalone/p1/rs/echo/src/main.rs
• after writing the program, compile and run it with one of the following

• Go

cd ~/notebooks/pl04_standalone/p1/go/echo
go build
./echo 123

• Julia

cd ~/notebooks/pl04_standalone/p1/jl/echo
julia echo.jl 123

• OCaml

eval $(opam env)
cd ~/notebooks/pl04_standalone/p1/ml/echo
dune build
./_build/default/bin/main.exe 123

• Rust

cd ~/notebooks/pl04_standalone/p1/rs/echo
cargo build
./target/debug/echo 123

• They all must print 123

Problem 2 : Read a file (like cat command)

• write a standalone program that takes a filename in its command line and prints its content, just like cat command
• filename convention is the same as p1. it must be written under ~/notebooks/pl04_standalone/p2/<lang> directory, where <lang> is one of ml, jl, go and rs
• always create cat folder under the above directory, either manually or by creating a project with the respective build system
• as a result, the source file name must be
  • Go : ~/notebooks/pl04_standalone/p2/go/cat/cat.go
  • Julia : ~/notebooks/pl04_standalone/p2/jl/cat/cat.jl
  • OCaml : ~/notebooks/pl04_standalone/p2/ml/cat/bin/main.ml
  • Rust : ~/notebooks/pl04_standalone/p2/rs/cat/src/main.rs
• create a file named expr.xml under the cat directory whose contents should be

<plus xmlns="https://expr.com">
  <num>1</num>
  <div>
    <num>2</num>
    <num>3</num>
  </div>
</plus>

• after writing the program, compile and run it with one of the following

• Go

cd ~/notebooks/pl04_standalone/p2/go/cat
go build
./cat expr.xml

• Julia

cd ~/notebooks/pl04_standalone/p2/jl/cat
julia cat.jl expr.xml

• OCaml

eval $(opam env)
cd ~/notebooks/pl04_standalone/p2/ml/cat
dune build
./_build/default/bin/main.exe expr.xml

• Rust

cd ~/notebooks/pl04_standalone/p2/rs/cat
cargo build
./target/debug/cat expr.xml

• They all must print the contents of expr.xml

Problem 3 : Read an XML string

• write a standalone program that takes a filename in its command line and parses it into a DOM tree, using an appropriate XML parser

  • Go : xmldom
  • Julia : LightXML
  • OCaml : markup
  • Rust : minidom

• filename convention is the same as p1. it must be written under ~/notebooks/pl04_standalone/p3/<lang> directory, where <lang> is one of ml, jl, go and rs

• always create readxml folder under the above directory, either manually or by creating a project with the respective build system

• as a result, the source file name must be

  • Go : ~/notebooks/pl04_standalone/p3/go/readxml/readxml.go
  • Julia : ~/notebooks/pl04_standalone/p3/jl/readxml/readxml.jl
  • OCaml : ~/notebooks/pl04_standalone/p3/ml/readxml/bin/main.ml
  • Rust : ~/notebooks/pl04_standalone/p3/rs/readxml/src/main.rs

• create expr.xml with the same contents as Problem 2

<plus xmlns="https://expr.com">
  <num>1</num>
  <div>
    <num>2</num>
    <num>3</num>
  </div>
</plus>

• after writing the program, compile and run it with one of the following

• Go

cd ~/notebooks/pl04_standalone/p3/go/readxml
go build
./readxml expr.xml

• Julia

cd ~/notebooks/pl04_standalone/p3/jl/readxml
julia readxml.jl expr.xml

• OCaml

eval $(opam env)
cd ~/notebooks/pl04_standalone/p3/ml/readxml
dune build
./_build/default/bin/main.exe expr.xml

• Rust

cd ~/notebooks/pl04_standalone/p3/rs/readxml
cargo build
./target/debug/readxml expr.xml

• make sure your program
  • normally exits without raising an error if given a valid XML file
  • raises an error if given a non-existing XML file
• the behavior given a file containing an invalid (slightly broken) XML expression can be up to the behavior of underlying XML library (some raise an error, others automatically skip or fix broken parts)

Problem 4 : Split it into multiple files

• split the program you just wrote into two files

  • readxml.{go,jl,ml,rs}, which contains a function readxml that takes a filename and returns a DOM tree
  • readxml2.{go,jl} or main.{ml,rs}, which contains a main function (Go or Rust) or a toplevel expression (Julia or OCaml) that calls readxml function

• filename convention is the same as p1. it must be written under ~/notebooks/pl04_standalone/p4/<lang> directory, where <lang> is one of ml, jl, go and rs

• always create readxml folder under the above directory, either manually or by creating a project with the respective build system

• as a result, the source file name must be

  • Go : ~/notebooks/pl04_standalone/p4/go/readxml/{readxml,readxml2}.go
  • Julia : ~/notebooks/pl04_standalone/p4/jl/readxml/{readxml,readxml2}.jl
  • OCaml : ~/notebooks/pl04_standalone/p4/ml/readxml/bin/{readxml,main}.ml
  • Rust : ~/notebooks/pl04_standalone/p4/rs/readxml/src/{readxml,main}.rs

• create expr.xml with the same contents as Problem 2 and 3

<plus xmlns="https://expr.com">
  <num>1</num>
  <div>
    <num>2</num>
    <num>3</num>
  </div>
</plus>

• after writing the program, compile and run it with one of the following

• Go

cd ~/notebooks/pl04_standalone/p4/go/readxml2
go build
./readxml2 expr.xml

• Julia

cd ~/notebooks/pl04_standalone/p4/jl/readxml2
    julia readxml2.jl expr.xml

• OCaml

eval $(opam env)
cd ~/notebooks/pl04_standalone/p4/ml/readxml2
dune build
./_build/default/bin/main.exe expr.xml

• Rust

cd ~/notebooks/pl04_standalone/p4/rs/readxml2
cargo build
./target/debug/readxml2 expr.xml

6-5. Hints

• First separate your program into readxml function and its caller in the same file and make sure it works
• Only then start splitting them into the two files
• What it takes to call a function defined in another file depends on the language
• Consult appropriate sections of the language manual, but here are some hints

6-5-1. Go

• If both files are in the same package (i.e., have the same package main at the head of the file), things are straightforward
• You can just call readxml from main and go build will take care of the rest
• Things get trickier if you put them in separate packages, a topic we do not go into any further

6-5-2. Julia

• All it takes is to include("readxml.jl") in the other file, whose effect is as if you put the contents of the file in place
• You can then just call readxml
• You could put them into separate "modules", in which case you have to qualify names in another module (we don't get into it)

6-5-3. OCaml

• All names defined in readxml.ml are put in Readxml module
• You refer to names (e.g., readxml function) in the Readxml module by Readxml.readxml

6-5-4. Rust

• All names defined in readxml.rs are put in readxml module
• To use anything in readxml module in another file (main.rs),
  • you put in main.rs

pub use readxml

• you also have to say in readxml.rs that the function readxml is "public" (callable from outside the module)

pub fn readxml(...) { ... }

7. Before you submit ...

• Make sure you remove unnecessary files before you submit this assignment
• If you press submit, it basically tries to submit everything under this directory, including files you created by {go,dune,cargo} build as well as all the libraries you installed or automatically installed
• This causes errors due to too large submissions (> 100MB)
• so make sure you do the following before you submit
  • remove the executable under 00hello/<lang>
  • in each {01build_sys,p1,p2,p3,p4}/<lang>/<project_name>, run {go,dune,cargo} clean; they will remove produced files
• before you submit, run

du -a

and see the total size of the directory.

• make sure the last line is < 1000
• otherwise find a large file in the output of du -a and do an appropriate command from the above