1. Introduction
Scala provides a few built-in ways to interact with the file system and, more generally, with the underlying operating system. These include scala.io to work with files and scala.sys to invoke shell commands. However, they are often difficult to work with.
In this tutorial, we discuss os-lib, an idiomatic and unopinionated Scala library to work with files and processes in Scala as we would in any scripting language. os-lib aims to provide the type safety, performance, and flexibility Scala programmers are used to.
We’ll focus on how we can execute OS commands in Scala using os-lib. First, we’ll take a look at the “traditional” way, i.e., scala.sys. Then, we’ll see how we can achieve the same result in a simpler and more declarative way using os-lib. Lastly, we’ll briefly look at how to use os-lib to work with files.
2. Importing os-lib
To use os-lib, we’ll first have to import it into our project:
libraryDependencies += "com.lihaoyi" %% "os-lib" % "0.9.0"
os-lib comes bundled with Ammonite.
3. Working With Processes
In this section, we’ll see two ways to deal with processes in Scala. The most common use case is to spawn a subprocess and wait for it to complete and, possibly, return a result. We’ll compare two ways of doing so: scala-sys and os-lib.
3.1. Working With Processes Using Native Scala
In the scala.sys package, the Scala Standard Library provides us with a few ways to invoke OS commands.
scala.sys defines two methods to spawn a new subprocess, ! and !!. The former returns the exit code of a subprocess, printing its output. The latter, on the other hand, returns the standard output of the process.
Let’s see an example:
val output = "ls -l".!!
println(output)
// This will print the standard output of the "ls -l" command, even though we didn't ask for it!
val result = Seq("ls", "-l").!
In the example above, we executed a simple command, ls -l, to list the content of the working directory of our application. First, we ran it with the !! method. In this case, the output value will be set to the standard output of the command, for example:
total 208
drwxrwxr-x 3 matteo matteo 4096 nov 15 2021 ammonite
drwxrwxr-x 5 matteo matteo 4096 apr 29 09:28 app-packaging
-rw-rw-r-- 1 matteo matteo 13612 ott 18 13:58 build.sbt
drwxrwxr-x 4 matteo matteo 4096 set 13 10:39 cats-effects
drwxrwxr-x 4 matteo matteo 4096 dic 15 2021 doobie
-rw-rw-r-- 1 matteo matteo 1065 mar 26 2021 LICENSE
In a typical application, we’d have to parse the content of the output variable to extract information from the output of the subprocess.
Secondly, we ran the same command, but this time using !. In this case, result will contain the return code of the subprocess – 0 in case of success. However, ! prints the output of the command on the standard output of our application. This is certainly an annoying side-effect one wouldn’t expect in a Scala method.
There’s another interesting thing to note in the examples above. In scala.sys, we can specify the command to be run in three different ways:
- as a String: “ls -l”
- as a Seq[String]: Seq(“ls”, “-l”), where every option is a separate element of the Seq
- using the Process constructor: Process(“ls -l”)
The ! and !! methods are actually defined on objects of type Process. Hence, if we use a plain String or a Seq[String] to define a command, the Scala compiler applies the implicit conversions stringToProcess() and stringSeqToProcess(), respectively, provided that they were imported.
The ! and !! have two very different error semantics. Let’s consider the following snippet of code, where we list the content of a non-existent directory:
val outputErr = "ls -l dir".!!
val resultErr = "ls -l dir".!
In the former case, the call to !! throws a RuntimeException, printing the error message of the subprocess:
ls: cannot access 'dir': No such file or directory
Exception in thread "main" java.lang.RuntimeException: Nonzero exit value: 2
at scala.sys.package$.error(package.scala:30)
at ...
However, the call to ! won’t throw any exceptions at all. In fact, it would print “ls: cannot access ‘dir’: No such file or directory” and set result to a nonzero exit code.
The simple examples above show why scala.sys might be tricky to use.
3.2. Working With Processes Using os-lib
Let’s see how to spawn a new subprocess and wait for it to complete, using os-lib:
val ls = os.proc("ls", "-l").call()
println(ls.exitCode)
println(ls.out.text())
println(ls.err.text())
*Executing a subprocess in os-lib is as simple as specifying the command via os.proc() and then running it via call().* This returns an object of type os.CommandResult, giving us access to the exit code, the standard output, and the standard error of the command we just ran.
This is much more Scala-like than the scala.sys alternative, where sometimes the standard output of the subprocess would get printed out in the standard output of our application.
In the example above, the println statements would print the following (note the final empty line, corresponding to the standard error):
0
total 208
drwxrwxr-x 3 matteo matteo 4096 nov 15 2021 ammonite
drwxrwxr-x 5 matteo matteo 4096 apr 29 09:28 app-packaging
-rw-rw-r-- 1 matteo matteo 13676 ott 18 14:11 build.sbt
drwxrwxr-x 4 matteo matteo 4096 set 13 10:39 cats-effects
drwxrwxr-x 4 matteo matteo 4096 dic 15 2021 doobie
-rw-rw-r-- 1 matteo matteo 1065 mar 26 2021 LICENSE
os.proc().call() comes with many optional parameters we can use to customize the execution of the subprocess. For example, we can specify a different working directory via the cwd parameter, set environment variables with env, inject data into the input stream and/or redirect standard output and error using stdin, stdout, and stderr.
One of the optional parameters of os.proc().call() is check. When set to true (the default), os-lib throws an exception if the subprocess exits with a non-zero code. For example, assuming that the dir directory doesn’t exist, here’s what os.proc(“ls”, “-l”, “dir”).call() prints:
ls: cannot access 'dir': No such file or directory
Exception in thread "main" os.SubprocessException: CommandResult 2
at os.proc.call(ProcessOps.scala:85)
at com.baeldung.scala.os.OsApp$.delayedEndpoint$com$baeldung$scala$os$OsApp$1(OsApp.scala:9)
at ...
However, if we set check to false, os-lib won’t throw an exception. In this case, it just prints the standard error of the subprocess:
ls: cannot access 'dir': No such file or directory
4. Working With Files
Working with files and file system is one of the most common operations in modern applications. In this section, we’ll see two ways to deal with files in Scala. We’ll compare the Scala way to the os-lib one by looking at how to count the lines of a file.
4.1. Working With Files Using Native Scala
Scala programmers usually work with files using the Source class from the scala.io package. Let’s see, for example, how to count the number of lines in a file:
val source = Source.fromFile("./LICENSE")
val lines = source.getLines().size
source.close()
Here we opened a Source for a given file and used it to count its lines. Nonetheless, Source.fromFile() leaves the file open. Hence, we also have to close the Source explicitly.
In a real-world application, we’d also have to take care of possible exceptions, resorting to a try/catch/finally or to Using (since Scala 2.13). This is quite error-prone.
4.2. Working With Files Using os-lib
In os-lib, the same operation is as simple as using os.read() and giving it a path built with os-lib‘s DSL:
val lines = os.read(os.pwd / "LICENSE").linesIterator.size
With os-lib, we have to deal with far fewer low-level details, such as closing the Source we used to read the file. As a matter of fact, we build a path to a local file using os.pwd, which represents the working directory of the application. Then, we call linesIterator on the String object returned by os.read() to get an Iterator[String], which we use to count the lines in the file.
5. Conclusion
In this article we looked at os-lib and learned how we can use it to execute OS commands. We compared the resulting code with the built-in Scala classes of the scala.sys package and analyzed the pros and cons of each solution. Lastly, we briefly compared os-lib and Scala in another major use case – the interaction with the file system.
As always, the source code for the examples is available over on GitHub.