1. Overview
In imperative programming languages, we use loops such as for-loop and while-loop to iterate over collections. The Scala programming language introduced a new kind of loop: the for-comprehension.
Like many other Scala constructs, the for-comprehension comes directly from Haskell. Its use goes far beyond simply looping over collections, helping us to deal with the complexity of the syntax when using a functional approach to programming.
In this tutorial, we’ll take a deep dive into Scala’s for-comprehension construct.
2. Conventional vs. Declarative Looping in Java
Let’s take a quick tour of looping in Java to see the motivation behind Scala’s for-comprehension.
2.1. Before Java 8
In Scala, we don’t like loops very much. In fact, it’s hard to find a Scala while-loop or for-loop containing some side effects over a collection.
However, in Java, it’s not uncommon to find code that looks like:
final List<TestResult> results =
Arrays.asList(new TestResult("test 1",10, 10), new TestResult("test 2",2, 6));
int totalPassedAssertsInSucceededTests = 0;
for (int i = 0; i < results.size(); i++) {
final TestResult result = results.get(i);
if (result.isSucceeded()) {
totalPassedAssertsInSucceededTests += result.getSuccessfulAsserts();
}
}
In Scala, this style of programming is discouraged in favor of a declarative style that focuses on the transformation to apply to a collection, rather than on how to loop over it.
2.2. Java 8 and Beyond
Java 8 introduced some constructs that developers can use to have a more declarative style of programming. For example, the introduction of the lambda expressions allows programmers to give a function or a procedure to apply to every element of a collection:
final long totalPassedAssertsInSucceededTests1 = results.stream()
.filter(TestResult::isSucceeded)
.mapToInt(TestResult::getSuccessfulAsserts)
.sum();
However, without any further facility to compose subsequent steps of iteration, the situation degrades quickly, despite the improved code readability and maintainability. It’s easy to find ourselves in situations where the code starts to be obscure:
results.stream().flatMap(
res -> f(res).flatMap(
res1 -> res1.flatMap(
res2 -> res2.map( /* and so on */ ))));
So, let’s see how Scala approaches a solution to this style of programming.
3. Declarative Looping in Scala: the For-Comprehension Structure
The for-comprehension is the Scala way to manage collections using a purely declarative style:
val listOfPassedAssertsInSucceededTests: List[Int] =
for {
result <- results
if result.succeeded
} yield (result.successfulAsserts)
val passedAssertsInSucceededTests: Int = listOfPassedAssertsInSucceededTests.sum
We can identify different constructs that, when taken together, form the for-comprehension syntax. This might look daunting, but let’s walk through this code and take each component step by step.
The for-comprehension in Scala has the form for (enumerators) yield e. The enumerators are the collective code that goes inside the parentheses that follow the for keyword. Enumerators bind values to variables. The for-comprehension body e evaluates for every value generated by enumerators, creating a sequence of such values.
We’ll take a closer look at each component of the for-comprehension in the next sections.
4. Enumerators
An enumerator can be either a generator or a filter. In our previous example, we have a for-comprehension containing both types of enumerators. Let’s take a closer look at each type.
4.1. Generators
The statement result <- results represent a generator. It introduces a new variable, result, that loops over each value of the variable results. So, the type of result is TestResult.
We can have as many generators as we want. They loop independently from each other, producing all the possible combinations of their variables. In the example, we loop over the list of results and the list of execution times. Then, we merge the two elements, listing the total number of asserts executed for each test result, along with the execution time:
val executionTimeList = List(("test 1", 100), ("test 2", 230))
val numberOfAssertsWithExecutionTime: List[(String, Int, Int)] =
for {
result <- results
(id, time) <- executionTimeList
if result.id == id
} yield ((id, result.totalAsserts, time))
The values contained in the numberOfAssertsWithExecutionTime list are:
List[("test 1", 10, 100), ("test 2", 6, 230)]
All the generators inside a for-comprehension must share the same type they loop over. In our previous example, both were instances of List. The type variable does not count. So, we can mix a generator of type List[TestResult] with a generator of type List[(String, Int)].
4.2. Filters
Inside a for-comprehension, filters have the form if boolean-condition. A filter acts as a guard that blocks all the values that do not respect the boolean condition.
Inside a filter, we can create a custom boolean condition using every variable that is available in the scope of the for-comprehension.
In the previous examples, we used the variables declared by the generators. However, we can also use variables declared outside the for-comprehension. Let’s see an example:
val hugeNumberOfAssertsForATest: Int = 10
val resultsWithAHugeAmountOfAsserts: List[TestResult] =
for {
result <- results
if result.totalAsserts >= hugeNumberOfAssertsForATest
} yield (result)
5. The For-Comprehension Body
As we said, the for-comprehension body evaluates for every value generated by enumerators, creating a sequence of such values. Inside the body, we can use any variable or value that is in the scope of the for-comprehension:
val magic: Int = 42
for {
res <- result
} yield res * magic
The type of yield body can be anything we want. Until now, our examples return something as a result of the for-comprehension. However, it’s feasible to return nothing, using an expression that evaluates to Unit. For example, we can use the yield body to print the variables bound by the generators:
for {
res <- result
} println(s"The result is $res")
In the case where the yield body evaluates to Unit, it is possible to omit the yield keyword.
6. For-Comprehension: Deep Dive
In the previous example, we saw how the semantics of a for**-comprehension are equal to that of a sequence of operations on streams or sequences. In Scala, the for-comprehension is nothing more than syntactic sugar to a sequence of calls to one or more of the methods:
- foreach
- map
- flatMap
- withFilter
We can use for-comprehension syntax on every type that defines such methods. Let’s see an example.
First of all, let’s define a class to work on. Let’s create a Result class that is a wrapper around an expression result:
case class Result[A](result: A)
First things first, we’ll try to print to standard output the value of a Result using a for-comprehension:
val result: Result[Int] = Result(42)
for {
res <- result
} println(res)
The compiler warns us that we cannot use the variable res in the for-comprehension:
Value foreach is not a member of com.baeldung.scala.forcomprehension.ForComprehension.Result[Int].
So, the Scala compiler desugars the above construct to a call to the foreach method. Let’s add it to the Result type:
def foreach(f: A => Unit): Unit = f(result)
Now, it’s time to try to give to the yield body some dignity. For example, we modify the result applying some function:
for {
res <- result
} yield res * 2
This time, the compiler gives us the following error:
Value map is not a member of com.baeldung.scala.forcomprehension.ForComprehension.Result[Int].
The Scala compiler is trying to desugar the yield body calling the map method. Let’s define such a method:
def map[B](f: A => B): Result[B] = Result(f(result))
We very much like our brand new Result type. We want to combine more than one result using many generators in a for-comprehension. Let’s do it:
val anotherResult: Result = 100
for {
res <- result
another <- anotherResult
} yield res + another
The methods we defined until now are not enough for the Scala compiler, which shows us a new error:
Value flatMap is not a member of com.baeldung.scala.forcomprehension.ForComprehension.Result[Int].
We still can’t understand why we need a flatMap method, but we add diligently what the compiler asks us:
def flatMap[B](f: A => Result[B]): Result[B] = f(result)
But, why do we need to define a flatMap method to use the for-comprehension? Because the desugared version of the above for-comprehension is:
result
.flatMap(res =>
anotherResult
.map(another => res + another)
)
The last operation we want to do with our Result type is to use a filter inside a for-comprehension. First, we need to define a value that represents an empty result:
object EmptyResult extends Result[Null](null)
Let’s try to use the filter:
for {
res <- result
if res == 10
} yield res
As usual, the compiler warns us that a method is missing:
Value withFilter is not a member of com.baeldung.scala.forcomprehension.ForComprehension.Result[Int].
Again, we define the missing method:
def withFilter(f: A => Boolean): Result[_] = if (f(result)) this else EmptyResult
In this case, we force the implementation a little bit. We do not return an instance of Result[A], because our empty result is an instance of Result[Null] and not an instance of Result[Nothing].
7. Conclusion
In this article, we reviewed the for-comprehension construct that is available in the Scala programming language.
We showed the difference between the declarative and the imperative style of looping. Then, we analyzed how a for-comprehension is made, describing the main features of both the enumerators and the yield body.
Finally, we discovered that a for-comprehension is just syntactic sugar for a sequence of calls to methods foreach, map, flatMap, and withFilter.
As usual, all the code implementations are available over on GitHub.