1. Overview
The Spliterator interface, introduced in Java 8, can traverse and partition sequences. It’s a base utility for Streams, especially parallel ones.
In this article, we’ll cover its usage, characteristics, methods and how to create our own custom implementations.
2. Spliterator API
2.1. tryAdvance
This is the main method used for stepping through a sequence. The method takes a Consumer that’s used to consume elements of the Spliterator one by one sequentially and returns false if there’re no elements to be traversed.
Here, we’ll look at how to use it to traverse and partition elements.
First, let’s assume that we’ve got an ArrayList with 35000 articles and that the Article class is defined as:
public class Article {
private List<Author> listOfAuthors;
private int id;
private String name;
// standard constructors/getters/setters
}
Now, let’s use Spliterator to process the list of articles and adds a suffix of “– published by Baeldung” to each article name:
@Test
public void givenAStreamOfArticles_whenProcessedInSequentiallyWithSpliterator_ProducessRightOutput() {
// ...
}
First, let’s generate the articles:
public void givenAStreamOfArticles_whenProcessedInSequentiallyWithSpliterator_ProducessRightOutput() {
List<Article> articles = Stream.generate(() -> new Article("Java"))
.limit(35000)
.collect(Collectors.toList());
// ...
}
We have used Stream to generate 35000 articles. Next, let’s create a spliterator from this articles list and use the tryAdvance method to process the articles.
Spliterator<Article> spliterator = articles.spliterator();
while (spliterator.tryAdvance(article -> article.setName(article.getName()
.concat("- published by Baeldung"))));
Here, our consumer is a simple function that adds a suffix to the article names.
Finally, we can do an assertion to verify if all articles were processed and their name was updated:
articles.forEach(article -> assertThat(article.getName()).isEqualTo("Java- published by Baeldung"));
Notice that this test case will execute successfully. All article names are already updated, and the new name is equal to Java- published by Baeldung.
Another key point is that we used the tryAdvance() method to process the next element.
2.2. trySplit
Next, let’s split Spliterators (hence the name) and process partitions independently.
The trySplit method tries to split it into two parts. Then the caller process elements, and finally, the returned instance will process the others, allowing the two to be processed in parallel.
We will generate our articles and spliterator as we did previously:
@Test
public void givenAStreamOfArticle_whenProcessedUsingTrySplit_thenSplitIntoEqualHalf() {
List<Article> articles = Stream.generate(() -> new Article("Java"))
.limit(35000)
.collect(Collectors.toList());
Spliterator<Article> split1 = articles.spliterator();
// ...
}
Then we create our second spliterator by applying the trySplit method on the first one:
Spliterator<Article> split2 = split1.trySplit();
Now let’s check the example of using these two splits; let’s create two lists that will store the results processed by these spliterators:
List<Article> articlesListOne = new ArrayList<>();
List<Article> articlesListTwo = new ArrayList<>();
Let’s consume the articles:
split1.forEachRemaining(articlesListOne::add);
split2.forEachRemaining(articlesListTwo::add);
After creating the list, we iterate through split1 and add all the articles in split1 to articlesListOne. Similarly, we perform the same operation for split2, saving each article of split2 into articlesListTwo.
Next, we can assert that these spliterators consumed exactly half of the articles, i.e. 17500:
assertThat(articlesListOne.size()).isEqualTo(17500);
assertThat(articlesListTwo.size()).isEqualTo(17500);
Finally, we can make an assertion to verify that both lists contain distinct elements:
assertThat(articlesLitOne).doesNotContainAnyElementsOf(articlesListTwo);
Notice that this test case will execute successfully. As the articles that are present in the articlesSplitOne are not present in articlesSplitTwo. This concludes we can process the partitions independently.
The splitting process worked as intended and divided the records equally.
2.3. estimatedSize
The estimatedSize method gives us an estimated number of elements:
log.info("Size: " + split1.estimateSize());
This will output:
Size: 17500
2.4. hasCharacteristics
This API checks if the given characteristics match the properties of the Spliterator. Then if we invoke the method above, the output will be an int representation of those characteristics:
log.info("Characteristics: " + split1.characteristics());
Characteristics: 16464
3. Spliterator Characteristics
It has eight different characteristics that describe its behaviour. Those can be used as hints for external tools:
- SIZED – if it’s capable of returning an exact number of elements with the estimateSize() method
- SORTED – if it’s iterating through a sorted source
- SUBSIZED – if we split the instance using a trySplit() method and obtain Spliterators that are SIZED as well
- CONCURRENT – if the source can be safely modified concurrently
- DISTINCT – if for each pair of encountered elements x, y, !x.equals(y)
- IMMUTABLE – if elements held by the source can’t be structurally modified
- NONNULL – if the source holds nulls or not
- ORDERED – if iterating over an ordered sequence
4. A Custom Spliterator
4.1. When to Customize
First, let’s assume the following scenario:
We’ve got an article class with a list of authors, and the article that can have more than one author. Furthermore, we consider an author related to the article if his related article’s id matches article id.
Our Author class will look like the this:
public class Author {
private String name;
private int relatedArticleId;
// standard getters, setters & constructors
}
Next, we’ll implement a class to count authors while traversing a stream of authors. Then the class will perform a reduction on the stream.
Let’s have a look at the class implementation:
public class RelatedAuthorCounter {
private int counter;
private boolean isRelated;
// standard constructors/getters
public RelatedAuthorCounter accumulate(Author author) {
if (author.getRelatedArticleId() == 0) {
return isRelated ? this : new RelatedAuthorCounter( counter, true);
} else {
return isRelated ? new RelatedAuthorCounter(counter + 1, false) : this;
}
}
public RelatedAuthorCounter combine(RelatedAuthorCounter RelatedAuthorCounter) {
return new RelatedAuthorCounter(
counter + RelatedAuthorCounter.counter,
RelatedAuthorCounter.isRelated);
}
}
Each method in the above class performs a specific operation to count while traversing.
First, the accumulate() method traverse the authors one by one in an iterative way, then combine() sums two counters using their values. Finally, the getCounter() returns the counter.
Now, to test what we’ve done so far. Let’s convert our article’s list of authors to a stream of authors:
Stream<Author> stream = article.getListOfAuthors().stream();
And implement a countAuthor()* method to perform the reduction on the stream using *RelatedAuthorCounter:
private int countAutors(Stream<Author> stream) {
RelatedAuthorCounter wordCounter = stream.reduce(
new RelatedAuthorCounter(0, true),
RelatedAuthorCounter::accumulate,
RelatedAuthorCounter::combine);
return wordCounter.getCounter();
}
If we used a sequential stream the output will be as expected “count = 9”, however, the problem arises when we try to parallelize the operation.
Let’s take a look at the following test case:
@Test
void
givenAStreamOfAuthors_whenProcessedInParallel_countProducesWrongOutput() {
assertThat(Executor.countAutors(stream.parallel())).isGreaterThan(9);
}
Apparently, something has gone wrong – splitting the stream at a random position caused an author to be counted twice.
4.2. How to Customize
To solve this, we need to implement a Spliterator that splits authors only when related id and articleId matches. Here’s the implementation of our custom Spliterator:
public class RelatedAuthorSpliterator implements Spliterator<Author> {
private final List<Author> list;
AtomicInteger current = new AtomicInteger();
// standard constructor/getters
@Override
public boolean tryAdvance(Consumer<? super Author> action) {
action.accept(list.get(current.getAndIncrement()));
return current.get() < list.size();
}
@Override
public Spliterator<Author> trySplit() {
int currentSize = list.size() - current.get();
if (currentSize < 10) {
return null;
}
for (int splitPos = currentSize / 2 + current.intValue();
splitPos < list.size(); splitPos++) {
if (list.get(splitPos).getRelatedArticleId() == 0) {
Spliterator<Author> spliterator
= new RelatedAuthorSpliterator(
list.subList(current.get(), splitPos));
current.set(splitPos);
return spliterator;
}
}
return null;
}
@Override
public long estimateSize() {
return list.size() - current.get();
}
@Override
public int characteristics() {
return CONCURRENT;
}
}
Now applying countAuthors() method will give the correct output. The following code demonstrates that:
@Test
public void
givenAStreamOfAuthors_whenProcessedInParallel_countProducesRightOutput() {
Stream<Author> stream2 = StreamSupport.stream(spliterator, true);
assertThat(Executor.countAutors(stream2.parallel())).isEqualTo(9);
}
Also, the custom Spliterator is created from a list of authors and traverses through it by holding the current position.
Let’s discuss in more details the implementation of each method:
- tryAdvance – passes authors to the Consumer at the current index position and increments its position
- trySplit – defines the splitting mechanism, in our case, the RelatedAuthorSpliterator is created when ids matched, and the splitting divides the list into two parts
- estimatedSize – is the difference between the list size and the position of currently iterated author
- characteristics – returns the Spliterator characteristics, in our case SIZED as the value returned by the estimatedSize() method is exact; moreover, CONCURRENT indicates that the source of this Spliterator may be safely modified by other threads
5. Support for Primitive Values
The Spliterator* API supports primitive values including double, int and *long.
The only difference between using a generic and a primitive dedicated Spliterator is the given Consumer and the type of the Spliterator.
For example, when we need it for an int value we need to pass an intConsumer. Furthermore, here’s a list of primitive dedicated Spliterators:
- OfPrimitive<T, T_CONS, T_SPLITR extends Spliterator.OfPrimitive<T, T_CONS, T_SPLITR>>: parent interface for other primitives
- OfInt: A Spliterator specialized for int
- OfDouble: A Spliterator dedicated for double
- OfLong: A Spliterator dedicated for long
6. Conclusion
In this article, we covered Java 8 Spliterator usage, methods, characteristics, splitting process, primitive support and how to customize it.
As always, the full implementation of this article can be found over on Github.