![Page 1: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/1.jpg)
1
Странности Stream API
Тагир ВалеевИнститут систем информатики СО РАН
![Page 2: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/2.jpg)
2
Что это за чувак на сцене?https://github.com/amaembo/streamex
![Page 3: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/3.jpg)
3
Что это за чувак на сцене?• JDK-8072727 Add variation of Stream.iterate() that's finite• JDK-8136686 Collectors.counting can use Collectors.summingLong to reduce boxing• JDK-8141630 Specification of Collections.synchronized* need to state traversal constraints• JDK-8145007 Pattern splitAsStream is not late binding as required by the specification• JDK-8146218 Add LocalDate.datesUntil method producing Stream<LocalDate>• JDK-8147505 BaseStream.onClose() should not allow registering new handlers after stream is
consumed• JDK-8148115 Stream.findFirst for unordered source optimization• JDK-8148250 Stream.limit() parallel tasks with ordered non-SUBSIZED source should short-circuit• JDK-8148838 Stream.flatMap(...).spliterator() cannot properly split after tryAdvance()• JDK-8148748 ArrayList.subList().spliterator() is not late-binding• JDK-8151123 Collectors.summingDouble/averagingDouble unnecessarily call mapper twice
![Page 4: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/4.jpg)
4
Что это за чувак на сцене?
![Page 5: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/5.jpg)
5
LongStream.range(1, 100) .count();
![Page 6: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/6.jpg)
6
LongStream.range(1, 100) .count();>> 99
![Page 7: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/7.jpg)
7
LongStream.range(0, 1_000_000_000_000_000_000L) .count();
![Page 8: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/8.jpg)
8
LongStream.range(0, 1_000_000_000_000_000_000L) .count();
![Page 9: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/9.jpg)
9
LongStream.range(0, 1_000_000_000_000_000_000L) .count();>> 1000000000000000000
Java 9:JDK-8067969 Optimize Stream.count for SIZED Streams
![Page 10: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/10.jpg)
10
Характеристики
SIZEDSUBSIZEDSORTEDORDERED
DISTINCTNONNULLIMMUTABLECONCURRENT
![Page 11: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/11.jpg)
11
toArray()IntStream.range(0, 100_000_000) .toArray();
IntStream.range(0, 100_000_000) .filter(x -> true) .toArray();
![Page 12: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/12.jpg)
12
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at java.util.stream.SpinedBuffer$OfInt.newArray at java.util.stream.SpinedBuffer$OfInt.newArray at java.util.stream.SpinedBuffer$OfPrimitive.asPrimitiveArray at java.util.stream.Nodes$IntSpinedNodeBuilder.asPrimitiveArray at java.util.stream.Nodes$IntSpinedNodeBuilder.asPrimitiveArray at java.util.stream.IntPipeline.toArray at ru.javapoint.streamsamples.ToArray.main
![Page 13: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/13.jpg)
13
toArray()
IntStream.range(0, 100_000_000) -Xmx560M 269ms .toArray();
IntStream.range(0, 100_000_000) -Xmx1330M 795ms .filter(x -> true) .toArray();
![Page 14: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/14.jpg)
14
.collect(toList())?JDK-8072840 Add a method to Collector that returns a sized supplying mutable result container
IntStream.range(0, 10_000_000) -Xmx320M 2.64±0.5s .boxed().collect(toList());
IntStream.range(0, 10_000_000) -Xmx320M 2.63±0.5s .boxed().filter(x -> true) .collect(toList());
![Page 15: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/15.jpg)
15
sorted()full-barrier operation
IntStream.range(0, 100_000_000) -Xmx560M .toArray(); IntStream.range(0, 100_000_000) -Xmx1330M .filter(x -> true) .toArray();
IntStream.range(0, 100_000_000) ? .sorted().sum(); IntStream.range(0, 100_000_000) ? .filter(x -> true) .sorted().sum();
![Page 16: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/16.jpg)
16
sorted()full-barrier operation
IntStream.range(0, 100_000_000) -Xmx560M .toArray(); IntStream.range(0, 100_000_000) -Xmx1330M .filter(x -> true) .toArray();
IntStream.range(0, 100_000_000) -Xmx1M .sorted().sum(); IntStream.range(0, 100_000_000) -Xmx1M .filter(x -> true) .sorted().sum();
![Page 17: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/17.jpg)
17
sorted()full-barrier operation
IntStream.range(0, 100_000_000) -Xmx580M .map(x -> x).sorted().sum(); IntStream.range(0, 100_000_000) -Xmx1330M .filter(x -> true) .map(x -> x).sorted().sum();
IntStream.range(0, 100_000_000) -Xmx560M .toArray(); IntStream.range(0, 100_000_000) -Xmx1330M .filter(x -> true) .toArray();
![Page 18: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/18.jpg)
18
skip()IntStream.range(0, 100_000_000) .toArray();
IntStream.range(0, 100_000_000) .skip(1) .toArray();
![Page 19: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/19.jpg)
19
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at java.util.stream.SpinedBuffer$OfInt.newArray at java.util.stream.SpinedBuffer$OfInt.newArray at java.util.stream.SpinedBuffer$OfPrimitive.asPrimitiveArray at java.util.stream.Nodes$IntSpinedNodeBuilder.asPrimitiveArray at java.util.stream.Nodes$IntSpinedNodeBuilder.asPrimitiveArray at java.util.stream.IntPipeline.toArray at ru.javapoint.streamsamples.ToArraySkip.main
![Page 20: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/20.jpg)
20
skip() и limit()IntStream.range(0, 100_000_000) .toArray();
IntStream.range(0, 100_000_000) .skip(1) .toArray();
IntStream.range(0, 100_000_000) .limit(99_999_999) .toArray();
![Page 21: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/21.jpg)
21
skip() и limit()new Random().ints() .limit(100_000_000) .toArray();
new Random().ints(100_000_000) .toArray();
![Page 22: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/22.jpg)
22
skip() и limit()list.stream() .limit(2000) .skip(1000) .forEach(System.out::println);
list.subList(1000, 2000) .stream() .forEach(System.out::println);
![Page 23: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/23.jpg)
23
parallel().skip()IntStream.range(0, 100_000_000) 104.5±4.4 ms .skip(99_000_000) .sum();
IntStream.range(0, 100_000_000) ? .parallel() .skip(99_000_000) .sum();
![Page 24: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/24.jpg)
24
![Page 25: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/25.jpg)
25
parallel().skip()API Note:While skip() is generally a cheap operation on sequential stream pipelines, it can be quite expensive on ordered parallel pipelines, especially for large values of n, since skip(n) is constrained to skip not just any n elements, but the first n elements in the encounter order.
![Page 26: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/26.jpg)
26
parallel().skip()IntStream.range(0, 100_000_000) 104.5±4.4 ms .skip(99_000_000) .sum();
IntStream.range(0, 100_000_000) 1.4±0.2 ms .parallel() (74.6×) .skip(99_000_000) .sum();
![Page 27: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/27.jpg)
27
parallel(): trySplit() (SIZED+SUBSIZED)
0..99_999_999
0..49_999_999 50_000_000..99_999_999
75_000_000..99_999_999
50_000_000..74_999_999
…………
25_000_000..49_999_999
……
0..24_999_999
……
![Page 28: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/28.jpg)
28
parallel().skip()(SIZED+SUBSIZED)
0..99_999_999
0..49_999_999 50_000_000..99_999_999
75_000_000..99_999_99950_000_000..74_999_999
……
![Page 29: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/29.jpg)
29
Характеристики
SIZEDSUBSIZEDSORTEDORDERED
DISTINCTNONNULLIMMUTABLECONCURRENT
![Page 30: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/30.jpg)
30
distinct() и orderingList<Integer> input = new Random(1) .ints(10_000_000, 0, 10) .boxed().collect(Collectors.toList());
input.stream() .distinct() .collect(Collectors.toList());
85.4 ± 0.7 ms
![Page 31: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/31.jpg)
31
distinct() и orderinginput.stream() .parallel() .distinct() .collect(Collectors.toList());
30.5 ± 1.7 ms (2.8×)
![Page 32: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/32.jpg)
32
parallel().distinct()API Note:Preserving stability for distinct() in parallel pipelines is relatively expensive (requires that the operation act as a full barrier, with substantial buffering overhead), and stability is often not needed. Using an unordered stream source (such as generate(Supplier)) or removing the ordering constraint with BaseStream.unordered() may result in significantly more efficient execution for distinct() in parallel pipelines, if the semantics of your situation permit.
![Page 33: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/33.jpg)
33
distinct() и orderinginput.stream() .parallel() .unordered() .distinct() .collect(Collectors.toList());
![Page 34: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/34.jpg)
34
distinct() и ordering
Sequential 85.4 ± 0.7 msParallel 30.5 ± 1.7 ms (2.8×)Parallel unordered 249.0 ± 1.5 ms (0.34×)
![Page 35: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/35.jpg)
35
LinkedHashSet
LinkedHashSet LinkedHashSet
distinct() ordered parallel
LinkedHashSet LinkedHashSet LinkedHashSet LinkedHashSet
![Page 36: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/36.jpg)
36
ConcurrentHashMap.newKeySet()
distinct() unordered parallel
![Page 37: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/37.jpg)
37
Характеристики
SIZEDSUBSIZEDSORTEDORDERED
DISTINCTNONNULLIMMUTABLECONCURRENT
![Page 38: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/38.jpg)
38
Stream.concat()IntStream s1 = IntStream.range(0, 50_000_000);IntStream s2 = IntStream.range(50_000_000, 100_000_000);
IntStream.concat(s1, s2).toArray(); -Xmx580M
![Page 39: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/39.jpg)
39
Промежуточные операции(красивые)
map(), filter(), flatMap(), …
Терминальные операции(умные)
forEach(), reduce(), count(), …
А мне что, разорваться?
Stream.concat()
![Page 40: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/40.jpg)
40
concat() и parallel()Integer[] aIntegers = { 101, 102, ..., 200 };Integer[] bIntegers = { 201, 202, ..., 300 };
Stream<Integer> a = Arrays.stream(aIntegers);Stream<Integer> b = Arrays.stream(bIntegers);List<String> list = Stream.concat(a, b) .map(num -> String.valueOf(num).replaceAll("((((.*)*)*)*)*!", "")) .collect(Collectors.toList());
868.9 ± 10.6 ms
![Page 41: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/41.jpg)
41
concat() и parallel()Integer[] aIntegers = { 101, 102, ..., 200 };Integer[] bIntegers = { 201, 202, ..., 300 };
Stream<Integer> a = Arrays.stream(aIntegers);Stream<Integer> b = Arrays.stream(bIntegers);List<String> list = Stream.concat(a, b) .parallel() .map(num -> String.valueOf(num).replaceAll("((((.*)*)*)*)*!", "")) .collect(Collectors.toList());
227.9 ± 3.5 ms (3.8×)
![Page 42: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/42.jpg)
42
concat() и parallel()Stream<Integer> a = Arrays.stream(aIntegers);Stream<Integer> b = Arrays.stream(bIntegers);List<String> list = Stream.concat(a, b) .parallel() 227.9 .map(num -> String.valueOf(num).replaceAll(...)) ± 3.5 ms .collect(Collectors.toList()); (3.8×)
Stream<Integer> a = Arrays.stream(aInts).boxed();Stream<Integer> b = Arrays.stream(bInts).boxed();List<String> list = Stream.concat(a, b) .parallel() ? .map(num -> String.valueOf(num).replaceAll(...)) .collect(Collectors.toList());
![Page 43: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/43.jpg)
43
concat() и parallel()Stream<Integer> a = Arrays.stream(aIntegers);Stream<Integer> b = Arrays.stream(bIntegers);List<String> list = Stream.concat(a, b) .parallel() 227.9 .map(num -> String.valueOf(num).replaceAll(...)) ± 3.5 ms .collect(Collectors.toList()); (3.8×)
Stream<Integer> a = Arrays.stream(aInts).boxed();Stream<Integer> b = Arrays.stream(bInts).boxed();List<String> list = Stream.concat(a, b) .parallel() 437.8 .map(num -> String.valueOf(num).replaceAll(...)) ± 5.2 ms .collect(Collectors.toList()); (1.98×)
![Page 44: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/44.jpg)
44
concat()1. Пусть A = aStream.spliterator(), B = bStream.spliterator()2. Создать новый spliterator из этих двух:• tryAdvance: вызывать A.tryAdvance(),
а если там кончилось, то B.tryAdvance().• forEachRemaining: вызвать A.forEachRemaining(),
затем B.forEachRemaining().• trySplit: разделить назад на A и B.
3. Создать новый stream по новому сплитератору.4. Повесить на onClose вызов aStream.close() и
bStream.close()
![Page 45: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/45.jpg)
45
Stream.spliterator()1. Нет промежуточных операций? Вернём исходный сплитератор.2. Есть промежуточные операции? Создадим новый WrappingSpliterator:
• forEachRemaining(): ≈ Stream.forEach()• tryAdvance(): вызвать tryAdvance у источника и собрать в буфер,
что накопилось, а потом идти по буферу• trySplit(): если исходный стрим параллельный, вызвать trySplit() у
источника и обернуть результат в такой же WrappingSpliterator
![Page 46: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/46.jpg)
46
concat() и parallel()Stream<Integer> a = Arrays.stream(aIntegers);Stream<Integer> b = Arrays.stream(bIntegers);List<String> list = Stream.concat(a, b) .parallel() 227.9 .map(num -> String.valueOf(num).replaceAll(...)) ± 3.5 ms .collect(Collectors.toList()); (3.8×)
Stream<Integer> a = Arrays.stream(aInts).boxed();Stream<Integer> b = Arrays.stream(bInts).boxed();List<String> list = Stream.concat(a, b) .parallel() 437.8 .map(num -> String.valueOf(num).replaceAll(...)) ± 5.2 ms .collect(Collectors.toList()); (1.98×)
![Page 47: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/47.jpg)
47
concat() и parallel()Stream<Integer> a = Arrays.stream(aInts).boxed().parallel();Stream<Integer> b = Arrays.stream(bInts).boxed().parallel();List<String> list = Stream.concat(a, b) .map(num -> String.valueOf(num).replaceAll(...)) .collect(Collectors.toList());
222.1 ± 2.3 ms (3.9×)
![Page 48: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/48.jpg)
48
concat() и orderingList<Integer> a = Arrays.asList(1,2,3,4,5,6,7,8,9,10);List<Integer> b = Arrays.asList();Stream.concat(a.parallelStream(), b.parallelStream()) .filter(x -> x % 2 == 0) .limit(3) .forEachOrdered(System.out::println);>> 2>> 4>> 6
![Page 49: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/49.jpg)
49
concat() и orderingList<Integer> a = Arrays.asList(1,2,3,4,5,6,7,8,9,10);List<Integer> b = Collections.emptyList();Stream.concat(a.parallelStream(), b.parallelStream()) .filter(x -> x % 2 == 0) .limit(3) .forEachOrdered(System.out::println);>> 2>> 6>> 10
![Page 50: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/50.jpg)
50
flatMap()Stream<Integer> a = ...;Stream<Integer> b = ...;Stream<Integer> c = ...;Stream<Integer> d = ...;Stream<Integer> res = Stream.concat( Stream.concat(Stream.concat(a, b), c), d);
Stream<Integer> res = Stream.of(a, b, c, d) .flatMap(Function.identity());
![Page 51: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/51.jpg)
51
concat() или flatMap()?Stream<Integer> a = Arrays.stream(aInts).boxed().parallel();Stream<Integer> b = Arrays.stream(bInts).boxed().parallel();List<String> list = Stream.of(a, b) .flatMap(Function.identity()) .parallel() .map(num -> String.valueOf(num).replaceAll(...)) .collect(Collectors.toList()); // 444.8 ± 7.3 ms (1.95x)
![Page 52: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/52.jpg)
52
concat() или flatMap()?IntStream s1 = IntStream.range(0, 50_000_000);IntStream s2 = IntStream.range(50_000_000, 100_000_000);
IntStream.concat(s1, s2).toArray(); // -Xmx580M
Stream.of(s1, s2).flatMapToInt(Function.identity()) .toArray(); // -Xmx1330M
![Page 53: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/53.jpg)
53
flatMap() и short-circuitingIntStream s1 = IntStream.range(0, 50_000_000);IntStream s2 = IntStream.range(50_000_000, 100_000_000);
IntStream.concat(s1, s2) .filter(x -> x > 2).findFirst(); // 0.13 μs
Stream.of(s1, s2).flatMapToInt(Function.identity()) .filter(x -> x > 2).findFirst(); // 301051 μs
![Page 54: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/54.jpg)
54
flatMap() и tryAdvance()Stream<Integer> s = IntStream.range(0, 1_000_000_000).boxed();s.spliterator().tryAdvance(System.out::println);>> 0
![Page 55: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/55.jpg)
55
flatMap() и tryAdvance()Stream<Integer> s = IntStream.of(1_000_000_000) .flatMap(x -> IntStream.range(0, x)).boxed();s.spliterator().tryAdvance(System.out::println);
![Page 56: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/56.jpg)
56
java.lang.OutOfMemoryError: Java heap space at java.util.stream.SpinedBuffer.ensureCapacity at java.util.stream.SpinedBuffer.increaseCapacity at java.util.stream.SpinedBuffer.accept at java.util.stream.IntPipeline$4$1.accept at java.util.stream.IntPipeline$7$1.lambda$accept$198 at java.util.stream.IntPipeline$7$1$$Lambda$7/1831932724.accept at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining at java.util.stream.IntPipeline$Head.forEach at java.util.stream.IntPipeline$7$1.accept at java.util.stream.Streams$IntStreamBuilderImpl.tryAdvance at java.util.Spliterator$OfInt.tryAdvance ... at java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance at ru.javapoint.streamsamples.FlatMapTryAdvance.main
![Page 57: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/57.jpg)
57
flatMap() и tryAdvance()Stream<Integer> s = IntStream.of(1_000_000_000) .flatMap(x -> IntStream.range(0, x)).boxed();s.spliterator().tryAdvance(System.out::println);
![Page 58: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/58.jpg)
58
flatMap() и concat()IntStream s = IntStream.of(1_000_000_000) .flatMap(x -> IntStream.range(0, x));
s.sum();
s.findFirst(); // non short-circuit
IntStream.concat(s, IntStream.of(1)).sum();
IntStream.concat(s, IntStream.of(1)).findFirst();
![Page 59: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/59.jpg)
59
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at java.util.stream.SpinedBuffer$OfInt.newArray at java.util.stream.SpinedBuffer$OfInt.newArray at java.util.stream.SpinedBuffer$OfPrimitive.ensureCapacity at java.util.stream.SpinedBuffer$OfPrimitive.increaseCapacity at java.util.stream.SpinedBuffer$OfPrimitive.preAccept at java.util.stream.SpinedBuffer$OfInt.accept ... at java.util.stream.Streams$ConcatSpliterator$OfInt.tryAdvance at java.util.stream.IntPipeline.forEachWithCancel at java.util.stream.AbstractPipeline.copyIntoWithCancel at java.util.stream.AbstractPipeline.copyInto at java.util.stream.AbstractPipeline.wrapAndCopyInto at java.util.stream.FindOps$FindOp.evaluateSequential at java.util.stream.AbstractPipeline.evaluate at java.util.stream.IntPipeline.findFirst at ru.javapoint.streamsamples.ConcatFlat.main
![Page 60: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/60.jpg)
60
flatMap() vs concat()concat() flatMap()
Сохраняет SIZED
Short-circuiting
Memory-friendly tryAdvance()
Полноценный параллелизм
Всегда сохраняет порядок
Многократная конкатенация
±
![Page 61: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/61.jpg)
61
Всё же concat()?Object[][] data = new Object[20000][4000];
Object[] flat = Arrays.stream(data) .flatMap(Arrays::stream).toArray();>> java.lang.OutOfMemoryError
![Page 62: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/62.jpg)
62
Всё же concat()?Object[][] data = new Object[20000][4000];
Object[] flat = Arrays.stream(data) .flatMap(Arrays::stream).toArray();>> java.lang.OutOfMemoryError
Stream<Object> s = Arrays.stream(data) .map(Arrays::stream).reduce(Stream::concat) .orElse(Stream.empty());Object[] flat = s.toArray();
![Page 63: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/63.jpg)
63
Exception in thread "main" java.lang.StackOverflowError at java.util.stream.Streams$ConcatSpliterator.forEachRemaining at java.util.stream.Streams$ConcatSpliterator.forEachRemaining at java.util.stream.Streams$ConcatSpliterator.forEachRemaining at java.util.stream.Streams$ConcatSpliterator.forEachRemaining at java.util.stream.Streams$ConcatSpliterator.forEachRemaining at java.util.stream.Streams$ConcatSpliterator.forEachRemaining at java.util.stream.Streams$ConcatSpliterator.forEachRemaining at java.util.stream.Streams$ConcatSpliterator.forEachRemaining at java.util.stream.Streams$ConcatSpliterator.forEachRemaining at java.util.stream.Streams$ConcatSpliterator.forEachRemaining at java.util.stream.Streams$ConcatSpliterator.forEachRemaining at java.util.stream.Streams$ConcatSpliterator.forEachRemaining at java.util.stream.Streams$ConcatSpliterator.forEachRemaining at java.util.stream.Streams$ConcatSpliterator.forEachRemaining at java.util.stream.Streams$ConcatSpliterator.forEachRemaining ...
![Page 64: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/64.jpg)
64
reduce(Stream::concat)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
![Page 65: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/65.jpg)
65
Распараллелим!Object[][] data = new Object[20000][4000];
Stream<Object> s = Arrays.stream(data) .parallel() .map(Arrays::stream) .reduce(Stream::concat) .orElse(Stream.empty());Object[] flat = s.toArray();
![Page 66: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/66.jpg)
66
parallel().reduce(Stream::concat)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
.reduce(Predicate::or)
.reduce(Predicate::and)
.reduce(Function::andThen)https://habrahabr.ru/post/255813/
![Page 67: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/67.jpg)
67
Stream и Iterator
![Page 68: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/68.jpg)
68
iterator()
Верните цикл for()!
![Page 69: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/69.jpg)
69
for vs forEach()for forEach()
Появился Java 1.5 Java 1.8
Нормальная отладка
Короткие стектрейсы
Checked exceptions
Изменяемые переменные
Досрочный выход по break
Параллелизм (всё равно не нужен)
![Page 70: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/70.jpg)
70
iterator()
![Page 71: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/71.jpg)
71
iterator()Stream<String> s = Stream.of("a", "b", "c");for(String str : (Iterable<String>)s::iterator) { System.out.println(str);}
![Page 72: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/72.jpg)
72
iterator()Stream<String> s = Stream.of("a", "b", "c");for(String str : (Iterable<String>)s::iterator) { System.out.println(str);}
// Joshua Bloch approvesfor(String str : StreamEx.of("a", "b", "c")) { System.out.println(str);}
![Page 73: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/73.jpg)
73
iterator()
IntStream s = IntStream.of(1_000_000_000) .flatMap(x -> IntStream.range(0, x));for(int i : (Iterable<Integer>)s::iterator) { System.out.println(i);}
![Page 74: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/74.jpg)
74
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at j.u.s.SpinedBuffer$OfInt.newArray at j.u.s.SpinedBuffer$OfInt.newArray at j.u.s.SpinedBuffer$OfPrimitive.ensureCapacity at j.u.s.SpinedBuffer$OfPrimitive.increaseCapacity at j.u.s.SpinedBuffer$OfPrimitive.preAccept at j.u.s.SpinedBuffer$OfInt.accept at j.u.s.IntPipeline$7$1.lambda$accept$198 ... at j.u.s.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer at j.u.s.StreamSpliterators$AbstractWrappingSpliterator.doAdvance at j.u.s.StreamSpliterators$IntWrappingSpliterator.tryAdvance at j.u.Spliterators$2Adapter.hasNext at ru.javapoint.streamsamples.IterableWorkaround.main
![Page 75: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/75.jpg)
75
Stream из итератораStreamSupport.stream( Spliterators.spliteratorUnknownSize( iterator, Spliterator.ORDERED), false);
![Page 76: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/76.jpg)
76
Stream из итератораStreamSupport.stream( Spliterators.spliteratorUnknownSize( iterator, Spliterator.ORDERED), false);
StreamSupport.stream( ((Iterable<String>)() -> iterator).spliterator(), false);
![Page 77: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/77.jpg)
77
Stream из итератораFiles.find();Files.lines();Files.list();Files.walk();BufferedReader.lines();Pattern.splitAsStream();
![Page 78: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/78.jpg)
78
Как распараллелить последовательное?
...
![Page 79: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/79.jpg)
79
Как распараллелить последовательное?
...
Thread-1
![Page 80: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/80.jpg)
80
Как распараллелить последовательное?
...
Thread-1
Thread-2
![Page 81: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/81.jpg)
81
Files.list()List<Record> list = Files.list(root) .map(path -> parse(path)) .collect(Collectors.toList()); 204 ms
![Page 82: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/82.jpg)
82
Files.list().parallel()List<Record> list = Files.list(root) .parallel() .map(path -> parse(path)) .collect(Collectors.toList()); 202 ms
![Page 83: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/83.jpg)
83
Files.list().parallel()List<Record> list = Files.list(root) .parallel() .sorted(comparingLong(p -> p.toFile().length())) .map(path -> parse(path)) .collect(Collectors.toList()); 57.5 ms (3.5×)
![Page 84: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/84.jpg)
84
StreamToolshttps://github.com/amaembo/streamtools
SplitTree tree = SplitTree.inspect(IntStream.range(0, 200));try(OutputStream os = Files.newOutputStream( Paths.get("range.xgml"))) { new XGMLFormatter().nodeWidth(60).nodeHeight(20) .nodeFormat("%f..%l", "(empty)") .writeTo(tree, os);}
![Page 85: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/85.jpg)
85
IntStream.range(0, 200)
![Page 86: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/86.jpg)
86
Stream из итератора (200)
![Page 87: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/87.jpg)
87
Stream из итератора (2000)
![Page 88: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/88.jpg)
88
Stream из итератора (10000)
![Page 89: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/89.jpg)
89
0 5000 10000 15000 20000 250000
0.5
1
1.5
2
2.5
3
3.5
4
4.5
Speed-up
Speed-up
![Page 90: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/90.jpg)
90
Stream из итератораStreamSupport.stream( Spliterators.spliterator( iterator, size, Spliterator.ORDERED), false);
![Page 91: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/91.jpg)
91
SIZED-Stream из итератора (10000)
![Page 92: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/92.jpg)
92
Files.list()List<Record> list = Files.list(root) .collect(Collectors.toList()) .parallelStream() .map(path -> parse(path)) .collect(Collectors.toList());
![Page 93: JPoint 2016 - Валеев Тагир - Странности Stream API](https://reader035.vdocuments.site/reader035/viewer/2022062412/5877df081a28abaa6c8b6f25/html5/thumbnails/93.jpg)
93
Спасибо за вниманиеhttps://twitter.com/tagir_valeev
https://github.com/amaembo
https://habrahabr.ru/users/lany