sprite2005

sprite2005

Concurrent Data Processing in Elixir: Using a mapper operation after reduce/3 is deprecated (page 114)

@svilen

warning: Using a mapper operation, such as map/filter/reject, after reduce/3 is deprecated. Use Flow.on_trigger/2 instead
(flow 1.2.0) lib/flow.ex:1916: Flow.add_mapper/3
(airports 0.1.0) lib/airports.ex:25: Airports.open_airports/0
(stdlib 4.0.1) erl_eval.erl:744: :erl_eval.do_apply/7
(elixir 1.13.4) src/elixir.erl:296: :elixir.recur_eval/3
(elixir 1.13.4) src/elixir.erl:274: :elixir.eval_forms/3
(iex 1.13.4) lib/iex/evaluator.ex:310: IEx.Evaluator.handle_eval/3

How would we rewrite this as an on_trigger function?

Most Liked

andybab

andybab

This should work, but is longer than Flow.map version from book:

...
     |> Flow.reject(&(&1.type == "closed"))
     |> Flow.partition(key: {:key, :country})
     |> Flow.group_by(& &1.country)
     |> Flow.on_trigger(fn map ->
        country_data = Enum.map(map, fn {country, data} -> {country, Enum.count(data)} end)
        {country_data, map}
      end)
     #|> Flow.map(fn {country, data} -> {country, Enum.count(data)} end)
     |> Enum.to_list()
rschenk

rschenk

@svilen

I ran into this problem as well. While @andybab’s solution will produce the correct answer, I believe the use of Enum.map means that this part of the computation will not be calculated concurrently, whereas (I think?) the version in the book which uses Flow.map will be run in parallel. I could be wrong, I’m totally new at this.

It sure would be nice if the author of the book could provide a version that is both concurrent and not deprecated.

Edit: looks like my thinking was wrong. If you throw an IO.inspect(self()) into the andybab’s on_trigger callback, you can see that this does get split in parallel across multiple stages. For a noob, it’s confusing why/how this works and why you can’t use Flow.map instead, hopefully the author can update this section with an explanation.

Where Next?

Popular Pragmatic Bookshelf topics Top

HarryDeveloper
Hi @venkats, It has been mentioned in the description of ‘Supervisory Job’ title that 2 things as mentioned below result in the same eff...
New
jeremyhuiskamp
Title: Web Development with Clojure, Third Edition, vB17.0 (p9) The create table guestbook syntax suggested doesn’t seem to be accepted ...
New
gilesdotcodes
In case this helps anyone, I’ve had issues setting up the rails source code. Here were the solutions: In Gemfile, change gem 'rails' t...
New
leonW
I ran this command after installing the sample application: $ cards add do something --owner Brian And got a file not found error: Fil...
New
jskubick
I think I might have found a problem involving SwitchCompat, thumbTint, and trackTint. As entered, the SwitchCompat changes color to hol...
New
jskubick
I’m under the impression that when the reader gets to page 136 (“View Data with the Database Inspector”), the code SHOULD be able to buil...
New
nicoatridge
Hi, I have just acquired Michael Fazio’s “Kotlin and Android Development” to learn about game programming for Android. I have a game in p...
New
brunogirin
When trying to run tox in parallel as explained on page 151, I got the following error: tox: error: argument -p/–parallel: expected one...
New
dsmith42
Hey there, I’m enjoying this book and have learned a few things alredayd. However, in Chapter 4 I believe we are meant to see the “>...
New
roadbike
From page 13: On Python 3.7, you can install the libraries with pip by running these commands inside a Python venv using Visual Studio ...
New

Other popular topics Top

axelson
I’ve been really enjoying obsidian.md: It is very snappy (even though it is based on Electron). I love that it is all local by defaul...
New
New
AstonJ
If you are experiencing Rails console using 100% CPU on your dev machine, then updating your development and test gems might fix the issu...
New
PragmaticBookshelf
“A Mystical Experience” Hero’s Journey with Paolo Perrotta @nusco Ever wonder how authoring books compares to writing articles?...
New
DevotionGeo
The V Programming Language Simple language for building maintainable programs V is already mentioned couple of times in the forum, but I...
New
AstonJ
Seems like a lot of people caught it - just wondered whether any of you did? As far as I know I didn’t, but it wouldn’t surprise me if I...
New
AstonJ
Saw this on TikTok of all places! :lol: Anyone heard of them before? Lite:
New
New
First poster: bot
zig/http.zig at 7cf2cbb33ef34c1d211135f56d30fe23b6cacd42 · ziglang/zig. General-purpose programming language and toolchain for maintaini...
New
AstonJ
This is a very quick guide, you just need to: Download LM Studio: https://lmstudio.ai/ Click on search Type DeepSeek, then select the o...
New

Sub Categories: