sprite2005

sprite2005

Concurrent Data Processing in Elixir: Using a mapper operation after reduce/3 is deprecated (page 114)

@svilen

warning: Using a mapper operation, such as map/filter/reject, after reduce/3 is deprecated. Use Flow.on_trigger/2 instead
(flow 1.2.0) lib/flow.ex:1916: Flow.add_mapper/3
(airports 0.1.0) lib/airports.ex:25: Airports.open_airports/0
(stdlib 4.0.1) erl_eval.erl:744: :erl_eval.do_apply/7
(elixir 1.13.4) src/elixir.erl:296: :elixir.recur_eval/3
(elixir 1.13.4) src/elixir.erl:274: :elixir.eval_forms/3
(iex 1.13.4) lib/iex/evaluator.ex:310: IEx.Evaluator.handle_eval/3

How would we rewrite this as an on_trigger function?

Most Liked

andybab

andybab

This should work, but is longer than Flow.map version from book:

...
     |> Flow.reject(&(&1.type == "closed"))
     |> Flow.partition(key: {:key, :country})
     |> Flow.group_by(& &1.country)
     |> Flow.on_trigger(fn map ->
        country_data = Enum.map(map, fn {country, data} -> {country, Enum.count(data)} end)
        {country_data, map}
      end)
     #|> Flow.map(fn {country, data} -> {country, Enum.count(data)} end)
     |> Enum.to_list()
rschenk

rschenk

@svilen

I ran into this problem as well. While @andybab’s solution will produce the correct answer, I believe the use of Enum.map means that this part of the computation will not be calculated concurrently, whereas (I think?) the version in the book which uses Flow.map will be run in parallel. I could be wrong, I’m totally new at this.

It sure would be nice if the author of the book could provide a version that is both concurrent and not deprecated.

Edit: looks like my thinking was wrong. If you throw an IO.inspect(self()) into the andybab’s on_trigger callback, you can see that this does get split in parallel across multiple stages. For a noob, it’s confusing why/how this works and why you can’t use Flow.map instead, hopefully the author can update this section with an explanation.

Where Next?

Popular Pragmatic Bookshelf topics Top

johnp
Running the examples in chapter 5 c under pytest 5.4.1 causes an AttributeError: ‘module’ object has no attribute ‘config’. In particula...
New
jeffmcompsci
Title: Design and Build Great Web APIs - typo “https://company-atk.herokuapp.com/2258ie4t68jv” (page 19, third bullet in URL list) Typo:...
New
Alexandr
Hi everyone! There is an error on the page 71 in the book “Programming machine learning from coding to depp learning” P. Perrotta. You c...
New
simonpeter
When I try the command to create a pair of migration files I get an error. user=> (create-migration "guestbook") Execution error (Ill...
New
mikecargal
Title: Hands-On Rust (Chapter 11: prefab) Just played a couple of amulet-less games. With a bit of debugging, I believe that your can_p...
New
rmurray10127
Title: Intuitive Python: docker run… denied error (page 2) Attempted to run the docker command in both CLI and Powershell PS C:\Users\r...
New
jeremyhuiskamp
Title: Web Development with Clojure, Third Edition, vB17.0 (p9) The create table guestbook syntax suggested doesn’t seem to be accepted ...
New
adamwoolhether
When trying to generate the protobuf .go file, I receive this error: Unknown flag: --go_opt libprotoc 3.12.3 MacOS 11.3.1 Googling ...
New
s2k
Hi all, currently I wonder how the Tailwind colours work (or don’t work). For example, in app/views/layouts/application.html.erb I have...
New
a.zampa
@mfazio23 I’m following the indications of the book and arriver ad chapter 10, but the app cannot be compiled due to an error in the Bas...
New

Other popular topics Top

ohm
Which, if any, games do you play? On what platform? I just bought (and completed) Minecraft Dungeons for my Nintendo Switch. Other than ...
New
AstonJ
Curious to know which languages and frameworks you’re all thinking about learning next :upside_down_face: Perhaps if there’s enough peop...
New
AstonJ
Do the test and post your score :nerd_face: :keyboard: If possible, please add info such as the keyboard you’re using, the layout (Qw...
New
AstonJ
In case anyone else is wondering why Ruby 3 doesn’t show when you do asdf list-all ruby :man_facepalming: do this first: asdf plugin-upd...
New
rustkas
Intensively researching Erlang books and additional resources on it, I have found that the topic of using Regular Expressions is either c...
New
Maartz
Hi folks, I don’t know if I saw this here but, here’s a new programming language, called Roc Reminds me a bit of Elm and thus Haskell. ...
New
AstonJ
If you get Can't find emacs in your PATH when trying to install Doom Emacs on your Mac you… just… need to install Emacs first! :lol: bre...
New
PragmaticBookshelf
Author Spotlight Jamis Buck @jamis This month, we have the pleasure of spotlighting author Jamis Buck, who has written Mazes for Prog...
New
PragmaticBookshelf
Author Spotlight: VM Brasseur @vmbrasseur We have a treat for you today! We turn the spotlight onto Open Source as we sit down with V...
New
PragmaticBookshelf
Get the comprehensive, insider information you need for Rails 8 with the new edition of this award-winning classic. Sam Ruby @rubys ...
New

Sub Categories: