Concurrent Data Processing in Elixir: JobSupervisor does not go down (page 54)

After finishing Chapter 2, Long-Running Processes Using GenServer, I was able to successfully run the code. but if I see the process observer, I can see the JobSupervisor processes still alive. Is this the expected behavior? If I have created 100,000 jobs, there will be 100,000 active supervisors.

This is how my supervision tree looks like when triggered 3 jobs. 2 jobs are finished.

[Elixir.Jobber.JobRunner] – [<0.176.6>]
– [<0.191.0>]
– [<0.194.0>] — [<0.223.0>]

Thanks for your question @kunna. This is indeed the expected behaviour. The idle supervisor processes are harmless, and they don’t take up much memory. It’s usually not an issue in practice.

However, you can shutdown these processes if you want to. Something like this would work:

    Jobber.JobRunner
    |> DynamicSupervisor.which_children()
    |> Enum.each(fn {_, pid, _, _} ->
      if Supervisor.count_children(pid).active == 0 do
        Supervisor.stop(pid)
      end
    end)

I haven’t tested this function, but the idea is to check if the are any running processes under each JobSupervisor, and if there are not, call Supervisor.stop/1. You can look up the which_children/1 and count_children/1 functions for more information. This code can run periodically in a GenServer using Process.send_after/2, which is the technique covered in section “Notifying the Process of Events”.

I hope this helps :slight_smile: I will also try to clarify this in the book for the next beta version.

I decided to include a short section at the end of Chapter 2, which talks about count_children/1 and which_children/1. I think they’re quite useful so should be a welcome addition. I’ll also add a clarification on the idle supervisor processes. Thanks again for your feedback @kunna!

1 Like

First of all, thank you for answering my question and updating the book. after thinking about your solution I now have a couple of questions.

  1. what if “Jobber.JobSupervisor” was restarting its children? Isn’t there a possibility “Supervisor.count_children(pid).active == 0” will return true for a while?
  2. who should be responsible to run this code? I cannot think of a good place to add this code, should we add a new GenServer for this code?

No problem at all @kunna.

If a child is being restarted, then which_children/1 will return :restarting for the second element in the tuple, so you can use that in combination with count_children/1. Please see the documentation for which_children/1.

Yes, you can make a JobTerminator GenServer that does that.

1 Like