Introducing Whistle, a different approach to building interactive web apps in Elixir

The day I first found out about Redux, everyone seemed to be talking about "isomorphic" JavaScript, code that could run in both the client and the server, this got me thinking about whether it would be possible to move the reducers to the server and only communicate messages and current state to the clients via the network.

This way, we could do, without much difficulty, apps like real time games, chats or any sort of soft real time applications that can require a persistent internet connection.

Much later on I discovered Elm, which interestingly enough, was created before Redux and inspired its creating. Elm seemed to get everything right, an arguable simple programming model built on top of a very robust language, which made making client side applications fun again, at least for me,

Now, a few weeks ago Phoenix LiveView was announced, and what seemed almost impossible to do in Javascript, it actually seemed much easier to do by ditching JavaScript and doing it all in Elixir instead.

Erlang has proven again and again capable of maintaining millions of persistent connections at the same time, and Elixir is a language functional enough to be able to do something similar to the Elm architecture (TEA).

So here is my attempt at implementing an Elm(ish) web framework on top of Elixir, in the server:


Let's start off with a very simple program and break it down in pieces:

defmodule Example.CounterProgram do
  use Whistle.Program

  def init(_params) do
    {:ok, 0}
  end

  def authorize(_state, socket, _params) do
    {:ok, socket, %{}}
  end

  def update({:increment, n}, state, session) do
    {:ok, state + n, session}
  end

  def update({:decrement, n}, state, session) do
    {:ok, state - n, session}
  end

  def view(state, _session) do
    Html.div([], [
      Html.button([on: [click: {:increment, 1}]], "+"),
      Html.text(to_string(state)),
      Html.button([on: [click: {:decrement, 1}]], "-")
    ])
  end
end
Program running in two different connected clients

What is a program?

defmodule Example.CounterProgram do
  use Whistle.Program
  
  ...
end

First, this is what we call a program, conceptually very similar to what an Elm program is, but very different in lots of ways, because most of the things that make Elm so robust, are nonexistent in Elixir, and the "let it crash" philosophy doesn't translate really well with TEA either, so we have to adjust the programing model accordingly.

The program is the main module, and everything related to the this counter app belongs here.

A program is a long running process that lives in the server and can receive multiple client connections via Websockets, that can interact and render it.

Here is a very simplified diagram to illustrate how the functions in the module work together with the client to update the state and render the view

Update -> View loop

Initialization functions

def init(_params) do
  {:ok, 0}
end

The first function we see is init/1, this function takes some parameters and returns the initial state of the program process, it will be called when the program is first spawned, its just like GenServer.init/1.

def authorize(_state, socket, _params) do
  {:ok, socket, %{}}
end

Authorize is called when a Websocket connection requests access to the program, very similar to how the join callback works in Phoenix Channels, this function can refuse or give access to specific clients. Once the client has joined, it will receive view updates and will be able to send messages to the program.

This function will receive the current state of the program, the socket which contains information about the client connection, and the parameters sent from the client, for example,  we could send an authorization token and verify it here.

Authorize must return a tuple with and updated socket, and the initial client session data that will be passed to everything else, in this case an empty map.

The update function (aka reducer)

def update({:increment, n}, state, session) do
  {:ok, state + n, session}
end

def update({:decrement, n}, state, session) do
  {:ok, state - n, session}
end

The update function receives a message, the program state, and the session state of the client that sent the message. It returns an updated program state and the updated session state.

What is the difference between the program state and the session state?

The difference is that the session state belongs to the connected client, so you would store things like User Ids, preferences or any UI state for that particular user, basically anything that only concerns the current user. The program state is shared between all clients that are connected to the running program.

In this case, the number is stored in the program state, which means that all connected clients will see the same number, as you can see in the preview.

They both exist so that applications that have common state can be shared. For example, if you're making a game, the program state could store a representation of the world and the session would store the player's ID. This way you can know from which player the message came from in the update function, and be able to run any logic on whether the player can or cannot do the desired action.

The view

def view(state, _session) do
  Html.div([], [
    Html.button([on: [click: {:increment, 1}]], "+"),
    Html.text(to_string(state)),
    Html.button([on: [click: {:decrement, 1}]], "-")
  ])
end

The view function is called every time the current state changes, it returns a Virtual DOM, that is then diffed and broadcasted via WebSockets to each client.

Before sending the DOM patches to the client, Whistle scans for all event handlers in the attribute :on and wires them to the browser so that they get sent directly to the update function. Something else we can do is the following:

def update({:change_search, text}, state, session) do
  {:ok, state, %{session | search: text}}
end

def view(state, %{search: search}) do
  Html.div([], [
    Html.text("You're searching for: #{search}"),
    Html.input([type: "text", on: [input: &{:change_search, &1}]])
  ])
end

In this example, Whistle listens for the input event in Javascript and calls the function with the first argument being the value of the input. Input handlers can be both just a plain message tuple or a function that receives event data and returns the actual message.

How does the browser connect to the program?

Whistle's way of mounting a program into the browser's DOM is similar to how Phoenix Channels work, here is how we would make the counter actually work:

<div id="target"></div>
<script>
    import Whistle from 'whistle';
    let target = document.querySelector("#target");
    let socket = Whistle.connect(`ws://${window.location.host}/ws`);
    socket.on("open", function() {
        socket.join("counter", {}).then(function(counter) {
            counter.mount(target);
        });
    });
</script>

In this example, we are opening a connection to a  socket handler listening at the /ws route. And then mount the program counter to the #target div. Whistle will automatically start rendering the current view and keep up with view changes that happen after.

This is how we define the routes for our programs:

defmodule Example.ProgramRouter do
  use Whistle.Router
  match("counter", Example.CounterProgram, %{})
end

Like in Phoenix, we can also receive parameters in the route, each unique route will spawn a different program instance, for example we could make a counter for each user and add some simple authentication for it:

defmodule Example.ProgramRouter do
  use Whistle.Router
  match("counter:*user_id", Example.CounterProgram, %{})
end

defmodule Example.CounterProgram do
  use Whistle.Program

  def init(%{"user_id" => user_id}}) do
    {:ok, %{user_id: user_id, count: 0}}
  end
  
  def authorize(state, _socket, %{"user_id" => user_id}) do
   if state.user_id == user_id do
     {:ok, socket, %{}}
   else
     {:error, "no counter for you :("}
   end
  end

  # ...

end

What is the use case?

Whistle works great for making small components and mounting them in your existing web application. But all that Whistle needs is a Cowboy Websocket handler and we're good to go. Imagine a program that could serve an entire application, with different pages and such, here's a little example:

def update({:navigate, new_route}, state, session) do
  {:ok, state, %{session | route: new_route}}
end

def view(state, %{route: "/"}) do
  Html.div([], [
    Html.text("Welcome to the homepage!"),
    Html.a([on: [click: {:navigate, "/shop"}]], "Go to the shop"),
  ])
end

def view(%{products: products}, %{route: "/shop"}) do
  Html.div([], [
    Html.text("This is the shop!"),
    Html.a([on: [click: {:navigate, "/"}]], "Back to the homepage"),
    render_products(products)
  ])
end

def view(_state, _session) do
  Html.div([], [
    Html.text("Not found!"),
  ])
end

Which makes it great for making prototypes too or apps that are OK with requiring a constant internet connection. Because everything works through messages, there is no need to implement a JSON API and all the tedious work that comes with it.

And because a program in the end is just a bunch of functions, you can always organize your program parts in different functions and modules, for example:

defp layout_view(main, state, session) do
  Html.div([], [
    header(state, session),
    main,
    footer(state, session)
  ])
end

defp match("/", state, session) do
  state
  |> WebsiteProgram.HomepageView.view(session)
  |> layout_view(state, session)
end

defp match("/shop", %{products: products}, session) do
  products
  |> WebsiteProgram.HomepageView.view(session)
  |> layout_view(state, session)
end

defp match(_, state, session) do
  Html.div([], [Html.text("Not found!")])
end

def view(state, session = %{route: route}) do
  match(route, state, session)
end

Obviously this example is very basic, but it works for illustration purposes, for it to work properly we need to hook into the browser history and do a couple of more things, which is something that I have yet to figure out :)

Wrapping up

This article turned out longer than I expected, and there is still a lot left to talk about, so expect other posts soon-ish, some planned features and things to cover next:

  • How to do styling (Styled components, first class CSS in the Virtual Dom)
  • How does the Virtual DOM and DOM patch stream work
  • Session persistence in the client to keep state between refreshes
  • Error handling and network loss recovery
  • Keeping state updated across distributed nodes
  • Initial rendering via plain HTTP for SEO

Note: Something I still haven't had time to properly look at is the actual viability of this to serve lots of clients, of course we can hold millions of connections with Elixir, but there is the added overhead of the update loop and diffing the DOM at every update for every client. Something to look at in the future, as lots of optimizations can be done here.

This project was very fun to develop during my winter holidays to its current state, I am currently working on this in a private repository as its lacking proper documentation and some bits are not robust enough yet :) but I am looking to open it up in the following weeks.

I am really looking forward to hear what people think about this, also, feel free to shoot me a DM if you want to help in any way!

Show Comments

Get the latest posts delivered right to your inbox.