Elm & Django #2

Elm is a great tool to build simple widgets like a search bar. The use case is simple, as the user types, we want to dynamically show a filtered list of results.

Let’s build a tweet searching bar

The end result will be similar to the search of this very website: search Eat some code. We’ll build a simple tweet search as a similar example. The user will type something like “cup” and the corresponding tweets will magically appears. We’ll proceed in 3 simple steps:

providing the data as an API call
showing all the results via Elm
filtering the data (searching) via Elm*

To make it really fast, we’ll filter the results directly in Elm (i.e. in Javascript in the browser). With a large dataset or to operate more sophisticated filtering (handling synonyms, relevance, etc.), we would obviously prefer a more traditional approach where the filtering is done via the API. The advantage here is the speed; it’s satisfying to type and see the results instantaneously change.

In this article, it’s assumed that you are familiar with Django (or a similar backend framework) and curious about Elm. Also, you should probably check how to run Elm code in Django first: Elm & Django #1.

This is basically an Elm Hello World++ - it includes a field and an API. Let’s make it happen:

Step 1 - DB + API (i.e. Django side)

In Django, let’s add simplistic TweetAuthor & Tweet models:

from django.db import models
from ede.mixins import TimestampsMixin


class TweetAuthor(TimestampsMixin, models.Model):
    """ The one who writes the tweets (or their minion) """
    name = models.CharField(max_length=100)
    username = models.CharField(max_length=100)
    [...]

class Tweet(TimestampsMixin, models.Model):
    author = models.ForeignKey(to="ede.TweetAuthor", related_name="tweets", on_delete=models.CASCADE)
    content = models.TextField()

Let’s not forget to run the usual makemigrations and migrate commands before adding a basic API view:

def search_tweets_json(request):
    """ All the tweets """
    return JsonResponse(
        data=get_tweets_search_data()
    )

def get_tweets_search_data():
    return dict(
        tweets=[
            {
                'author_url': tweet.author.url,
                'author_name': tweet.author.name,
                'content': tweet.content,
                'search_string': "{username} {name} {content}".format(
                    username=tweet.author.username,
                    name=tweet.author.name,
                    content=tweet.content
                ).lower(),
            } for tweet in Tweet.objects.prefetch_related('author').all()
        ]
    )

The search_string contains in lowercase all the details. The reason to make it lower case is because our search filtering will be case insensitive. For example, when the user enters “Cup”, we will look for the presence of “cup” in that big string.

We’ll also need some data. You could fill it up via Django admin or via a fixture file. I personally prefer to write a simple command:

    jim = TweetAuthor.objects.create(
        name="Jim Jefferies",
        username="jimjefferies"
    )
    
    for tweet in (
        """It feels like there is a World Cup camera man who's only job is to find hot girls in the crowd""",
        (...)
        """With all this time spent protesting, when are the teachers going to learn how to shoot their guns?"""
    ):
        Tweet.objects.create(author=jim, content=tweet)

Finally, we’ll need a Django page to embed our Elm code. The template will look like this (as explained in Elm & Django #1):

{% load static %}
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Elm Django Example - Search</title>
</head>
<body>
  <h1>Elm Django Example - Search</h1>
  <div id="elm-search"></div>
  <script src="{% static 'ede/elm.js' %}"></script>
  <script type="text/javascript">Elm.Search.embed(document.querySelector("#elm-search"));</script>
</body>
</html>

I’ve skipped other the URLs, and optional command. To find the full code, please refer to git:

Step 2 - Loading & showing the content via Elm

Now that we have some data and an API to fetch it, we want to show it to the user. This is the trickiest part as we have to use the infamous Elm JSON decoder. That being said, that example is really simple since we are in control of the data format (we build the API) and we are only handling strings.

First, we’ll need to install the http package. We’ll install it and add it to elm-package.json (Elm equivalent of requirements.txt).

elm-package install elm-lang/http

Architecture Overview

In Elm (like in React, View, etc.), developers don’t have to deal with the DOM directly. The code describes how to represent the current “state” and how to deal with events. The code is divided like this:

Model: the data presented in the browser
Msg: the messages or events that match user interactions, API call results, etc.
view: the HTML (via a virtual DOM)
init: initial Model and command call (if any)
update: handle messages

In our example, these are:

Model: allTweets (yes, all of them, since the filtering is done directly via Elm)
Msg: GotTweets (the message sent once the tweets are received via the API)
view: a listing of tweets in HTML
init: first set allTweets as an empty list of tweets and call the API to get the tweets
update: handle GotTweets (successful or not)

The flow is self-explanatory in Elm. First the init method is called; here we want to get all the tweets straight away so we call the API via the getTweets function. getTweets will either be successful or not, in both cases it will send a GotTweets message. That message is then caught by update*.

* The beauty of Elm is that the compiler forces you to handle all cases: the update function must include all possible messages. In other words, the code won’t compile until update takes care of the GotTweets message; both successful and unsuccessful versions.

In init, we’ve asked to get all the tweets from the API. The API call is successful so GotTweets is sent. GotTweets/success is caught in the update method where allTweets gets set.

The code (finally)

We start by defining the Model. It’s basically a list of tweets. The model doesn’t have to match exactly the Django one; there is no reason at this stage to split the author in a separate type:

    type alias Tweet =
        { content : String
        , searchString : String -- lower case with all content
        , authorUrl : String
        , authorName : String
        }
    
    
    type alias Model =
        { allTweets : Array.Array Tweet
        }

Then the Msg listing (necessary to make sure all cases are handled):

    type Msg
        = GotTweets (Result Http.Error (Array.Array Tweet))

Note that GotTweets also specifies what the result must contain (an array of Tweet).

The view (not a Django view, the HTML representation - Elm uses a virtual DOM):

view model =
    div []
        [ renderTweets model ]


renderTweets : Model -> Html Msg
renderTweets model =
    div []
        [ h3 [] [ text "All the tweets" ]
        , model.allTweets
            |> Array.map renderTweet
            |> Array.toList
            |> div []
        ]


renderTweet : Tweet -> Html Msg
renderTweet tweet =
    li []
        [ strong [] [ text tweet.authorName ]
        , text " - "
        , span [] [ text tweet.content ]
        ]

The syntax is quite unfamiliar but I’m sure you can appreciate how succinct it is (& beautiful somehow). The |> operator fills in the function last parameter. I personally find it easier to reason by reading the code upside down.

In Elm the function div takes 2 parameters: a list of attributes and a list of content. Normally it looks like this: div [] [text "Some content"].

Reading the above code upside/down basically becomes: div has an empty list of parameter (l3); its content is a list (l2) of tweets (

tags) (l1); these tweets are all the tweets present in the model (l0).

init and corresponding code:

init : ( Model, Cmd Msg )
init =
    ( getInitialModel, getTweets )


getInitialModel : Model
getInitialModel =
    Model Array.empty


getTweets : Cmd Msg
getTweets =
    let
        url =
            "/search-tweets.json"
    in
    Http.send GotTweets (Http.get url tweetsDecoder)


tweetsDecoder : Decode.Decoder (Array.Array Tweet)
tweetsDecoder =
    Decode.at [ "tweets" ] (Decode.array tweetDecoder)


tweetDecoder : Decode.Decoder Tweet
tweetDecoder =
    Decode.map4
        Tweet
        (Decode.at [ "content" ] Decode.string)
        (Decode.at [ "search_string" ] Decode.string)
        (Decode.at [ "author_url" ] Decode.string)
        (Decode.at [ "author_name" ] Decode.string)

Elm is robust and extremely rigid. The response must be decoded no matter what; you cannot just create a JS object containing the results of the query. Lucky for us, we are lazy backend developers so we’ve defined a really simple flat JSON.

Also note that getTweets doesn’t have an if/else statement to handle errors. This is par of the message sent.

Again, the syntax is unfamiliar but succinct and nice considering that the decoder is mapping the JSON result to our Model.

Finally, here is the update code:

update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
    case msg of
        GotTweets (Ok newTweets) ->
            ( { getInitialModel | allTweets = newTweets }
            , Cmd.none
            )

        GotTweets (Err e) ->
            Debug.log (toString e)
                ( model, Cmd.none )

Roughly speaking, if the call was successful, model.allTweets is updated. Technically, a brand new model is created based on the initial model.

The full code for that second step can be seen here: Elm step 2 code.

Step 3 - Searching tweets

At this stage, we’ve got the full list of tweets in Elm but instead of showing it straight away, we want the user to search through it. With Elm architecture and compiler, all we need to do is to complete our Model and add the new messages we want to handle; we can then rely on the compiler to tell us what to complete. I invite you to often look at the compiler error while you work with Elm.

Let’s update our Model, we’ll need to know what the user is typing and the result for the current query (we still need the full list of tweets):

    type alias Model =
        { search : String
        , allTweets : Array.Array Tweet
        , resultTweets : Array.Array Tweet
        }

We’ll only add one message: the user is searching.

    type Msg
        = MsgSearch String
        | GotTweets (Result Http.Error (Array.Array Tweet))

To complete step 3 we have to:

display a search bar
- send the MsgSearch message on input
display the list of results instead of the full list

Then, that’s pretty much it ! The compiler will complain about the other missing bits:

MsgSearch isn’t handled in update
- we’ll fix that by filtering messages when the user is searching
the full model must always be properly created / updated - under init for example

The corresponding code is fairly simple. Here are the most interesting parts:

Display the search bar (we’ll call renderSearchBar in view):

renderSearchBar : Model -> Html Msg
renderSearchBar model =
    div [ class "search" ]
        [ input
            [ placeholder "search some tweets"
            , autofocus True
            , onInput MsgSearch
            ]
            []
        ]

Handling MsgSearch in update:

update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
    case msg of
        MsgSearch newSearch ->
            searchTweets model newSearch

        GotTweets (Ok newTweets) ->
            ...


searchTweets : Model -> String -> ( Model, Cmd Msg )
searchTweets model search =
    let
        newModel =
            { model | search = search, resultTweets = filterTweets model.allTweets search }
    in
    ( newModel, Cmd.none )


filterTweets : Array.Array Tweet -> String -> Array.Array Tweet
filterTweets allTweets search =
    if search == "" then
        Array.empty
    else
        allTweets
            |> Array.filter (\tweet -> String.contains (String.toLower search) tweet.searchString)
            |> Array.slice 0 22

The full code can be seen at: Elm step 3 code.

Going further

You’ve probably realised that that code doesn’t handle entering multiple words; I let you figure this one out ! I would strongly encourage any web developer to give Elm a try. It’s full of interesting ideas.

October 2018