The Tessera Book
Chapter 16

Final Project: A Link Shortener

Let's finish by building something real: a small web service that shortens links. You send it a long URL, it gives you back a short code, and visiting /CODE redirects to the long URL, counting each visit. It's small enough to read in one sitting, and it uses most of what the book has covered: structs, enums and match, modules, Result and ?, tests, capabilities, groups, HTTP, JSON and SQL.

Here's the plan, as an API:

RequestAnswer
POST /links with {"url": "https://..."}201 and the new link as JSON
GET /CODE302, redirecting to the link's URL, and one more hit
GET /links/CODE200 and the link with its hit count
anything else404

And here's how the code is split, so that each part needs as little authority as possible:

ModuleDoesNeeds
codes.tsrshort codes, URL checksnothing: pure
store.tsrsaving and finding linksa database
api.tsrHTTP requests to responsesa database
main.tsrstarting the serverListen, and Net for the demo

The project

The project is one directory with a tessera.toml, so that the grants it needs are written down once:

[package]
name = "links"
description = "A small link shortener"

[grants]
listen = ["127.0.0.1"]
net = ["127.0.0.1"]
examples/ch16-links/tessera.toml

listen = ["127.0.0.1"] lets the program listen on any port of the loopback interface, and net = ["127.0.0.1"] lets the demo's client talk to it. Nothing else is granted: no files, no other hosts, no environment.

Codes: the pure core

Start with the part that needs nothing. A link stored with id 125 gets the code 21: the id written in base 62, with digits, lowercase and uppercase letters.

// Short codes for links, and checks on the URLs people send. Everything
// here is pure: no capabilities, so it can only compute.

fn alphabet() -> List[Str] {
    "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ".chars()
}

// The code for the link with this id: the id written in base 62.
pub fn encode(id: Int) -> Str {
    let digits = alphabet()
    let mut n = id
    let mut code = ""
    while true {
        code = digits[n % 62] + code
        n = n / 62
        if n == 0 {
            break
        }
    }
    code
}

// The id a code stands for, or None if it is not a code.
pub fn decode(code: Str) -> Option[Int] {
    if code.is_empty() || code.len() > 10 {
        return None
    }
    let digits = alphabet()
    let mut n = 0
    for c in code.chars() {
        let d = digits.find_index(c)?
        n = n * 62 + d
    }
    Some(n)
}

fn find_index(self: List[Str], x: Str) -> Option[Int] {
    for (i, y) in self.with_index() {
        if y == x {
            return Some(i)
        }
    }
    None
}

// Whether a URL is one we are willing to redirect to.
pub fn valid_url(url: Str) -> Bool {
    let rest = if url.starts_with("https://") {
        url.slice(8, url.len())
    } else if url.starts_with("http://") {
        url.slice(7, url.len())
    } else {
        return false
    }
    let host = rest.split("/")[0]
    host.contains(".") && !host.contains(" ") && url.len() <= 2000
}

test "encode" for (id, want) in [(0, "0"), (61, "Z"), (62, "10"), (125, "21")] {
    assert_eq(encode(id), want)
}

test "decode undoes encode" {
    for id in [0, 1, 61, 62, 3843, 1_000_000] {
        assert_eq(decode(encode(id)), Some(id))
    }
}

test "decode rejects what is not a code" {
    assert_eq(decode(""), None)
    assert_eq(decode("ab-c"), None)
}

test "valid_url" for (url, ok) in [
    ("https://example.com/a?b=c", true),
    ("http://example.com", true),
    ("ftp://example.com", false),
    ("https://localhost", false),
    ("javascript:alert(1)", false),
] {
    assert_eq(valid_url(url), ok)
}
Listing 16-1: codes.tsr: short codes and URL checks examples/ch16-links/codes.tsr

encode builds the code digit by digit from the right. decode goes the other way and returns an Option, since not every string is a code; it uses a small helper method, find_index, which is private to this file because it's a method on a built-in type. valid_url decides what the service is willing to redirect to: http and https URLs with a dotted host name, so javascript: URLs and the like are refused.

Because this module is pure, it is easy to test thoroughly, with table tests for the interesting cases:

$ tessera test codes.tsr
ok    encode (0, "0") (0ms)
ok    encode (61, "Z") (0ms)
ok    encode (62, "10") (0ms)
ok    encode (125, "21") (0ms)
ok    decode undoes encode (0ms)
ok    decode rejects what is not a code (0ms)
ok    valid_url ("https://example.com/a?b=c", true) (0ms)
ok    valid_url ("http://example.com", true) (0ms)
ok    valid_url ("ftp://example.com", false) (0ms)
ok    valid_url ("https://localhost", false) (0ms)
ok    valid_url ("javascript:alert(1)", false) (0ms)
11 tests, 0 failed (0ms)

Storage

Links live in SQLite. Every function in store.tsr takes the database, a sql.Db, and nothing else:

// Where links live: a SQLite database. Every function takes the Db and
// nothing else, so this module can touch that database and no other thing.
use sql
use codes

pub struct Link {
    code: Str,
    url: Str,
    hits: Int,
}

pub fn setup(db: sql.Db) -> Result[(), IoError] {
    db.exec("CREATE TABLE IF NOT EXISTS links (id INTEGER PRIMARY KEY, url TEXT NOT NULL, hits INTEGER NOT NULL DEFAULT 0)", [])?
    Ok(())
}

pub fn add(db: sql.Db, url: Str) -> Result[Link, IoError] {
    let row = db.query_one("INSERT INTO links (url) VALUES (?) RETURNING id", [sql.Text(url)])?
    let id = row?.int("id") ?? 0
    Ok(Link { code: codes.encode(id), url, hits: 0 })
}

pub fn find(db: sql.Db, code: Str) -> Result[Option[Link], IoError] {
    let id = match codes.decode(code) {
        Some(id) => id,
        None => {
            return Ok(None)
        },
    }
    let row = db.query_one("SELECT url, hits FROM links WHERE id = ?", [sql.Int(id)])?
    Ok(row.map(|r| Link { code, url: r.text("url") ?? "", hits: r.int("hits") ?? 0 }))
}

// Follows a link: counts the visit and returns where it goes.
pub fn visit(db: sql.Db, code: Str) -> Result[Option[Str], IoError] {
    let id = match codes.decode(code) {
        Some(id) => id,
        None => {
            return Ok(None)
        },
    }
    let row = db.query_one("UPDATE links SET hits = hits + 1 WHERE id = ? RETURNING url", [sql.Int(id)])?
    Ok(row?.text("url"))
}
Listing 16-2: store.tsr: links in a database examples/ch16-links/store.tsr

add inserts a row and uses RETURNING id to get its id from the insert itself, so two requests at once can't see each other's ids. visit counts the hit and returns the URL in one statement. Both turn a code into an id with codes.decode first, and treat a string that isn't a code as "no such link".

Values go in as parameters (sql.Text(url), sql.Int(id)), never into the SQL text, so a URL can't change the query.

The HTTP API

The handler turns a request into a response. It splits the path into segments and matches on the method and the segments together, with list patterns (chapter 6) for the routes: ["links", code] is a path of exactly two segments, the first of which is links, and binds the second to code:

// The HTTP side: requests in, responses out. The handler gets the database
// and nothing else, whatever a request asks for.
use http
use json
use sql
use codes
use store

pub fn handle(db: sql.Db, r: http.Request) -> http.Response {
    let path = r.path.split("?")[0]
    let parts = path.split("/").filter(|p| p != "")
    let result = match (r.method, parts) {
        ("POST", ["links"]) => create(db, r.body),
        ("GET", ["links", code]) => stats(db, code),
        ("GET", [code]) => follow(db, code),
        ("GET", []) => Ok(http.text(200, "POST /links with \{\"url\": \"...\"\} to shorten a link\n")),
        _ => Ok(http.text(404, "not found\n")),
    }
    match result {
        Ok(resp) => resp,
        Err(e) => http.text(500, "storage error: {e}\n"),
    }
}
Listing 16-3: api.tsr: routing examples/ch16-links/api.tsr, lines 1 to 23

Each route returns a Result, so storage errors from all three meet in one place and become a 500. The routes themselves are short, because the modules below do the work:

fn create(db: sql.Db, body: Str) -> Result[http.Response, IoError] {
    let url = match json.parse(body) {
        Ok(v) => v.get("url")?.as_str() ?? "",
        Err(_) => "",
    }
    if !codes.valid_url(url) {
        return Ok(http.text(400, "send \{\"url\": \"https://...\"\}\n"))
    }
    let link = store.add(db, url)?
    Ok(http.json(201, link_json(link)))
}

fn stats(db: sql.Db, code: Str) -> Result[http.Response, IoError] {
    match store.find(db, code)? {
        Some(link) => Ok(http.json(200, link_json(link))),
        None => Ok(http.text(404, "no link {code}\n")),
    }
}

fn follow(db: sql.Db, code: Str) -> Result[http.Response, IoError] {
    match store.visit(db, code)? {
        Some(url) => Ok(http.Response { status: 302, headers: [http.Header { name: "Location", value: url }], body: "" }),
        None => Ok(http.text(404, "no link {code}\n")),
    }
}

fn link_json(l: store.Link) -> Str {
    json.stringify(json.Obj([
        json.field("code", json.Str(l.code)),
        json.field("url", json.Str(l.url)),
        json.field("hits", json.Num(l.hits.to_float())),
    ]))
}
Listing 16-4: api.tsr: the routes examples/ch16-links/api.tsr, lines 25 to 57

create reads the JSON body with ?. and ??, so a body that isn't JSON, or has no url, or a url that isn't a string, all end up as an empty string, which valid_url refuses. follow builds a response by hand to add the Location header.

Look at what the handler can do: it takes a sql.Db and a request. It can't read files, open connections or run programs, whatever a request contains, because it was never given a capability to do so.

Testing the API without a network

Since the handler is a plain function, tests call it directly, with an in-memory database, and never open a socket:

// ---------- tests: the handler is a plain function, so tests call it ----------

fn request(db: sql.Db, method: Str, path: Str, body: Str) -> http.Response {
    handle(db, http.Request { method, path, version: "HTTP/1.1", headers: [], body })
}

fn test_db() -> sql.Db {
    let db = sql.memory() ?? panic("no database")
    store.setup(db) ?? panic("no table")
    db
}

test "shorten, follow, count" {
    let db = test_db()
    let made = request(db, "POST", "/links", "\{\"url\": \"https://example.com/a\"\}")
    assert_eq(made.status, 201)
    assert_eq(made.body, "\{\"code\":\"1\",\"url\":\"https://example.com/a\",\"hits\":0\}")

    let go = request(db, "GET", "/1", "")
    assert_eq(go.status, 302)
    assert_eq(go.header("location"), Some("https://example.com/a"))

    assert_eq(request(db, "GET", "/links/1", "").body.contains("\"hits\":1"), true)
}

test "refuses bad input" for (method, path, body, status) in [
    ("POST", "/links", "\{\"url\": \"javascript:alert(1)\"\}", 400),
    ("POST", "/links", "not json", 400),
    ("GET", "/zzzz", "", 404),
    ("GET", "/links/!", "", 404),
    ("DELETE", "/links", "", 404),
] {
    assert_eq(request(test_db(), method, path, body).status, status)
}
Listing 16-5: api.tsr: tests examples/ch16-links/api.tsr, lines 59 to 92
$ tessera test api.tsr
ok    shorten, follow, count (2ms)
ok    refuses bad input ("POST", "/links", "\{\"url\": \"javascript:alert(1)\"\}", 400) (1ms)
ok    refuses bad input ("POST", "/links", "not json", 400) (0ms)
ok    refuses bad input ("GET", "/zzzz", "", 404) (0ms)
ok    refuses bad input ("GET", "/links/!", "", 404) (0ms)
ok    refuses bad input ("DELETE", "/links", "", 404) (0ms)
6 tests, 0 failed (3ms)

The table test sends five kinds of bad input and checks each status, and each case shows up as its own line.

main

Finally, main decides where the authority goes:

// A link shortener.
//
//   tessera run main.tsr                  a demo: server and client in one run
//   tessera run main.tsr -- serve 8080    serve on 127.0.0.1:8080
use http
use sql
use store
use api

fn main(root: Root) -> Result[(), dyn Error] {
    let db = sql.memory()? "opening the database"
    store.setup(db)? "creating the table"
    let args = root.args()
    if args.get(0) == Some("serve") {
        let port = args.get(1) ?? "8080"
        let l = root.listen.on("127.0.0.1:{port}")?
        root.stdout.print("serving on http://127.0.0.1:{port}")
        return Err(http.serve_forever(l, |r| api.handle(db, r)))
    }
    demo(root.listen, root.net.only("127.0.0.1"), root.stdout, db)
}

// Starts the server on a free port and talks to it.
fn demo(listen: Listen, net: Net, out: Stdout, db: sql.Db) -> Result[(), dyn Error] {
    let l = listen.on("127.0.0.1:0")?
    let base = "http://127.0.0.1:{l.port()}"
    let lines = group g {
        g.go(|| {
            let _ = http.serve(l, 4, |r| api.handle(db, r))
        })
        g.go(|| client(net, base))
    }
    for line in lines[0] {
        out.print(line)
    }
    Ok(())
}

fn client(net: Net, base: Str) -> List[Str] {
    let mut log: List[Str] = []
    let steps = [
        ("POST", "/links", "\{\"url\": \"https://example.com/a/long/page\"\}"),
        ("GET", "/1", ""),
        ("GET", "/links/1", ""),
        ("GET", "/nope", ""),
    ]
    for (method, path, body) in steps {
        let headers = [http.Header { name: "Content-Type", value: "application/json" }]
        match http.request(net, method, base + path, headers, body) {
            Ok(r) => {
                let to = r.header("location").map(|u| " -> {u}") ?? ""
                log.push("{method} {path}: {r.status}{to} {r.body}".trim())
            },
            Err(e) => log.push("{method} {path}: {e}"),
        }
    }
    log
}
Listing 16-6: main.tsr: serving, or a demo examples/ch16-links/main.tsr

With -- serve 8080, main listens on that port and passes each request to api.handle, through a closure that captures only the database. Without arguments, it runs a demo: demo starts the server on a free port in one task and a client in another, inside a group, and prints what the client saw. The client gets a Net narrowed to 127.0.0.1.

tessera caps confirms the layering. Here it is with the standard library's own functions filtered out:

$ tessera caps main.tsr | grep -v -E '^(sql|http|json|url|text|hex|format)\.'
store.setup                       pure
store.add                         pure
store.find                        pure
store.visit                       pure
codes.alphabet                    pure
codes.encode                      pure
codes.decode                      pure
codes.List::find_index            pure
codes.valid_url                   pure
api.handle                        pure
api.create                        pure
api.stats                         pure
api.follow                        pure
api.link_json                     pure
api.request                       pure
api.test_db                       pure
main                              Root
demo                              Net, Stdout, Listen
client                            Net

Only main, demo and client hold capabilities. Everything in codes, store and api is reported as pure, because none of it takes a capability. The store and the API do change a database, but only the one whose sql.Db they were handed, the same way a function given a list can only change its own copy of that list.

And the demo:

$ tessera run main.tsr
POST /links: 201 {"code":"1","url":"https://example.com/a/long/page","hits":0}
GET /1: 302 -> https://example.com/a/long/page
GET /links/1: 200 {"code":"1","url":"https://example.com/a/long/page","hits":1}
GET /nope: 404 no link nope

To serve it for real, run tessera run main.tsr -- serve 8080 and try it with curl:

$ curl -X POST localhost:8080/links -d '{"url": "https://example.com/some/long/page"}'
$ curl -i localhost:8080/1

Where to go from here

Some ideas to extend the service, each of which uses something from an earlier chapter:

  • Keep links across restarts: open the database with sql.open(root.fs, "links.db") and grant a directory in tessera.toml (chapters 12 and 14).
  • Add DELETE /links/CODE: one more arm in handle, one more function in store, and a case in the table test (chapters 6 and 11).
  • Give each request a deadline with time.within (chapter 13).
  • Move codes.tsr into a package of its own, with no caps, and use it as a dependency (chapter 7).

Thank you

That's the end of the guided tour. You've seen a language where every effect a function can have is in its signature, where values never change behind your back, where concurrent output is ordered, and where the compiler can tell you exactly what each part of a program, and each dependency, is able to do. The appendices that follow are for reference: keywords and operators, the tools, how the compiler works, and what isn't there yet.