Final Project: A Link Shortener
Let's finish by building something real: a small web service that shortens links. You send it a long URL, it gives you back a short code, and visiting /CODE redirects to the long URL, counting each visit. It's small enough to read in one sitting, and it uses most of what the book has covered: structs, enums and match, modules, Result and ?, tests, capabilities, groups, HTTP, JSON and SQL.
Here's the plan, as an API:
| Request | Answer |
|---|---|
POST /links with {"url": "https://..."} | 201 and the new link as JSON |
GET /CODE | 302, redirecting to the link's URL, and one more hit |
GET /links/CODE | 200 and the link with its hit count |
| anything else | 404 |
And here's how the code is split, so that each part needs as little authority as possible:
| Module | Does | Needs |
|---|---|---|
codes.tsr | short codes, URL checks | nothing: pure |
store.tsr | saving and finding links | a database |
api.tsr | HTTP requests to responses | a database |
main.tsr | starting the server | Listen, and Net for the demo |
The project
The project is one directory with a tessera.toml, so that the grants it needs are written down once:
[package]
name = "links"
description = "A small link shortener"
[grants]
listen = ["127.0.0.1"]
net = ["127.0.0.1"]listen = ["127.0.0.1"] lets the program listen on any port of the loopback interface, and net = ["127.0.0.1"] lets the demo's client talk to it. Nothing else is granted: no files, no other hosts, no environment.
Codes: the pure core
Start with the part that needs nothing. A link stored with id 125 gets the code 21: the id written in base 62, with digits, lowercase and uppercase letters.
// Short codes for links, and checks on the URLs people send. Everything
// here is pure: no capabilities, so it can only compute.
fn alphabet() -> List[Str] {
"0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ".chars()
}
// The code for the link with this id: the id written in base 62.
pub fn encode(id: Int) -> Str {
let digits = alphabet()
let mut n = id
let mut code = ""
while true {
code = digits[n % 62] + code
n = n / 62
if n == 0 {
break
}
}
code
}
// The id a code stands for, or None if it is not a code.
pub fn decode(code: Str) -> Option[Int] {
if code.is_empty() || code.len() > 10 {
return None
}
let digits = alphabet()
let mut n = 0
for c in code.chars() {
let d = digits.find_index(c)?
n = n * 62 + d
}
Some(n)
}
fn find_index(self: List[Str], x: Str) -> Option[Int] {
for (i, y) in self.with_index() {
if y == x {
return Some(i)
}
}
None
}
// Whether a URL is one we are willing to redirect to.
pub fn valid_url(url: Str) -> Bool {
let rest = if url.starts_with("https://") {
url.slice(8, url.len())
} else if url.starts_with("http://") {
url.slice(7, url.len())
} else {
return false
}
let host = rest.split("/")[0]
host.contains(".") && !host.contains(" ") && url.len() <= 2000
}
test "encode" for (id, want) in [(0, "0"), (61, "Z"), (62, "10"), (125, "21")] {
assert_eq(encode(id), want)
}
test "decode undoes encode" {
for id in [0, 1, 61, 62, 3843, 1_000_000] {
assert_eq(decode(encode(id)), Some(id))
}
}
test "decode rejects what is not a code" {
assert_eq(decode(""), None)
assert_eq(decode("ab-c"), None)
}
test "valid_url" for (url, ok) in [
("https://example.com/a?b=c", true),
("http://example.com", true),
("ftp://example.com", false),
("https://localhost", false),
("javascript:alert(1)", false),
] {
assert_eq(valid_url(url), ok)
}encode builds the code digit by digit from the right. decode goes the other way and returns an Option, since not every string is a code; it uses a small helper method, find_index, which is private to this file because it's a method on a built-in type. valid_url decides what the service is willing to redirect to: http and https URLs with a dotted host name, so javascript: URLs and the like are refused.
Because this module is pure, it is easy to test thoroughly, with table tests for the interesting cases:
$ tessera test codes.tsr
ok encode (0, "0") (0ms)
ok encode (61, "Z") (0ms)
ok encode (62, "10") (0ms)
ok encode (125, "21") (0ms)
ok decode undoes encode (0ms)
ok decode rejects what is not a code (0ms)
ok valid_url ("https://example.com/a?b=c", true) (0ms)
ok valid_url ("http://example.com", true) (0ms)
ok valid_url ("ftp://example.com", false) (0ms)
ok valid_url ("https://localhost", false) (0ms)
ok valid_url ("javascript:alert(1)", false) (0ms)
11 tests, 0 failed (0ms)
Storage
Links live in SQLite. Every function in store.tsr takes the database, a sql.Db, and nothing else:
// Where links live: a SQLite database. Every function takes the Db and
// nothing else, so this module can touch that database and no other thing.
use sql
use codes
pub struct Link {
code: Str,
url: Str,
hits: Int,
}
pub fn setup(db: sql.Db) -> Result[(), IoError] {
db.exec("CREATE TABLE IF NOT EXISTS links (id INTEGER PRIMARY KEY, url TEXT NOT NULL, hits INTEGER NOT NULL DEFAULT 0)", [])?
Ok(())
}
pub fn add(db: sql.Db, url: Str) -> Result[Link, IoError] {
let row = db.query_one("INSERT INTO links (url) VALUES (?) RETURNING id", [sql.Text(url)])?
let id = row?.int("id") ?? 0
Ok(Link { code: codes.encode(id), url, hits: 0 })
}
pub fn find(db: sql.Db, code: Str) -> Result[Option[Link], IoError] {
let id = match codes.decode(code) {
Some(id) => id,
None => {
return Ok(None)
},
}
let row = db.query_one("SELECT url, hits FROM links WHERE id = ?", [sql.Int(id)])?
Ok(row.map(|r| Link { code, url: r.text("url") ?? "", hits: r.int("hits") ?? 0 }))
}
// Follows a link: counts the visit and returns where it goes.
pub fn visit(db: sql.Db, code: Str) -> Result[Option[Str], IoError] {
let id = match codes.decode(code) {
Some(id) => id,
None => {
return Ok(None)
},
}
let row = db.query_one("UPDATE links SET hits = hits + 1 WHERE id = ? RETURNING url", [sql.Int(id)])?
Ok(row?.text("url"))
}add inserts a row and uses RETURNING id to get its id from the insert itself, so two requests at once can't see each other's ids. visit counts the hit and returns the URL in one statement. Both turn a code into an id with codes.decode first, and treat a string that isn't a code as "no such link".
Values go in as parameters (sql.Text(url), sql.Int(id)), never into the SQL text, so a URL can't change the query.
The HTTP API
The handler turns a request into a response. It splits the path into segments and matches on the method and the segments together, with list patterns (chapter 6) for the routes: ["links", code] is a path of exactly two segments, the first of which is links, and binds the second to code:
// The HTTP side: requests in, responses out. The handler gets the database
// and nothing else, whatever a request asks for.
use http
use json
use sql
use codes
use store
pub fn handle(db: sql.Db, r: http.Request) -> http.Response {
let path = r.path.split("?")[0]
let parts = path.split("/").filter(|p| p != "")
let result = match (r.method, parts) {
("POST", ["links"]) => create(db, r.body),
("GET", ["links", code]) => stats(db, code),
("GET", [code]) => follow(db, code),
("GET", []) => Ok(http.text(200, "POST /links with \{\"url\": \"...\"\} to shorten a link\n")),
_ => Ok(http.text(404, "not found\n")),
}
match result {
Ok(resp) => resp,
Err(e) => http.text(500, "storage error: {e}\n"),
}
}Each route returns a Result, so storage errors from all three meet in one place and become a 500. The routes themselves are short, because the modules below do the work:
fn create(db: sql.Db, body: Str) -> Result[http.Response, IoError] {
let url = match json.parse(body) {
Ok(v) => v.get("url")?.as_str() ?? "",
Err(_) => "",
}
if !codes.valid_url(url) {
return Ok(http.text(400, "send \{\"url\": \"https://...\"\}\n"))
}
let link = store.add(db, url)?
Ok(http.json(201, link_json(link)))
}
fn stats(db: sql.Db, code: Str) -> Result[http.Response, IoError] {
match store.find(db, code)? {
Some(link) => Ok(http.json(200, link_json(link))),
None => Ok(http.text(404, "no link {code}\n")),
}
}
fn follow(db: sql.Db, code: Str) -> Result[http.Response, IoError] {
match store.visit(db, code)? {
Some(url) => Ok(http.Response { status: 302, headers: [http.Header { name: "Location", value: url }], body: "" }),
None => Ok(http.text(404, "no link {code}\n")),
}
}
fn link_json(l: store.Link) -> Str {
json.stringify(json.Obj([
json.field("code", json.Str(l.code)),
json.field("url", json.Str(l.url)),
json.field("hits", json.Num(l.hits.to_float())),
]))
}create reads the JSON body with ?. and ??, so a body that isn't JSON, or has no url, or a url that isn't a string, all end up as an empty string, which valid_url refuses. follow builds a response by hand to add the Location header.
Look at what the handler can do: it takes a sql.Db and a request. It can't read files, open connections or run programs, whatever a request contains, because it was never given a capability to do so.
Testing the API without a network
Since the handler is a plain function, tests call it directly, with an in-memory database, and never open a socket:
// ---------- tests: the handler is a plain function, so tests call it ----------
fn request(db: sql.Db, method: Str, path: Str, body: Str) -> http.Response {
handle(db, http.Request { method, path, version: "HTTP/1.1", headers: [], body })
}
fn test_db() -> sql.Db {
let db = sql.memory() ?? panic("no database")
store.setup(db) ?? panic("no table")
db
}
test "shorten, follow, count" {
let db = test_db()
let made = request(db, "POST", "/links", "\{\"url\": \"https://example.com/a\"\}")
assert_eq(made.status, 201)
assert_eq(made.body, "\{\"code\":\"1\",\"url\":\"https://example.com/a\",\"hits\":0\}")
let go = request(db, "GET", "/1", "")
assert_eq(go.status, 302)
assert_eq(go.header("location"), Some("https://example.com/a"))
assert_eq(request(db, "GET", "/links/1", "").body.contains("\"hits\":1"), true)
}
test "refuses bad input" for (method, path, body, status) in [
("POST", "/links", "\{\"url\": \"javascript:alert(1)\"\}", 400),
("POST", "/links", "not json", 400),
("GET", "/zzzz", "", 404),
("GET", "/links/!", "", 404),
("DELETE", "/links", "", 404),
] {
assert_eq(request(test_db(), method, path, body).status, status)
}$ tessera test api.tsr
ok shorten, follow, count (2ms)
ok refuses bad input ("POST", "/links", "\{\"url\": \"javascript:alert(1)\"\}", 400) (1ms)
ok refuses bad input ("POST", "/links", "not json", 400) (0ms)
ok refuses bad input ("GET", "/zzzz", "", 404) (0ms)
ok refuses bad input ("GET", "/links/!", "", 404) (0ms)
ok refuses bad input ("DELETE", "/links", "", 404) (0ms)
6 tests, 0 failed (3ms)
The table test sends five kinds of bad input and checks each status, and each case shows up as its own line.
main
Finally, main decides where the authority goes:
// A link shortener.
//
// tessera run main.tsr a demo: server and client in one run
// tessera run main.tsr -- serve 8080 serve on 127.0.0.1:8080
use http
use sql
use store
use api
fn main(root: Root) -> Result[(), dyn Error] {
let db = sql.memory()? "opening the database"
store.setup(db)? "creating the table"
let args = root.args()
if args.get(0) == Some("serve") {
let port = args.get(1) ?? "8080"
let l = root.listen.on("127.0.0.1:{port}")?
root.stdout.print("serving on http://127.0.0.1:{port}")
return Err(http.serve_forever(l, |r| api.handle(db, r)))
}
demo(root.listen, root.net.only("127.0.0.1"), root.stdout, db)
}
// Starts the server on a free port and talks to it.
fn demo(listen: Listen, net: Net, out: Stdout, db: sql.Db) -> Result[(), dyn Error] {
let l = listen.on("127.0.0.1:0")?
let base = "http://127.0.0.1:{l.port()}"
let lines = group g {
g.go(|| {
let _ = http.serve(l, 4, |r| api.handle(db, r))
})
g.go(|| client(net, base))
}
for line in lines[0] {
out.print(line)
}
Ok(())
}
fn client(net: Net, base: Str) -> List[Str] {
let mut log: List[Str] = []
let steps = [
("POST", "/links", "\{\"url\": \"https://example.com/a/long/page\"\}"),
("GET", "/1", ""),
("GET", "/links/1", ""),
("GET", "/nope", ""),
]
for (method, path, body) in steps {
let headers = [http.Header { name: "Content-Type", value: "application/json" }]
match http.request(net, method, base + path, headers, body) {
Ok(r) => {
let to = r.header("location").map(|u| " -> {u}") ?? ""
log.push("{method} {path}: {r.status}{to} {r.body}".trim())
},
Err(e) => log.push("{method} {path}: {e}"),
}
}
log
}With -- serve 8080, main listens on that port and passes each request to api.handle, through a closure that captures only the database. Without arguments, it runs a demo: demo starts the server on a free port in one task and a client in another, inside a group, and prints what the client saw. The client gets a Net narrowed to 127.0.0.1.
tessera caps confirms the layering. Here it is with the standard library's own functions filtered out:
$ tessera caps main.tsr | grep -v -E '^(sql|http|json|url|text|hex|format)\.'
store.setup pure
store.add pure
store.find pure
store.visit pure
codes.alphabet pure
codes.encode pure
codes.decode pure
codes.List::find_index pure
codes.valid_url pure
api.handle pure
api.create pure
api.stats pure
api.follow pure
api.link_json pure
api.request pure
api.test_db pure
main Root
demo Net, Stdout, Listen
client Net
Only main, demo and client hold capabilities. Everything in codes, store and api is reported as pure, because none of it takes a capability. The store and the API do change a database, but only the one whose sql.Db they were handed, the same way a function given a list can only change its own copy of that list.
And the demo:
$ tessera run main.tsr
POST /links: 201 {"code":"1","url":"https://example.com/a/long/page","hits":0}
GET /1: 302 -> https://example.com/a/long/page
GET /links/1: 200 {"code":"1","url":"https://example.com/a/long/page","hits":1}
GET /nope: 404 no link nope
To serve it for real, run tessera run main.tsr -- serve 8080 and try it with curl:
$ curl -X POST localhost:8080/links -d '{"url": "https://example.com/some/long/page"}'
$ curl -i localhost:8080/1
Where to go from here
Some ideas to extend the service, each of which uses something from an earlier chapter:
- Keep links across restarts: open the database with
sql.open(root.fs, "links.db")and grant a directory in tessera.toml (chapters 12 and 14). - Add
DELETE /links/CODE: one more arm inhandle, one more function instore, and a case in the table test (chapters 6 and 11). - Give each request a deadline with
time.within(chapter 13). - Move
codes.tsrinto a package of its own, with nocaps, and use it as a dependency (chapter 7).
Thank you
That's the end of the guided tour. You've seen a language where every effect a function can have is in its signature, where values never change behind your back, where concurrent output is ordered, and where the compiler can tell you exactly what each part of a program, and each dependency, is able to do. The appendices that follow are for reference: keywords and operators, the tools, how the compiler works, and what isn't there yet.