REST APIs with CHICKEN Scheme

Estimated reading time: 5 minutes

A few months ago, I stumbled upon the Matrix network and protocol. A pretty interesting chat system with cool features such as distributed chat rooms, chat log replication, VoIP, end to end encryption…

A few weeks ago I came back to it and thought “hey! why not write a client in CHICKEN for this network?”, even though I had no previous experience with REST APIs whatsoever.

I then took a look a the CHICKEN egg index to see if there was anything to help me do that, and guess what? There are plenty of great stuff for web programming!

As Matrix’s network uses HTTP for REST requests, I figured I would use the http-client egg to do all the network stuff for me. It takes care of https handling by transparently using the openssl egg, it automatically reuses connections when possible and it’s super easy to use, as you just have to give it a URL and two procedures that will write and read the request’s and response’s body.

Matrix’s data is serialized in the JSON format, for that I used the medea egg: a very cool JSON parser that lets you choose how the JSON should be decoded into Scheme data.

The last egg I decided to use is rest-bind. It exports a very neat macro called define-method that defines procedures that will execute HTTP requests for you and return the response. For example, this declaration would define a new procedure versions that returns the scheme representation of the JSON data sent by the server when the https://matrix.org/_matrix/client/versions URL is requested:

(define-method (versions)
  "https://matrix.org/_matrix/client/versions"
  #f read-json)
(versions) ;; -> ((versions . #("r0.0.1" "r0.1.0" "r0.2.0")))

One problem I faced when using this macro, is that the Matrix API wants the client to put a session token called access_token in every request once you’re logged in, as a query parameter. Of course I didn’t want to have to supply that to every procedure call defined with define-method.

Hopefully define-method doesn’t use the http-client egg directly! It excepts a procedure called call-with-input-request to be defined. This procedure usually comes from the http-client egg but you can define it yourself if you want to fiddle with the request before handing it to http-client!

Of course that’s what I did, I defined my own version of call-with-input-request that adds the access_token query parameter to the request URL from a Scheme parameter. I also took this opportunity to make the server’s URL a Scheme parameter as well, that way the user of the library can have multiple Matrix sessions, to different servers, in their program. This custom procedure also adds a bunch of headers to the HTTP request.

Here is how it looks like (where http:call-with-input-request is the actual procedure from http-client):

(define (call-with-input-request req writer reader)
  (unless (server-uri)
    (error "Server URI not set, use (init!) first"))
  (let* ((uri-rewritten (update-uri (request-uri req)
                                    scheme: (uri-scheme (server-uri))
                                    host: (uri-host (server-uri))
                                    port: (uri-port (server-uri))
                                    query: (append (if (access-token) `((access_token . ,(access-token))) '())
                                                   (uri-query (request-uri req)))))
         (headers-rewritten (headers (cons* '(accept application/json)
                                            (if (member (request-method req) '(PUT POST))
                                                '((content-type application/json))
                                                '()))
                                     (request-headers req)))
         (request-rewritten (update-request req
                                            uri: uri-rewritten
                                            headers: headers-rewritten)))
    (http:call-with-input-request request-rewritten writer reader)))

The last thing I did was to define my own little macro, define-endpoint, because I’m lazy and I hate repeating the same stuff over and over. This macro is a simple pattern-matching syntax-rules macro that expands to define-method.

(define-syntax define-endpoint
  (syntax-rules (GET POST PUT)
    ((define-endpoint GET decl)
     (define-method decl api-uri #f read-json))
    ((define-endpoint POST decl)
     (define-method decl api-uri json->string read-json))
    ((define-endpoint PUT decl)
     (define-method decl (make-request uri: api-uri method: 'PUT) json->string read-json))))

With that macro, the only code I have to write to add new API endpoints to my library looks like this:

(define-endpoint GET (login-schemes "login"))
(define-endpoint POST (login "login"))
(define-endpoint POST (logout "logout"))
(define-endpoint GET (sync "sync" #!key filter since timeout full_state set_presence timeout))
(define-endpoint PUT (room-send "rooms" room-id "send" event-type transaction-id))
(define-endpoint GET (get-filter "user" user-id "filter" filter-id))
(define-endpoint POST (create-filter "user" user-id "filter"))

And I use them as regular Scheme procedures like so:

(sync since: "whatever" timeout: 30000)
(room-send "!vfFxDRtZSSdspfTSEr:matrix.org"
           'm.room.message
           'some-transaction-id
           '((msgtype . "m.text")
             (body . "Hello world!")))