Can we setup PeerTube on naked OpenBSD ?

Without Nginx actually.

Why ?

Several reasons for this questioning:

  • Nginx is not in Base.
  • We have http daemons in Base, could we use them instead of Nginx?
  • These daemons are designed with the OpenBSD culture in mind and use its successive improvements, notably pledge().
  • Nginx is, from what I understand, a really twisted thing.

Basically, we have the basics, why not use them. What’s more, Relayd will then use LibreSSL directly and we’ll have a TLS connection with the latest protocols.

Inconvenient

We’ll use TLS 1.3 (when Nginx only delivers 1.2 on OpenBSD) but we’ll only use http/1.1 when Nginx delivers http/2. So it’s a blessing in disguise.

We have to transform the entire Nginx configuration that is provided by the PeerTube developer into a relayd + httpd configuration, which is far from simple.

Preliminaries

I recommend reading Michael W Lucas' books for OpenBSD, Relayd and TLS. It will save you a lot of trouble.

Also, have the relayd.conf manual open. Seriously, this thing makes no sense without the manual. It does not makes sense with it either. But you have a chance to go through somehow.

I’m going to take important bits from previous articles to describe the installation process. The advantage is that you don’t need to go back and read them, unless you want the history or a more detailed explanation.

Installation / Upgrade

You have to install PostgreSQL, Redis and nodejs. There is no way around it.

    doas pkg_add postgresql-contrib redis node python3 youtube-dl ffmpeg

Don’t forget to check the PostgreSQL port doc, it doesn’t just come oven-ready like the Brexit deal - sic. At the same time you want to set up a PeerTube instance, it will be a challenge sometimes…

You have to create a dedicated user.

    doas useradd -L daemon -d /var/www/peertube _peertube
    doas chown _peertube:_peertube /var/www/peertube
    doas -u _peertube mkdir /var/www/peertube/(versions|storage|config) 

Note that as the name implies, storage will contain the videos and all other data of the instance apart from the configuration and the program itself. You might want to put it on a dedicated mount point. With a big partition. A very large partition.

Oh, by the way. PeerTube itself, the program… it weighs around 700 MB. It also requires space to compile and install. As does PostgreSQL, which will happily exceed one gigabyte. So don’t forget to regularly make room in /var, get rid of old versions, check that the logs are running and not cluttering up the space. We don’t play in the small pool anymore.

I don’t use the PeerTube upgarde script. The symbolic link created at the end is buggy. And node plays tricks with /tmp.

Well, let’s start again. We download the latest version of PeerTube to the adequate directory.

    cd /var/www/peertube/versions
    doas -u _peertube wget https://github.com/Chocobozzz/PeerTube/releases/download/$VERSION/peertube-v$VERSION.zip
    doas -u _peertube unzip peertube-v$VERSION.zip

During compilation, npm/yarn will ask for /tmp/node. Better take care of it now. And also install yarn itself.

    cd /tmp
    ln -s /usr/local/bin/node .
    doas npm install --global yarn

Now let’s compile.

    cd /var/www/peertube/versions/peertube-$VERSION
    doas -u _peertube yarn install --production --pure-lockfile

Here is it running …

    yarn install v1.22.4
    [1/5] Validating package.json...
    warning peertube@3.1.0: The engine "postgres" appears to be invalid.
    warning peertube@3.1.0: The engine "redis-server" appears to be invalid.
    warning peertube@3.1.0: The engine "ffmpeg" appears to be invalid.
    [2/5] Resolving packages...
    warning Resolution field "oauth2-server@3.1.0-beta.1" is incompatible with requested version "oauth2-server@3.0.0"
    warning Resolution field "http-signature@1.3.5" is incompatible with requested version "http-signature@~1.2.0"
    ...
    [4/4] Building fresh packages...
    Done in 39.90s.
    Done in 60.43s.

Tada

Don’t forget to link the directory like following. And copy the default.yaml to your configuration.

    cd /var/www/peertube
    doas rm peertube-latest/ && doas ln -s versions/peertube-v$VERSION ./peertube-latest
    cp /var/www/peertube/peertube-latest/config/default.yaml /var/www/peertube/config/default.yaml

Rcctl

Now, we need to be able launching PeerTube like a regular daemon. Ok.

Never did I succeed.

So actually I finally used a Node hypervisor, just like there are for PHP or Python. PM2. Let’s install it like the doc says.

doas npm install pm2 -g

I write a ecosystem.config.js file in /etc/pm/. And actually as you understood, it’s pm2 which starts. Which itself shall start peertube.

module.exports = {
apps : [{
    name: 'peertube',
    cwd:  '/var/www/peertube/peertube-latest/',
    "listen_timeout" : 2000,
    env: {
        "USER":         "_peertube",
        "NODE_ENV":     "production",
        "HOME":         "/var/www/peertube/",
        "NODE_CONFIG_DIR": "/var/www/peertube/config/",
    },
    script: '/var/www/peertube/peertube-latest/dist/server',
    }, ]
};

Just start PM2 like any other daemon now with rcctl.

So let’s write a /etc/rc.d/peertube file, and don’t forget to enable it in /etc/rc.conf.local :

#!/bin/ksh

daemon="/usr/local/bin/pm2"

daemon_user="_peertube"

. /etc/rc.d/rc.subr

pexp="node: PM2.*God Daemon.*"

rc_start() {
    ${rcexec} "${daemon} start /etc/pm/ecosystem.config.js"
}

rc_reload() {
    ${rcexec} "${daemon} reload /etc/pm/ecosystem.config.js"
}

rc_cmd $1    

We cannot yet launch the whole thing as it lacks the http(s) conf'. PeerTube checks during its initialization that it can reach its own webserver with https. So lets' take care of it.

Httpd

Httpd web server configuration is extremly simple as it serves only static files.

/etc/httpd.conf

server "tube.example.com" {
        listen on * port www

        location "/.well-known/acme-challenge/*" {
                root "/acme"
                request strip 2
        }
        location * {
                block return 302 "https://$HTTP_HOST$REQUEST_URI"
        }
}

server "tube.example.com" {
        listen on * port 8000

        root "/var/www/peertube/storage"

        location "/.well-known/acme-challenge/*" {
                root "/acme"
                request strip 2
        }
}

You will notice that I do not redirect the ACME challenge to https. This corresponds well to my installation with Buypass.

Relayd does the most complicated work. And we’re going to tackle it.

Relayd

Relayd will be used to terminate the TLS connection and dispatch the outgoing http streams

  • either to Httpd to serve static content.
  • or to the peertube server itself (you know, the one listening on port 9000 on localhost).
  • in addition to that we can terminate the live connection in ipv6, which is not possible natively.

We’ll analyse the relayd.conf, starting from the top.

NB: as described by M Lucas, protocols are before relays, no matter if it makes sense or not.

/etc/relayd.conf

    # Macros
    #
    ext4="192.0.2.10"
    ext6="2001:DB8:96b1::100"
    webhost="lo0"

    #
    # Global Options
    #
    # interval 10
    # timeout 1000
    # prefork 5

    #
    # Each table will be mapped to a pf table.
    #
    table <webhosts> { $webhost }
    table <apihosts> { 127.0.0.1 }

At the top of the file, macros are used to indicate the addresses where incoming requests arrive. In other words: your public addresses.

The webhosts and apihosts tables can actually be separated. For example, you can run the peertube and httpd daemons on separate hosts. However, the peertube daemon is listening in priority in ipv4. Hence my 127.0.0.1 in apihosts. I have no idea if it is possible to use load balancing (building a large instance with multiple backend servers distributed via these tables).

The rtmp protocol is presented so that it can be relayed in ipv6 at the very end of the file.

    protocol rtmp {
            tcp { nodelay, sack, socket buffer 65536, backlog 128 }
    }

Well, now we’re really getting down to business.

The first part describes the generic options. I have tried to use the same values as the configuration provided by PeerTube.

The tls keypair options are described in the manual. That you go back and read from time to time isn’t it… ISN’T IT …

http websockets is important for PeerTube.

The ciphers line can’t be replaced by HIGH because apparently you include declassified suites. And also: you need one line. You cannot have several options and I couldn’t make a return with \ . Deal with it.

    http protocol https {
            match request header append "X-Forwarded-For"   value "$REMOTE_ADDR"
            match request header append "X-Forwarded-By"    value "$SERVER_ADDR:$SERVER_PORT"

            match response header set "Feature-Policy" value "camera 'none'; microphone 'none'"
            match response header set "Strict-Transport-Security" value "max-age=31536000"
            match response header set "X-Content-Type-Options" value "nosniff"

            # Various TCP options
            tcp { nodelay, sack, socket buffer 65536, backlog 128 }

            tls { tlsv1.3 tlsv1.2, ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384 }

            tls edh params auto
            tls keypair tube.example.com

            http websockets

Here we start to differentiate the requests according to their destination. I will try to stick as much as possible to the configuration provided by PeerTube. We could certainly do a lot more. The Nginx conf file is way more granular. For example, it restricts the allowed HTTP methods, which I can’t do.

            match request path "/api/v1/videos/upload" tag "UPLOAD"
            match response tagged "UPLOAD" header set "Access-Control-Allow-Methods" value "POST, HEAD"
            match response tagged "UPLOAD" header set "X-File-Maximum-Size 8G always"

            match request path "/client/*" tag "CLIENT"
            match response tagged "CLIENT" header set "Cache-Control" value "public, max-age=31536000, immutable"

            match request path "/static/*" tag "STATIC"
            match response tagged "STATIC" header set "Access-Control-Allow-Methods"        value "GET, OPTIONS"
            match response tagged "STATIC" header set "Access-Control-Max-Age"      value "1728000"
            match response tagged "STATIC" header set "Content-Type"                value "text/plain charset=UTF-8"

            match response tagged "STATIC" header set "Access-Control-Allow-Headers" value "Range,DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type"

            pass request tagged "STATIC" forward to <webhosts>

            pass request path "/tracker/socket"     forward to <apihosts>
            pass request path "/socket.io"          forward to <apihosts>
            pass request path "/*"                  forward to <apihosts>
    }

If you can do better, please comment. I will update my installation and this article.

Now we have the three relays, tls in ipv6, tls in ipv4 and rtmp (for live broadcasting) relayed from ipv6 to PeerTube listening in ipv4.

    relay wwwtls6 {
            # Run as a SSL/TLS accelerator
            listen on $ext6 port https tls

            protocol https

            # httpd serve static and the rest on port 8000
            forward to <webhosts> port 8000

            # peertube stuff listens on 9000
            forward to <apihosts> port 9000
    }

    relay wwwtls4 {
            # Run as a SSL/TLS accelerator
            listen on $ext4 port https tls

            protocol https

            # httpd serve static and the rest on port 8000
            forward to <webhosts> port 8000

            # peertube stuff listens on 9000
            forward to <apihosts> port 9000
    }

    relay live {
            listen on $ext6 port 1935
            forward to <apihosts> port 1935
            protocol rtmp
    }

Conclusion

There it is.

The whole installation of PeerTube on OpenBSD, without Nginx but Httpd and Relayd.

You can open the bottle.

    doas rcctl restart peertube