Webhotel using traefik, docker and ssh

I’m the sysadmin behind some of my family’s basic websites. My mom runs her own website where she blogs, post pictures and writes about environmental topics. It’s one of those “old-fashioned” websites made completely in just raw HTML. It’s pretty cool, and I’m proud of having such a cool mom :)

Anyway, for some years now I’ve been maintaining the domain, email and the little server that runs her website. Just a cheap VPS. Up until last month everything was running fine on a simple VPS with nginx installed, but then LetsEncrypt decided to not work any more on the very old kernel I was running. So it was time for an upgrade.

I’ve been wanting a place to run my docker containers for a while, and I’ve been wanting to try out both traefik and ansible. Traefik because I’ve been working a fair bit with building a reverse-proxy for docker containers at work and had heard that traefik was pretty nice. Ansible because it’s a pretty well established DevOps tool to configure servers, and it feels more tailored to the task than say terrafrom (I’m only running a single cheap VPS). I’m also sticking with the same provider, glesys, because I don’t like how aws, azure and other big cloud providers are eating up the internet.

At the same time, I still want to respect how my users were connecting to my old server. Teaching them to use new tool would be hard. Docker, git, aws s3 sync? Lol, no way. They have their tried and tested filezilla client and they know how it works. It also makes it harder that I currently live in a different country than them, meaning I’m limited in being able to support them only via video chat or when I come back to visit.

Ansible playbook

Here’s what my ansible playbook looks like. I found it generally pretty easy to use. The structure looks roughly like this:

├── ansible.cfg
├── hosts
├── main.retry
├── main.yml
├── readme.md
├── roles
│   └── static-sites
│       ├── tasks
│       │   └── main.yml
│       └── templates
│           ├── docker-compose.yml.j2
│           └── sshd_config.j2
├── templates
│   └── traefik
│       └── traefik.toml.j2
└── vars
    └── static-sites.yml

There were predefined playbooks I could download from ansible-universe that I used for both docker and traefik. For setting up sftp I built my own role, static-sites. Here’s my main.yml.

- hosts: webservers
  roles:
    - geerlingguy.docker
    - kibatic.traefik
    - static-sites
  vars:
    traefik_template: templates/traefik/traefik.toml
  vars_files:
    - vars/static-sites.yml

vars/static-sites.yml is how I supply the multiple sites to later enumerate on. I structured it as follows:

---
static_sites:
  - site: hugallery.com
    password: <snip of hash copied from old server's /etc/shadow>
    state: present
    domains:
      - hugallery.com
      - www.hugallery.com
  - site: joakim.uddholm.com
    password: <snip of hash>
    state: present
    domains:
      - joakim.uddholm.com
  - site: server.blacknode.se
    password: <snip of hash>
    state: present
    domains:
      - server.blacknode.se
      - blacknode.se

In the templates/traefik/traefik.toml I basically supply the traefik configuration file. I enable the traefik dashboard on localhost:8080, meaning it won’t be exposed to the internet. I can just proxy to it via ssh using ssh -N -f -L 8080:localhost:8080 root@server.blacknode.se. The acme part is to enable LetsEncrypt and having traefik automagically fetch certificates for me. The important part is basically onHostRule, which I believe tells traefik to fetch for each frontend host I later define.

logLevel = "INFO"

defaultEntryPoints = ["http", "https"]

[docker]
endpoint = "unix:///var/run/docker.sock"
domain = "localhost"
watch = true

[api]
entryPoint = "api"
dashboard = true


[entryPoints]
  [entryPoints.http]
  address = ":80"

  [entryPoints.https]
  address = ":443"
    [entryPoints.https.tls]

  [entryPoints.api]
  address = "127.0.0.1:8080"


[acme]
email = "<my email>"
storage = "/etc/acme.json"
onHostRule = true
caServer = "https://acme-v02.api.letsencrypt.org/directory"
entryPoint = "https"
  [acme.httpChallenge]
  entryPoint = "http"

static-sites role

To actually serve the static sites I use nginx docker containers. Currently one container per site, although I’m thinking of using just a single container instead and configuring the different sites as virtualhosts. I then mount for each site a www inside the home directory of the webuser.

To illustrate how this is set up, here’s the docker-compose.yml.j2 template. The traefik.frontend.rule are what tell traefik which domains to route for.

version: "3"

services:
{% for site in static_sites if site.state != "absent" %}
  {{site['site']}}:
    image: nginx:alpine
    labels:
      - "traefik.frontend.rule=Host:{% for domain in site['domains'] %}{{domain}}{% if not loop.last %},{% endif %}{% endfor %}"
      - "traefik.frontend.headers.SSLRedirect=true"
    volumes:
      - /home/{{site['site']}}/www:/usr/share/nginx/html:ro
    restart: always
{% endfor %}

To create the users, configure sshd and start the nginx docker containers I have the following steps inside the role’s main.yml. Here ansible’s user and file modules are pretty handy. To enable removal of sites I use when: item['state'] != "absent". That way I can remove a website later on by change the state of it in the vars/static-site.yml file.

- name: "Add new sshd_config"
  template:
    src: sshd_config.j2
    dest: /etc/ssh/sshd_config

- name: "Reload sshd"
  service:
    name: sshd
    state: reloaded

- name: "Add the sftpusers group"
  group:
    name: sftpusers
    state: present

- name: Add the sftp users
  user:
    append: true
    name: "{{item['site']}}"
    groups: sftpusers
    create_home: true
    home: "/home/{{item['site']}}"
    password: "{{item['password']}}"
    shell: /bin/false
    state: "{{item['state']}}"
    remove: yes
  with_items:
    - "{{static_sites}}"

- name: Create web chroots
  file:
    path: "/home/{{item['site']}}"
    state: directory
    owner: "root"
    mode: 0755
  with_items:
    - "{{static_sites}}"
  when: item['state'] != "absent"

- name: Create web directories
  file:
    path: "/home/{{item['site']}}/www"
    state: directory
    owner: "{{item['site']}}"
    mode: 0755
  with_items:
    - "{{static_sites}}"
  when: item['state'] != "absent"

- name: Create dir for static sites docker-compose
  file:
    path: /root/static-websites
    state: directory
    owner: "root"

- name: "Add docker-compose for static sites"
  template:
    src: docker-compose.yml.j2
    dest: /root/static-websites/docker-compose.yml

- name: "Start the nginx containers"
  command: docker-compose up -d
  args:
    chdir: /root/static-websites

Finally I add some configuration to sshd to sandbox the ftp users. For this I was actually able to reuse an old blog post. Hooray for documentation :)

Migrating from the old server

I wanted the migration to be as smooth as possible. To do this I copied the server ssh keys (so that my users won’t get a warning about a the new host keys) from /etc/ssh/ on the old server to the new. I also copied the old password hashes from /etc/shadow, and of course the actual content of the sites.

That’s it!

Yup. That was basically it. It took some time to configure the webroots correctly with the right permissions, and it also took some time to get the correct traefik letsencrypt and frontend rules in place. Other than that though it went very smooth. Super happy with how easy to use traefik is. When I next have time I hope to start deploying some more containers with some actual running code inside them, but I don’t suspect it will be very hard.