I wrote some tools to help with migrating an existing imap account from one server to another, so that you can keep your emails when you switch your domain name to a new provider. I think normally your email client might do this for you automatically just by you changing the server settings, but I have some users only using a webmail client, meaning they don't have a local copy to sync to the new server. Plus it's always good to have a backup.

To solve this I wrote some scripts that let me download an entire mailbox and then reupload it to a new server. You can find them here: https://github.com/Tethik/imap-tools

I initially intended to use these for migrating a domain name and the email to a new provider. Unfortunately I never got to carry out the migration, so I only got to try this out with some test accounts. The way I imagine the whole process of moving email servers with a domain name looks something like this:

1. Download all accounts and their emails locally using my script.
2. Do the domain transfer.
3. Set up the new imap accounts at the new provider.
4. Use the upload script for each account to upload all old emails to the new accounts.

At work we use git flow to organize our git repositories. master is the production branch, develop is the staging branch, and longer features get their own feature/* branch. On these branches, everything is built into docker images that are uploaded to a registry to later be deployed. git flow also gives us some basic version tagging that we want to use to track our releases.

I wanted to tag these docker images so that we could easier refer to them in our deployment. Where there would be a latest tag that would always be the latest version, and maybe a tag for every version.

I came up with a python script that wraps around docker and git to automatically generate these tags for me, with some basic customization. It's pretty rough, but you can find it here: https://github.com/Tethik/lame-cli-programs/tree/master/docker-branch-tagging

Usage

docker-branch-tagging init generates a default .docker-branch-tagging file that looks something like the following.

{
    "develop": ["latest","develop-{CIRCLE_BUILD_NUM}","{git_branch}"],
    "feature/(.+)": ["{git_branch}"],
    "master": ["master","master-{CIRCLE_BUILD_NUM}","{git_latest_version_tag}"]
}

The keys are regex, and the values are python format strings. The values get passed the current environment variables as well as two special case variables git_branch and git_latest_version_tag. The script will simply look for any keys matching the current git branch and perform the templating on the values to generate the different tags.

Doing a docker-branch-tagging build aws-blahabhla.com/example on the master branch would then result in something like the following.

docker build -t aws-blahabhla.com/example:master -t aws-blahabhla.com/example:master-123 -t aws-blahabhla.com/example:0.2.1 .

docker-branch-tagging push would then perform the docker push. Ungefär like so:

docker push aws-blahabhla.com/example:master 
docker push aws-blahabhla.com/example:master-123
docker push aws-blahabhla.com/example:0.2.1

On CircleCI, which is where we do our continous integration, the step for building and pushing the docker containers generally looks something like this now.

  build:
    docker:
      - image: circleci/python3
    steps:
      - run: sudo pip install awscli
      - run: sudo pip install "git+https://github.com/Tethik/lame-cli-programs#egg=docker_branch_tagging&subdirectory=docker-branch-tagging"
      - checkout
      - attach_workspace:
          at: .
      - setup_remote_docker
      - run: "docker login -u AWS -p $(aws ecr get-authorization-token --output text --query authorizationData[].authorizationToken | base64 --decode | cut -d: -f2) $DOCKER_REPOSITORY"
      - run: docker-branch-tagging build $DOCKER_REPOSITORY
      - run: docker-branch-tagging push $DOCKER_REPOSITORY

The attach_workspace is used to copy over whatever dependencies that may have been installed via e.g. npm or binaries/webpacks that may have been built in a previous workflow step. DOCKER_REPOSITORY is the environment variable I set in the project configuration to the AWS ECR uri.

Just for fun, I decided to try making a CI/CD pipeline for my CV. This post will be a bit rough and jumbled, sorry about that, probably would have been better to split this into several posts. Anyhow, here's how I did it.

Building the LateX file via Docker

Before I used to just update my CV via ShareLatex, so I didn't have any Latex packages installed on my system. Installing Latex is usually a confusing mess of packages, so I figured using docker might be a good fit. In addition I knew that CircleCI takes a docker image to launch it's jobs in, so I could reuse it later.

Luckily I found that someone else had made a docker image for xelatex. With some adjustments I made my own image that contained everything I needed for compiling my tex files.

Then I uploaded it to docker hub for free (since it's open source).

Setting up CircleCI

With the previously mentioned docker image I defined a CircleCI job step as follows.

  build_pdf:
    docker:
      - image: tethik/xelatex:latest
    steps:
      - checkout
      - run: make 
      - attach_workspace:
          at: .  
      - persist_to_workspace:
          root: .
          paths:
            - cv.pdf

The persist_to_workspace step saves the pdf so that it can be reused later in the deploy job using the attach_workspace step. The current attach_workspace step in the config above coming before the persist_to_workspace is to collect some scripted parts of the CV that I try to generate automatically. More on this later.

Deploying to Github Pages

For CD (Continuous Delivery) part of the process I needed somewhere to publish the document. I do have my own domains and servers, but I'd rather not have to give access to CircleCI to ssh or ftp into these. The hacky and cheap solution was to just reuse the same github repository and enable the Github pages feature. To do this I enabled Github Pages on the master branch, which I kept empty except for the final pdf output. Then I set up another step in the CircleCI config to commit and push the new pdf file to the master branch.

The CircleCI config step looks like this.

  deploy:
    docker:
      - image: circleci/node:8.9
    steps:
      - checkout
      - attach_workspace:
          at: .      
      - run: .circleci/deploy.sh

The deploy script itself looks like this.

#!/bin/sh

git config --global user.email "circleci@blacknode.se"
git config --global user.name "CircleCI Deployment"
mv cv.pdf ..
git fetch --all
git reset --hard origin/develop
git checkout master
mv ../cv.pdf .
git add cv.pdf
git commit -m "PDF build $CIRCLE_SHA1"
git push origin master

By default CircleCI generates a key on your github repo which only has READ access. To get around this I created a new ssh key and added it as a deploy key to the github repository. Then I removed the original key from the CircleCI project configuration, and added the new ssh key that I generated.

The final step was to add the redirect from my main homepage (this site) to the github pages link where the document would be hosted. Since the server is running Apache I could use the following config.

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteRule ^(.)cv$ https://tethik.github.io/curriculum-vitae/cv.pdf
RewriteRule ^curriculum-vitae https://tethik.github.io/curriculum-vitae/cv.pdf
</IfModule>

It's a bit hacky, but it works :)

Automatically updating the document

The next fun step I wanted to do was add some parts to the document that would be automatically generated. Because writing CVs is boring and too manual. For now though I just coded something simple: A script that summarizes all my pull requests made on Github that are in some sense Open Source. i.e. not to repos that I own myself or repos that were created for e.g. schoolwork.

To organize the LateX files I set up the repo as follows. I wanted to keep the generated files separate from the main latex file that I copied from before.

partials/ ->  generated tex files.
src/github/ -> python script that generates into partials/
cv.tex

Inside the main cv.tex file I could then refer to the scripted content using the subfile package.

\subsection{Github Open Source Contributions}
\subfile{partials/pull_requests}

Inside the src/ folder I mean to keep scripts that generate the partials. The src/github/ folder contains a script that generates a pull_requests.tex into the partials/ folder. It talks the to Github GraphQL API and summarizes the info into a table.

I added the following CircleCI job config. This goes before the previously defined build_pdf step.

  build_partials:
    docker:
      - image: kennethreitz/pipenv
    steps:
      - checkout      
      - run: 
          command: pipenv install 
          working_directory: ~/project/src/github/
      - run: 
          command: pipenv run make
          working_directory: ~/project/src/github/
      - persist_to_workspace:
          root: .
          paths:
            - partials/

Again I persist_to_workspace to keep the resulting partial/*.tex files. Pipenv handles the python dependencies beautifully. The github api key and a blacklist of repos to ignore is passed through environment variables.

This is what the table looks like in the PDF.

In the future I'd like to add more content that's automatically generated. E.g. pypi and npm packages published, total github commit stats etc.

Final Result

You can find the repository here, the latest pdf built by the pipline here, and the CircleCI project here.

I still need to update my CV though.

Over the weekend I wrote a small tool that will automatically update the /etc/hosts file with your running docker container. You can find the script here: https://github.com/Spielstunde/docker-hosts-update

Example usage

First create a docker network. This will allow the containers to connect to each other and automatically resolve hostnames. This is most of the magic tbh.

docker network create evilcorp.internal

Now you can start new containers in the network:

docker run --network evilcorp.internal --rm -it nginx
docker run --network evilcorp.internal --name hello --rm -it nginx

After starting the above two container you can then start ...

sudo docker-hosts-update

... and you should see a section like this in your /etc/hosts file.

# ! docker-hosts-update start !
# This section was automatically generated by docker-hosts-update
# Don't edit this part manually :)
172.20.0.2   hello.evilcorp.internal
172.20.0.3   friendly_golick.evilcorp.internal
# ! docker-hosts-update end   !

Because these networks are on the same bridged network, docker will first of all ensure that their hostnames resolve to each other. E.g. in the first container it will be able to connect to the "hello" container, either via hello.evilcorp.internal or just hello.

What my script does is easily enable the host machine to resolve the hostnames too. This saves you the trouble of either getting the ip manually (because hostname, duh) but also removes the need for other approaches where you need to assign different ports to different containers, and then use some sort of service discovery tool and proxy to manage them. On both host and container the url http://hello.evilcorp.internal will resolve to the correct container.

Finally when you stop the containers using Ctrl+C or docker stop hello, you'll see the lines automatically removed from the hosts file, if docker-hosts-update is still running.

Just a small howto for creating new application launchers in ubuntu, and I guess other similar distros. These will typically automatically show up in your start menu or launchers.

Simply create a <application>.desktop file in the ~/.local/share/applications/ or /usr/share/applications/ folder. <application> can be whatever.

At work I use the following desktop entry as a shortcut to open up an editor for our documentation. Here's an example file:

[Desktop Entry]
Encoding=UTF-8
Name=Documentation
Exec=code code/docs/
Icon=accessories-text-editor
Terminal=false
Type=Application
Categories=Development;

The following is copied for reference from https://help.ubuntu.com/community/UnityLaunchersAndDesktopFiles

Version is the version of this .desktop file.

Name is the name of the application, like 'VLC media player'.

Comment is a phrase or two describing what this program does, like 'Plays your music and videos files'.

Exec is the path to the executable file. The full path to the executable file must be used only in case it isn't in any of the paths specified in the $PATH variable. For example, any files that are inside the path /usr/bin don't need to have their full path specified in the Exec field, but only their filename.

Icon field is the icon that should be used by the launcher and represents the application. All icons that are under the directory /usr/share/pixmaps don't need to have their full path specified, but their filename without the extension. For example, if the icon file is /usr/share/pixmaps/wallch.png, then the Icon field should be just 'wallch'. All other icons should have their full path specified.

Terminal field specifies whether the application should run in a terminal window or not.

Type field specifies the type of the launcher file. The type can be Application, Link or Directory, but this article covers the 'Application' type.

Categories field specifies the category of the application. It is used by the Dash so as to categorize the applications.

Some other very good references: https://wiki.archlinux.org/index.php/Desktop_entries#Autostart https://specifications.freedesktop.org/desktop-entry-spec/latest/

Hello again, blog. It's been a while. I had a lot of topics that I wanted to write about the past year, but in the end I never managed to finish and publish anything. I think I'm generally ok with that though.

In the german lesson I had this week, we came across the "Ferryman problem" as part of an exercise. I found the following English explanation on the Mathswork website.

A man needs to cross a river with a wolf, a goat and a cabbage. His boat is only large enough to carry himself and one of his three possessions, so he must transport these items one at a time. However, if he leaves the wolf and the goat together unattended, then the wolf will eat the goat; similarly, if he leaves the goat and the cabbage together unattended, then the goat will eat the cabbage. How can the man get across safely with his three items?

During the lesson I then spent way too much time trying to draw a graph of all the different states. So instead I made a script using python and the graphviz library to draw this graph for me.

The different states are represented by a tuple of three. The leftmost value of the tuple is the set of who are on the initial (left) bank of the river. The second or middle value is the set for who are on the boat. The rightmost value is the destination bank of the river. Each character of the problem is identified by the letters. W is the wolf, S is the (Schafe), K is the cabbage (Kohl) and F is the ferryman.

Some example states:

('FKSW', '', '') # Our initial starting state with everyone the first riverbank.
('KW', 'FS', '') # The cabbage and the wolf are on the first bank. The ferryman and the sheep are on the boat in the river.

Anyhow, too much writing for something that is overkill. I'm pretty sure there are much simpler solutions to this. You can find the graph below.

Last week I went to the cinema with some friends. My friend paid for the ticket so I decided to use the "Swish" app to transfer them the money. As a joke I put <script src="https://blacknode.se/xss"></script> in the description.

Imagine my surprise as the alert popped when I visited the transaction page later on.

I reported this vulnerability to Skandia by calling their head office directly and getting patched through to their security guy. One or two days later I got an irl-mail with a thank you letter as well as two cinema vouchers.

My guess is that because they use aspx-pages they assumed that all input was already escaped for XSS. Aspx-pages typically throws an error and rejects detects any input that has html tags. However, since this came from an (I assume) external system that filtering never applied.

As of today it looks like it's been fixed. So I feel ok publishing it. My friend uses SEB, which was not affected by the vulnerability. It might be worth testing if the vulnerability exists on other banks.

I'd like to write about a part of what was my master thesis project. For my thesis I wrote about a mostly theoretical whistleblowing system. A part of that system was a component which verified that the javascript client, used for the submission of new whistleblowing leaks, had not been tampered with. This was to ensure that we could better trust the client, which may be hosted somewhere outside the organisation.

The idea is simple. Basically I've created a program which allows you to verify a static content on a hidden service. I assume that most people when using Tor are using the Tor Browser. This is the recommended way to stay anonymous, by not standing out you are part of a larger anonymity set. Compare this to the normal internet, where there are multiple browsers with versions and operating systems to keep track of. Because most people on Tor use the same client, we can emulate that client programmatically to create requests that are indistinguishable from those that would come from a real user. Since the verifier is using the same client, it will display the same behaviour and send the same HTTP headers like that of a real user. On the internet this would not work, because these things are so diverse that they can uniquely identify a user. On Tor however, the Tor Browser has been specifically engineered to not fingerprint individual users, meaning that all users mostly look the same.

For a verifier this is useful because it will be difficult for a server to tell apart a verification request from a real request. This allows it to monitor the server without the server being able to alter its behaviour by knowing when its being watched.

One problem this might solve is the untrustworhtiness of doing clientside cryptography in javascript. One problem why we can't trust javascript crypto is that there is no guarantee that a server hosting the cryptographic routines has not introduced a backdoor (like removing the encryption altogether). With this verifier we can have some guarantee that the javascript has not been altered by the server.

Prototype

I implemented a prototype to test out this idea. Here's a link to the repository.

Run it once to stamp the "state" of a hidden service...

tamperfree stamp <url.onion>

...then run it again later to verify that the state has not changed.

tamperfree verify <url.onion>

When stamping a website tamperfree tries to identify and save a secure hash of the raw content received from the server for each path visited. It does this by working as a proxy between the Tor Browser and the Tor Socks proxy that it uses to connect to the network. Then it opens the targeted url in a selenium controlled instance of the Tor Browser. When verifying it captures the same raw content, computes the hashes etc and then it compares against the saved hashes.

There are some major caveats to this tool:

• Can't use it on HTTPS. The proxy can't see the plaintext data as it passes directly to the browser and the encrypted content is not static since I'm guessing different IVs or somesuch are used.
• Can't use it on dynamic websites (i.e. websites that change the content).
• Using it on non-Tor websites is kind of pointless since the server would easily identify it by the user agent.
• Sites requiring user interactions or multiple page loads are not supported. This would require that I spoof user behaviour, adding further complexity to the tool. I think it's doable for smaller sites to build a tree of possible user interactions and generate fake traffic based on every sub-tree. However, I am certain that most users will likely navigate a site the same way, meaning that certain patterns will emerge and make real users stand out from the fake.

So the use case for my tamperfree tool is pretty slim. You're limited to having a single page webapp which loads everything it needs on the first given url. Oh, and it has to run as a tor hidden service.

You want to run this tool often. So often that a malicious server has a higher probability of encountering a verification request than a real request if attempting to tamper by picking a request at random. To do this we simply want the tool to send out more requests to the server than the real requests. For my use-case, a whistleblowing site, the number of real visitors expected is actually pretty low (few visitors per day), so this is something we can do.

Timing is also important. If the verifier runs on a regular schedule then it's trivial for the server to figure out that incoming requests in that specified schedule are verification requests. Therefore it's important that the verifier runs on a random schedule that the server can not determine. E.g. running the verifier every 5 minutes would make it easy for the server to figure out the pattern and only serve it's malicious responces to requests outside that schedule. However, just running a ´sleep(rand())´ would also lead to scenarios where if there are multiple requests happening in quick succession, then the server can see that it's unlikely to be the verifier. I'm currently trying to figure this part out, and I think I will dedicate my next blogpost to it.

That's it for now. This post is a bit rambling and I apologize for that. When I get the time I will go back and try to clean it up, maybe add some pictures to make it easier to read. I'm publishing this now to try to get into the groove of posting again. If you have any feedback or ideas on this, feel free to contact me.

Signing in to your mobile apps should be easier in my opinion than having to type a username and password. In my case it tends to be especially painful since I usually use KeePass to generate my passwords as random strings. Typing these long and random passwords on a touch device sucks. I'd like to see more alternatives when it comes to authenticating that are easier to use.

These are some ideas I'd like to see used more often.

The Sign-In Link

Slack uses a great method to authenticate mobile users. They get you to type your team name and your email address. After which you can use a "magic sign-in link" that signs you in automagically. The way it works is that they send you a link to your email, when pressed it signs you into your slack account. So no password required, instead proving that you have access to your email account is enough to log you in.

(I took the above image from https://auth0.com/blog/2015/12/04/how-to-implement-slack-like-login-on-ios-with-auth0/)

I really like this idea. I would love it if the HBO app had such a system, because it seems to have a knack for forgetting that I was logged in whenever I want to watch Game of Thrones. Especially for apps where users don't really interact with billing and paying for stuff, I can see that such a scheme would be nice to use.

Access to your email already meant access to most of your accounts. Only select few sites enable some sort of two factor authentication that don't just allow a potential hacker to reset your password via a "forgot password" function.

It also means one less password to remember. One less password that is potential reused, and then later hacked.

It does mean reliance on email infrastructure though. Which is already not so secure, depending on who your provider is. Spoofing and phishing is probably still going to be a problem for email a long time into the future. These could probably also be used against this type of scheme.

A protocol of such an authentication scheme might look as follows:

1. Mobile App Requests Sign In from server, providing email address.
2. Server sends an email with random token embedded in link, unique to email address.
3. User presses link in the email. Link opens server webpage which in the backend sends a token to the mobile app.
4. Mobile app receives fresh authentication token and uses it for future authentication.
5. Server keeps track of active tokens in case they need to be revoked.

Of course, this is very similar to other authentication schemes where you rely on a third party to verify your identity. Other popular choices are to use Facebook, Google or other website logins that you have already authenticated for the device. However Facebook and Google both have privacy concerns (as large companies selling your data for adverts) as to why you may not want to allow them to determine your authentication. They will likely track the application you used.

But other transports could be interesting too. For example, you could authenticate over SMS if it's a phone device. This would require a phone number instead of an email address. It would also incur a cost on your business to, since you have to send out the SMS.

Key-Exchange over Local Network

In general I'd like to see more apps that communicate locally. That is my computer to my phone, or vice versa. An app I'd like to code when I get the time is a basic notifier to forward alerts from my PC to my phone, for example. However, I'd like it if the applications did not always rely on a backend server on the Internet. When I'm in my home and I share a local network, it seems very unnecessary and sometimes unsecure to have to route my messages between my devices via a server on the internet.

Instead I would like to see a protocol which implements a Key-Exchange over the local network instead. This way you can set up a secret key for devices to communicate via, so long as you trust your local network.

I think one way for this would be to communicate via UDP. One client broadcasts on UDP for the other to search for it. Once they find each other, they perform a key-exchange like diffie-hellman, and finally ask the user to verify the connection by showing the fingerprint. This is probably very similar to how bluetooth is done, but over an IP network instead.

Once the secret keys have been set up, your devices can then safely communicate end-to-end encrypted over the Internet.

Verification via QR-code

Continuing on devices that communicate locally, another way of exchanging a set of keys could be to use QR codes. This could done by outright sharing a secret key by showing it as a QR code for the another device to scan via camera. Or in the case of authenticating to a third party service, it could contain a token or signature which the new device could use to forward proof that the user already has access.

Obvious advantage here is that the exchange can be done offline. It does however require a camera for at least one of the devices. For a key-exchange the key may also have to fit in a QR code.

For a token based system, where two devices connect to the same api. You could allow an authenticated device help a second device authenticate the following way using a QR code.

1. Authenticated device generates token. Sends token to backend server to save. Shows token in QR code.
2. Unauthenticated device scans token. Uses it to authenticate with backend server.

As part of my thesis, I'm looking at using Tor for an anonymous submission system. For this I set up a small hidden service to test, and I figured I'd write down how it's done. It's pretty easy.

Assuming you have some sort of TCP server you want to serve over Tor, proceed as follows. Install tor first. This is pretty much the only package you need. In Arch-Linux:

pacman -S tor

Then all you need to do is edit the torrc-file which is usually found at /etc/tor/torrc. Here all you need to do is add two lines, and the default config file describes it pretty well. In my case it looked as follows.

HiddenServiceDir /var/lib/tor/hidden_service/
HiddenServicePort 80 127.0.0.1:5000

Here the HiddenServiceDir is the location where tor will store the private key for the hidden service, as well as the hostname. Note that you can have several hidden services running on different addresses, just more HiddenServiceDir lines with different directories. HiddenServicePort will act as a port forward from the first specified port at the onion-address to the specified IP-address and port. In my case this forwarded traffic from my onion address at the normal http address to a local python webservice I was developing on.

Once you've added the lines to your configuration, you can then restart the tor service to start forwarding traffic.

systemctl restart tor.service

Finally you can get the hostname of your hidden service by opening the /var/lib/tor/hidden_service/hostname file

cat /var/lib/tor/hidden_service/hostname

Now use this onion address to connect to in e.g. your Tor-browser.