Since my last article about the EU's new copyright restrictions, I started thinking of other ways around these laws. As one would expect, there is a clause in these laws that exempt "small platforms". So I thought what if, instead of people communicating through large centralised platforms like Facebook and Twitter, what if everyone hosted their own "platform" but there was some means to connect these individual platforms together via an open protocol. This way, each individual "platform" would consist of a small number of people, as small as 1 person on platform, but because all of the platforms are interconnected, information can spread just as it does on a big centralised platform.

Turns out that there are several projects based on several different protocols that already do this. Collectively these are known as the Fediverse. There are a lot of different ones that all try in their own way to clone various commercial social media platforms. Frendica for example is very much an attempt at cloning facebook. PeerTube is a very interesting one that is trying to be a distributed YouYube clone. Personally what I was looking for was something similar to twitter and the closest thing I could find was Mastodon.

Installing Mastodon

If you're an experienced sysadmin, or have no interest in sysadmin activities at all you might want to skip this bit.

At this point the obvious question is "why not just run the docker container?", well for several reasons

The instructions I followed can be found here. However I suspected there was going to be trouble from the bit where it said "Setting up Mastodon on Debian Jessie is generally a copy/paste exercise" and "It is strongly advised to proceed with the installation on a freshly installed Debian to avoid interfering with production environment.". The latter meant that this was almost certainly not going to be a copy and paste job. That said, apart from having to change the "-s" to "-shell" in the adduser command when adding the Mastodon user everything was fine until I hit the part where I had to install bundler. I got the error "rbenv: version `2.5.1' is not installed (set by /home/mastodon/live/.ruby-version)", but this was fixed by running "rbenv install 2.5.1"

There were more problems to follow when it came to the next line, running the "bundle install" step. It started with the error "icu required (brew install icu4c or apt-get install libicu-dev)", which itself was solved by simply running "apt-get install libicu-dev", but a new error came up. This is something I find quite often in instructions posted online: whoever wrote the instructions had some dependencies already installed, so didn't know to add them to the instructions. It's an easy enough mistake to make. When you are on the receiving end of this kind of error the only thing there is to do is to try and work out what the missing dependency is, and add it in. I went through the process of adding a new package, trying again, and getting a new error a few times until was able to complete this step. By the end of this I had installed the following packages: libicu-dev, libprotobuf-c-dev, pkg-config, libprotobuf-dev, libidn11-dev, and libpq-dev.

The next issue was with creating the secrets, but the error that was given was very helpful and told me that I needed to specify an environment. A quick look at the script revealed that all I needed to do was add "RAILS_ENV=production" to the front of the command. I then blindly plowed ahead and tried to generate the database which failed because the Mastodon user had not been created, so I just did that myself with the regular SQL commands. More troubling was the next error "PG::InvalidParameterValue: ERROR: new encoding (UTF8) is incompatible with the encoding of the template database (SQL_ASCII)". This basically meant that the character set for the database was wrong. Luckily this seems to be a common error and a bit of googling tuned up this solution. There were a couple of other ways to have fixed this problem, and if I had not found that I would probably have just grabbed the table schemas out of postgres, dropped the database and rebuilt it by hand with the correct character encoding. Still, this feels like a bit of sloppiness on the part of whoever wrote this part of the installation script: If it is important the character encoding should have been explicitly set rather than relying on the system default to be correct (I'm assuming that is why the scripts worked for whoever wrote them).

Next up I added the systemd scripts and tried to bring everything up only to be hit with a bunch of errors. The issue turned out be due to the fact that I had the wrong version of nodejs installed. However, this took me an embarrassingly long time to figure out because the version of nodeJS that was coming up when I tried to run it in my terminal was different to the one that was getting picked up by the systemd scripts. In the end it was a simple fix to get working (just change the systemd scripts to use the correct version). The important lesson to learn from this bit is that you can't always expect the behaviour you are seeing in your terminal to reflect the environment the software on your system is running in.

Next up was a bunch of webserver errors. Even though everything runs over https, not regular http the instructions had failed to mention that you needed to generate certificates and where to put them. So, I installed certbot using the instructions here and ran "sudo certbot --nginx -d" to generate some certificates. I looked at the nginx config files to figure out where they needed to go and tried again. I then hit an error where I was getting the message "Security verification failed. Are you blocking cookies?". After a bit of googling (again) I found that the answer was that there were some headers missing from the nginx config. This post details the fix.

At this point I had what appeared to be a working Mastodon instance. So, I went to the front page and created myself a user account. Because I had been lazy and not set up the email system I didn't get a confirmation email, so I was not able to confirm my account the tradition way, but a quick trip into the database lead me to the users tables, and the column "confirmed_at", which I set to a couple of seconds after the "created_at" time. From here I was able to log in and make a quick post to make sure everything was working.

There were, however, still a few issues to fix. Initially remote follows didn't work at all (i.e. following people on other Mastodon instances). This was all down to the setup on my end. My first attempt at fixing this was to set the STREAMING_URL (the endpoint where other Mastodon instances communicate with mine). This failed to fix the problem, but opening chrome developer tools while loading my account revealed the problem: I had set the URL to be a https URL and not a wss (secure websockets) url. After changing this I was able to remote follow people and I was away!.

Wow, that turned out longer than I expected. I think the lesson I want people to take away from this section of the article is that if you are trying to get a big complicated piece of software like Mastodon up and running, don't give up at the first sign of trouble. A lot of the time a simple Google will fix the problem you are facing. But if not, delve into the individual parts of the system. If you are having trouble building the database or nodeJS the solution might lie in an investigation of those individual components rather than the overarching piece of software. If you are having trouble loading the software in a web browser the information on the problem might not be in the logs but in the developer tools in the web browser. The information you need to solve a problem might not be in the place you first expect it to be.

Looking back at what I have just written, I can forgive my readers at this point for wanting to run away screaming and just go for the docker version. All you have to learn is one piece of software (the container manager) and in theory everything else should have been done for you. But I would caution anyone who is uncomfortable with what I have written above against taking that path. I'm mainly a programmer, not a sysadmin, but I have had to take on some of those types of responsibilities in the past and my gut feeling is that an inexperienced sysadmin just jumping in and using docker is a bit like someone who has just got their driving license hoping into the lastest McLaren and trying to set top speeds around silverstone. It seems like it's just going to end in tears.

Why not just join an already existing instance?

One of the problems with joining a specific instance on Mastodon is that you will be limited to which other Mastodon instances you can interact with based on the blocking policies of whoever is running the instance. On the one hand this is great for communities to self organise and set their own rules, but not so great if you want to be in total control of what you can access. The problem with just finding an instance that has a complete open access policy is that those tend to be full of shitposters and trolls, and hence get blocked by other instances. So, since I wanted access to as much as possible I decided to set up my own instance on my VPS. It's the only way I could have total control over who I am able to follow without hitting the policies of the instance I joined.

So, what's it like in there?

I wasn't on twitter in the early days, but it feels like what I imagine twitter felt like before all of the famous people got there. Lots of people broadcasting their opinions to the world. Unlike twitter, there does not seem to be as many conversations going on. Very few posts I see seem to have replies. Not many posts generate conversations. This might be a glitch in the system, or a function of the fact that I can't see a lot of the conversations because I am on my own instance. Or it could simply be because, due to the fact that there are less famous people, the people I do follow have less followers themselves and therefore don't have as many people to comment on their posts.

One of the problems with running your own instance is finding people to follow. There aren't many people putting links to their Mastodon profile on their business cards (although I suspect in certain circles it'll be common to find them embedded in QR Codes on people's laptops). For the most part I started out looking for people saying interesting things in the toot (toots are Mastodon's equivalent of tweets on twitter) stream and following them as well as a few admins of instances that seemed to be dedicated to topics I was interested in. From there I was able to see what those people were "boosting" ("retweeting" for those from twitterland).

Something that holds up very well is the promise of social networking between different system. Out of the people I have followed so far there are a couple on GNU Social (including Richard Stallman, unsurprisingly the only social network he is on) and a couple on another system called Pleroma. From what I can gather GNU Social is a bit more limited in its functionality. For example, if I reply to a post from GNU social it'll appear in my federated timeline, but not with the original post. Pleroma seems to be a lot closer to Mastodon in functionality, and I might end up switching to that in the future if there is some way to export the data out of Mastodon.

There are a couple of, possibly extinct, elephants in the room that need to be dealt with at this point. The first is the political angle: everyone in the fediverse I have encountered so far is quite left wing. It seems that the people who have really taken to Mastodon are the people who were getting really harassed on twitter by right wing nuts, so have built themselves their own safe spaces online. This to me seems like the correct response to that issue: public spaces (which twitter kinda is) need to strike a balance between allowing people the most freedom possible while maintaining some kind of rules of conduct such that everyone can generally get along. If you fall foul of this, on either end of the spectrum, what you probably need is a more private and exclusive space to express yourself in. So, people who are on the more sensitive side (most of which seem to be left wing today, but I suspect if this were happening 50 years ago it would be conservatives) should seek out and join instances that have strict policies on blocking other instances that allow things they don't like. On the other end, if someone wants to be a total shitbag they should go and join an instance that allows them to act like that.

But that's it for now. If you want to follow me on Mastodon I'm here https://empoknor.speedfox.co.uk/@fox. See you all over in the fediverse.

Links