Terence Eden’s Blog

2024-12-14 12:34:25

Mastodon Now Sends Referer Headers! Hurrah!

shkspr.mobi/blog/2024/12/masto…

Back in 2022, I wrote this rather grumpy post on Mastodon, the federated social media platform.

@Edent@mastodon.social

Terence Eden

Mastodon enforces a "noreferrer" on all external links.

I have mixed feelings about that.

As a blogger, I want to see *where* visitors are coming from. I also like to see (and sometimes join in) with the conversations they're having.

But, I get that people want privacy and don't want to "leak" where they're visiting from.

Is it such a bad thing to tell a website "I was referred from this specific server"?

---

❤️ 61💬 16🔁 2907:09 - Fri 11 November 2022

When you click on this link - bbc.co.uk/news - your browser says "Hey! BBC! Please can I have your /news page? BTW, I was referred here by shkspr.mobi. THANKS!" This is called the "Referer" and, yes, it is mispelt.

One the one hand, sending the referer is good; it lets the linked-to server know who is linking to it. That allows them to see where traffic is coming from. On the other hand, this could be bad for much the same reason.

If you run a server anarcho_terrorists.biz, you probably don't want the FBI knowing that your members are sharing links to their pages. If you run a small personal server, you may not want anyone knowing that you personally linked to them. If you run a server for a marginalised community, you may not want a hate-site to know your members are linking to you.

But if you're a large-ish, general purpose, non-private site - like Mastodon.social - where's the harm in allowing referer headers?

Anyway, for historic reasons, Mastodon blocked the referer header. This, I believe, was sensible for smaller servers but a miss-step for larger servers. As I pointed out last week:

@Edent@mastodon.social

Terence Eden

Two years later.

Want to know one of the major reasons Mastodon didn't catch on with journalists and large website owners?

It is *invisible* in referrer statistics.

Here's my blog from the last month.

BlueSky now sends me more traffic than Bing.

How much traffic does Mastodon send? It is impossible to know due to the "noreferrer" header in all links.

(I'm not saying your privacy isn't important. But you can't grow a community if no-one knows you exist.)

---

❤️ 305💬 57🔁 24812:48 - Sat 07 December 2024

I'm not the only one to make this point - it has been a popular complaint for some time.

A few days ago, Mastodon changed to allow this to be configurable.

This is excellent news. Website owners will be able to (somewhat) accurately see how much traffic Mastodon sends them. That way they can determine if there is a suitably large audience to engage with on the Fediverse.

It is, of course, slightly more complicated than that!

• Instance owners can opt-in to allowing Referer headers (it is off by default).
• The policy means that only the domain name is sent; not the full page.
• Mastodon is federated and there are thousands of sites. Even if they all opted-in, their statistics will be fragmented.
• Apps can set their own Referer header - leading to more fragmentation.
• Even if they do opt-in, users can set their browsers not to send Referer headers.

Nevertheless, I'm delighted with this change. Hopefully it will allow the Fediverse to grow and attract more users.

#fediverse #http #mastodon

Change referrer policy to be controlled by header in web UI by Gargron · Pull Request #33214 · mastodon/mastodon

A couple of changes here, ultimately with the goal of making it easier to control the referrer policy by setting ALLOW_REFERRER_ORIGIN to true in the environment. The abundance of rel="norefer...

^GitHub