set X-Robots-Tag to disable indexing API endpoints

This commit is contained in:
Hazelnoot 2025-06-13 23:18:06 -04:00
parent 99bf315351
commit b44abe0eaa
5 changed files with 23 additions and 0 deletions

View file

@ -791,6 +791,10 @@ export class ActivityPubServerService {
reply.header('Access-Control-Allow-Origin', '*');
reply.header('Access-Control-Expose-Headers', 'Vary');
// Tell crawlers not to index AP endpoints.
// https://developers.google.com/search/docs/crawling-indexing/block-indexing
reply.header('X-Robots-Tag', 'noindex');
/* tell any caching proxy that they should not cache these
responses: we wouldn't want the proxy to return a 403 to
someone presenting a valid signature, or return a cached