everyone make sure you're not blocking search.marginalia.nu in your robots.txt file! (also if you're one of those weirdos who threatens viktor with physical violence if he doesnt remove your pages, get rekt and gtfo my profile)
6 likes
Delete
Are you sure you want to remove this comment from the news feed?
All you do is put a file in your root directory (where your homepage is) called "robots.txt". As for what to put in it, you can have a look at mine. badgraph1csghost.neocities.org/robots.txt Heck you can just copy the whole thing, it's pretty well curated!
you can keep it out of folders if you need to. just use Disallow: /foldername/
How do you go about formatting and adding a robots.txt file? Always wanted to include one but never really understood how.
All you do is put a file in your root directory (where your homepage is) called "robots.txt". As for what to put in it, you can have a look at mine. badgraph1csghost.neocities.org/robots.txt Heck you can just copy the whole thing, it's pretty well curated!
obvs /spidervault/ only applies to me. that's the folder i dont want marginalia looking in.