A soft launch of wisp.place

You can do anything with the PDS. It's just a NoSQL DB. Some may be recoiling reading this, but its true. Anyway, I've been wanting an easier way to host simple static sites for my projects, and also wanting to use my own PDS for something useful and willing to share publicly. So, came wisp.place, an indexer to host static sites onto your PDS. Kinda

The content, when you go to a site like https://nekomimi.wisp.place/ , it isn't served directly off the PDS. It's being served off my servers*. There's two services to it. The main backend that makes requests to the PDS for the user as well as manage domain routing (I give out one free wisp.place subdomain per DID as well as Bring Your Own Domain) and a microservice for consuming the firehose and serving on disk caches of content. (If the cache is stale or for some reason there isn't a cache on hand, it will grab a fresh copy off the PDS)

Two reasons for the latter microservice existing. One is that the PDS does mime type sniffing, and it fucking sucks at doing it. https://github.com/bluesky-social/atproto/issues/3151 If you upload say a HTML file that contains a SVG to the PDS as a blob, the PDS will mime type sniff it as a image, break, then abort the request entirely. Not great. Upload a JS source map? it thinks its an image and breaks. I don't think the PDS should be doing mime type sniffing to begin with, but I don't know why this hasn't been fixed at the very least.

The second reason is that, no offense, some of y'all have shit networks connected to your PDSes. I wondered why the Atproto dev community always made a big deal of having appviews, this is why. A simple listRecords of 10 on some of y'all's PDSes can take upwards of 30 seconds. So I download a cache and serve it off my servers*.

Why is servers asterisked? When I'm confident in actually launching it, I'll move wisp.place to my VPSes I control and maybe even use an actual CDN. Right now it's on my m4 Mac Mini connected to my residential home uplink that only has a upload speed of 40 mbit/s (small b.)

The lexicon right now is actually just one giant manifest of the virtual file system and where each blob should go. It's a bit silly, but in my performance testing its actually fine. https://pdsls.dev/at://did:plc:ttdrpj45ibqunmfhdsb4zdwq/place.wisp.fs/cobblemon Here's one with over a thousand files, and it syncs fine with my microservice.

Also, path rewriting. You might be asking, how do I handle someone doing on their html if the site is served on https://sites.wisp.place/did:plc:ttdrpj45ibqunmfhdsb4zdwq/wispPlaceDocs ? I rewrite absolute paths on HTML to then be /{did}/{rkey}. Not on the JS or CSS. I have a JS backend to worry about, and I don't see CSS importing CSS ever. I can revisit this.

Things will break. Real Launch Soon*. I want to fix the frontend more. Backend needs more stabilizing. And I do want to draw anime fairy girl branding for the site.

There's even more WIP stuff I have cooked up. One is a cli to bypass my backend and upload directly to the PDS.

The other is a tangled spindle that uses this CLI to push files from your repo to the PDS for serving,

The processes here need cleaning up though and a little bit more time in the oven.