--- urn: "urn:penemure:d4d0c7a3-a002-4680-952f-691d52933683" backend: "pub" ancestors: - 📁 Penemure Development Blog created: 2025-01-22 14:22:02.395955+00:00 updated: 2025-01-22 14:22:02.395961+00:00 tags: --- # 📓 Post-post-experiment thoughts # Multi Repo Support is multi-repo support a must? do i want to store stuff in the different repos? that really assumes someone else will want to look at my project management or interact with it. is this just for me? truly? no collaborative uses? I fear that i still want that option. If there was something like a web ui, someone like sas might make use of it too. Do we allow dangling refs? I guess we do. It would be better UX to hint to where we could get that from, so, do we store a repo url with every ref? (Yes) will the design really be: - "primary backend" (for non-git-repo attached docs) - N other git repos which have their own docs (and anything under that tree is stored in that repo, unless noted otherwise(?)) but of course you should be able to run it and point at one of the other repos without a parent, you might just lose things like styling of the home page but that's fine, should still be able to see the whole tree. ## OverlayFS - should be aware of where the file comes from (which repo is real source), and any masked files (not sure if we need to show those?) - syncing will be, hmm. a thing. every time we start we're going to want to sync at _minimum_ the primary repo, maybe the others as well. Just to make sure we're in sync with the latest bugs for that repo. And after (every?) save we're going to want to queue a background sync to ensure that state is persisted. Some of this complexity gets less bad if we're editing via a gui rather than locally, but, i really do want it to work completely offline. on my laptop i'd be using the local version. # Multi User / Hosting @74f09fe0-0bb8-44f4-90e5-dbe55aa54734 something like this brings the question, do i want a multi-user system really. The likelihood anyone uses my thing is approaching zero. But it would be a hard decision to unfuck later if I changed my mind. we can at least annotate the 'owner' of a document. And if we're storing it in git, we'll have history at least in case we decide to recover that later. If we move away from go we don't get the tsnet lib, but, we could throw a tsnet proxy in place if we really needed, so that shouldn't influence our decisions. But essentially this _must_ be hosted because the options are: - a webserver - a local application, that needs to be able to talk to N git repos. and that would require writing an android application and I'm not about that lifestyle. Or shipping it as JS and I'm also not about that lifestyle! So if we assume that multiple users will access a single server, then they'd want to have their own preferred 'main' repo for themselves. Maybe it's sensible to mandate a specific account (user or group) is designated as the owner for this server that's running. On my laptop it might be account/hexylena (or accounts/gh/github.com/hexylena) and on the family server it might be accounts/galaxians (a group). everyone logging in would get the relevant home page for that group with whatever dashboards/etc. this would work via shadowing, the 'home' would be loaded last and shadow any other home pages loaded via individual user accounts that are connected. when unauthenticated access happens, we'll just show whatever is marked as 'public' maybe? do we need a classification system, public, protected, secret? Where e.g. titles of the 'protected' documents would be shown? That seems a bit, extra. # Activity/history is activity/history a must? (primarily in multi-user, but ok, i would like to know what i worked on recently i guess.) "list of recently edited" is super easy, e.g. ``` SELECT title, type, created FROM tasks order by created desc limit 4 ``` but this doesn't say who did that. Does it matter? It'll be stored in git, in theory, so, anyone who truly needs to attribute a change to an individual could, possibly. # git bug i've accepted i'm competing with them (in)directly. gross. it'll be an 'application' within the document store, the github information, storing the state/metadata about each issue that can be pulled. maybe there's a future where we can run 'hxpm sync git-bug' and it'll sync with the locally stored git-bug discussion information. # serialisation I don't want to have to revisit the data model later, that feels important, that we pick a sensible model. for all intents and purposes it is a document database, where each document is a collection of key/values. I think we need separate areas of the database, per 'application' a bit - project/{uuid} - account/hexylena - repo/{url} - stores information about this repo, could be useful for 'ref' links. url should be something human readable(ish), e.g. github.com/hexylena/project-manager and a new application could create/use a new area if needed. ## Namespacing This presumes the ability to have a unified namespace, really. _Especially_ for accounts. We wouldn't want to have the same account info stored multiple places right? especially if we want to attach information to it (pfp, contact info). But we also wouldn't want the situation where Jane can see A (personal), B, and C and Alice sees C and D (personal) but can't figure out who someone is. Wouldn't want to disclose something from a personal account that shouldn't be on a work account. but maybe that's dealt with by letting folks have multiple identities, better for plural folks anyway. But it remains a problem, where is the canonical storage location for account information? I might want to assign something to myrthe even though she isn't using this system and see issues assigned to her. maybe even a pfp if i was new to that job and struggling to put a face to a name. attaching other links like email/github account/etc. we could namespace them somehow? and maybe store a 'sameAs' or something to link them together if that was useful? e.g. 'github.com/hexylena' Doing an AT could search client side for them by name/id. Maybe that's reasonable. Information I know about e.g. Alice should remain private by default in my own repo but I can still AT them and have a reference that's useful for other people. And if we see a bunch of references to github.com/does-not-exist even if we don't have 'local' account information for it, we could have a plugin that pulls some useful metadata from github into the account info and then could show something reasonably useful. ## Potential Applications - page/home - a special page maybe, for deciding the home page layout? that would enable an /about as well or similar. maybe someone wants /terms or /stats. (or /metrics lol) If we're doing a layered FS, we'll want to take our personal layer last to shadow anything else. (or maybe we need to be shadow-aware?It's better ux for sure.) - template/base - now we're cookin with gas (only present if they want to override the default template maybe, otherwise default to built in one?) (Implied a bit by /metrics) so we're seeing 'app' namespaced things pop up in both accounts/ and repo/ is that reasonable or should it be flipped? no, i think this makes sense. - file/meta/ - file/blob/ k, yup. that'll give us somewhere to store those. ## Production Apps that brings us to - projects (the default notes space) - templates (yes an 'app' with its own namespace) - files (parent implementation supports referencing and then plugins: 'default', and 's3' maybe, to do actual storage/retrieval?) - accounts (github as plugin, would this be where s3 creds are stored) - repos (github as plugin) - emoji/shortcodes? (this could be a plugin that depends on files? i just want :akko_fingerguns: to render) - pages (for /index and /about) - secrets (or would s3/gh api creds go here, if we wanted to do something like that?) we can ask the files plugin for how to embed a referenced file, and it can generate some sensible html for us. We can store meta about the file (original name, caption/description/alt text in the json 'note' for it) and the blob separately either locally or in a remote store of some sort (http, s3) apps can declare 'common' metadata that the user should provide for this type of entity. E.g. the image metadata could declare that it suggests providing a license/citation/source url for uploaded files where known. ## Accounts so for the case of accounts where we want to say 'so and so edited this' then we should essentially say "it must be tied to a publicly resolvable identity" right? so that if someone else encounters this (e.g. stored in a public repo) that they can at least know who did what. Rather than tying to a private identity? should there be a special account/self with sameAs pointing to public identities, and we'll just document every edit as being done by the preferred public identity? # Editing ## blocks are they really needed? why did i want them? because notion did? I guess i wanted them for 2col support but, that isn't necessary. we could tolerate HTML support instead. if anything gets exported to another format (TeX) it'll be done via pandoc anyway. it would permit transcluding subsections of another document. how likely am i to want that feature? we could retain the option to have blocks by just forcibly storing everything in a single 'markdown' type block. It'd make some of the 'fancy' styling easier, but, that can also be done client side. that's fine. details/summary, it's ok, it can be html in md. I could see wanting to copy e.g. the description of N grants into a main document which reminds me of them, but, then maybe that shouldn't go in the blocks but in the metadata as a 'description' field? In which case we go back to not needing transclusion. ## DO NOT WANT - to reimplement pandoc (custom templates would be ok.) - inherently tying myself to some other javascript project. ## Rendering / Editing We've got all of these fancy apps but how do we ensure they're rendered correctly? Does :akko_fingerguns: when input in markdown get turned into an AT reference to the emoji app? An image? I guess it's the only thing that makes sense but then we're also further away from markdown and have to do a _lot_ of custom processing on an incoming markdown file. And when the user goes to edit that markdown, they might want to edit the file while they're there, so should we de-serialise somehow? This all points back to blocks or a very fancy editor. If it was an 'image' block we could be sure that when it's presented to the user to edit, that we've got alt text / url available for editing. it's one thing to turn an image markdown ref into a local file and replace with AT in the backend. It's another when the user wants to edit this file again. ``` :akko_fingerguns:sometext ``` would also be a problem to render, because: - it's in a pre so we shouldn't touch it based on that (oh fuck we need to understand markdown) - it's got no space after it so if we're doing AT UUID then we don't know when to stop reading (well, one can assume, but, ew.) So maybe we need a 'better' format like LT uuid GT where we can be sure we aren't adding/removing whitespace. is anyone gonna stick those in pres? can we just ignore that aspect? I don't want to have to understand markdown, just wanna find and replace since that keeps from having to hook into every level of a markdown ast. some of it is down to editing experience, users (me) want that nice editing experience. And potentially me holding onto the weird desire for a vim-compatible editing experience, even though i'd probably use the web version the vast majority of the time. # Data Model ## special keys? **Tabs.** I think I want a tabs key for a page that lets you add tabs for other pages to the current one. ``` _tags: - title: Tabs type: tabs value: [deadbeef, cafebabe, etc] ``` # Conclusions - layering multiple via something resembling an overlayfs that's aware of the source of the layer / can list masked files, and potentially their union(!?). - a primary repo for storing private metadata (preferred home page, private details about accounts/friends) - apps have their own private namespace, that no other app should talk to. I guess then it needs to be declared in advance. - apps have plugins for various backends (e.g. local/s3 for file storage or github/gitlab for repos and accounts) - we store markdown in one primary block for future proofing where we might want to split those blocks or attribute them to a specific author. - editing is still a bit of an open question but i think it's safe to assume website, not vim. realistically i won't be sitting at my terminal like 'yea lemme add a note'. and if i wanted to, that could go through the API just as easily.