The most broken part of the WorldWidWeb is the interlinking of content creating brittle, unmanageable dependencies. This was a fucking stupid idea, but it was way before anyone even dreamed up search engines.
A much better idea is to build into the entire thing a semantic structure so that rather than linking to specific content, you link to a search for the relevant keywords about that content on a specific site. This is roughly equivalent to linking to a search that runs and produces results instead of linking directly to any content at all. This way, if new content on that topic has been generated by the source of the content that will show up as well. This also prevents the insanity of adding Updated blocks to the beginning of old content just because you cannot move it or lose all your SEO money. It’s fucking idiotic!
The Web got so many things so fucking wrong. I want to correct a bunch of this with KEG. The linking problem is avoided all together because only linking to other locations within a local document or another relative location in the same KEG site is allowed. There won’t be anything stopping people from providing Web URLs in their KEG content, but KEG itself will not depend on it at all. Instead, like StackExchange has learned, people will be encouraged to cite or copy from a source and include it in their own content rather than linking to it and having it disappear. KEG draws heavily on lessons learned from the software industry and that problem is why Go encourages “vendoring” of source so that the imported dependency can disappear without affecting the code. It just makes fucking sense (but the Web designers totally didn’t think about that, they were physicists, after all, not software engineers).
Again, there is no need to ever link to anything because everything you ever want to look for is in your KEG Cache. That’s right your own personal, full copy of everything on KEG that you ever want or need to query, there for you locally available to search any time you want, any way you want. This is also why only the most permissive (attribution) licenses will be allowed for content on KEG. And attribution will be fucking implied by the ability to GPG sign every fucking file that gets cached. The Web has nothing even similar to this at all.
Let’s role play a user story. You start your own KEG Master Node (site). You learn from a friend or on the InterWebz of a cool person/group/organization with a KEG site of their own. So you add the authoritative URL to your list of KEG sites you follow. You decide what level of update is wanted, how frequently you want to check for updates and how much of the site you want to cache locally. You can cache just the KEG directory containing indexes and meta data only for high level searching, or you can cache the entire thing at whatever interval you want.
Here’s the magic: your list of KEG sites is saved in your own KEG site meaning you automatically share your list with all of your followers and so on. You can also blacklist a site with a reason so that all your followers can see why you blacklisted the site. This exchange of recommendations is fundamental to the entire idea of trusted knowledge exchange. It completely shuts down normal Web fuckery.
Your KEG service that runs locally is constantly rendering, indexing, and syncing your knowledge as you add it so there is no need for a monstrous, centralized, unethical search engine to do the work. The hard part of the search engine work, the indexing, is done automatically in a distributed way, on every machine hosting an authoritative KEG site. And because the result is a bunch of pre-indexed static artifacts the search engine becomes trivial, even unneeded, and if you do need to do additional search indexing of a KEG site that you follow, you can set the parameters of that indexing in your own KEG site service where the agent will automatically, dynamically index all new content that arrives or is synchronized.
In terms of scale, let’s say you have 1000 people that you really want to hear from, honestly, most humans won’t even have that many. The beauty is that every individual participant is motivated to do their own content curation at any given moment. Say your interests change, you can unfollow certain sites and start following others. If you really live a double life you can setup another KEG site for a different persona.
But let’s do some math. I have 1400 zettels now. Assuming we have long ones, 100x72 that’s 7200 bytes per zettel, about 7KiB considering 100 lines is a very long zettel. That’s 9500 KiB, about 10MiB. Let’s say you have an usually large number of figures in your KEG (which must be raster images if they are in zettels). A regular image is about 170KiB, so you have 100, add another 17 MiB. We are at about 30MiB for all the content I have generated for the last two years. Longest lifespan from someone who knows how to type until death we’ll say is 90 years, rounded to 100, divided by two, 50, times 30MiB, 1.5GiB for everything I’ve ever captured in my KEG. Considering a 32GiB thumb drive is $16 dollars these days, you could store everything from all the minds of people you care about for 30 people on one thumb drive.
All of this is the extreme case. Most people won’t want to follow everything another person has to say, they will be caching their metadata and searching it and only caching the individual knowledge nodes that they care about. This is how all the content from 1000s of sources is something you could reasonably carry around with you on your phone. People just aren’t thinking in terms of local cache and search, and they need to start. Ironically, I get people already doing this. My zet has already been forked a dozen or so times. People get afraid that they are going to lose the content created personally from others who make that available. It’s almost a collector mentality that gives a sense of ownership to the person doing the curating of great people and content.
By the way, KEG is mostly about individuals. It doesn’t work as well if you have a “team” or an “organization” writing everything, even though that is totally allowed.
Perhaps my favorite part of KEG is how it intrinsically motivates people to produce good content instead of tearing down others content. Sure you can rant about something or someone you dislike, but you have to do in in your forum, on your KEG site, not in some other moderated, bullshit forum. This is the true democratization of communications. You can talk shit, but you have to own that shit and risk being blacklisted for your behavior. You also cannot hijack the comment section of someone else’s blog or forum for your own marketing. And you cannot launch a fucking spam bot advertising campaign. Maybe you don’t care about being blacklisted, but there are no fucking moderators because the readers and producers are the moderators. This completely fixes the single biggest problem with Internet communication today.
I ran into this exact problem even on GitHub when that Twitch prick decided to attack me and everyone started creating PR requests and issues on my fucking zet repo that should have gotten them permanently banned from GitHub. Had I been doing it on a KEG site, they would have been absolutely powerless in their attempts to harass me. KEG fixes all. I just have to fucking make it. Luckily, it is more of a convention and API than any specific technology. I just have to have that standard really well documented so we can all start experimenting with making applications based on it, kinda like how Kubernetes started out as an API.