I think it is time to stop calling this stuff "biweekly update", as I seem to be unable to publish every two weeks.

How notion cost me a week of life

I was running a bit late with my bi-weekly update, but I finally managed to put together everything I wanted. Then I exported the blog from the notion and run a conversion script, which translates the notion structure into the simple HTML and adds a bunch of other functionality, that is missing in the export.

Except, that this time, it failed;

I've poked the code with a debugger for a while and then I've realized that it is not a bug in my code, that indeed, notion changed the structure of HTML exports. Apparently there are now hashes or UUIDs or whatever they are, in filenames. So for example, English section.html is now English section 8f6665fa0621410daa32502748e3cc5d.html.

"Easy enough to fix this", I thought. Just run a simple regexp and replace them. Of course, it's not so simple, because you need to replace also all links and meta tags and such. But my whole blog generator is a list of progressive transformers made in this manner, so this was easy enough to code.

It was 3:30 in the morning when I run the export, hash got removed from the filenames, and I've realized, that no, this not that easy. Notion also shortened the names and now when you replace the hash, you get collisions and some content gets overwritten. And I can't just let the hash in the export, because then I would break links pointing to my blog.

Thus began the week-long journey to completely rewrite my blog generator, to prevent this from happening ever again.

Old system worked like a set of postprocessors on a html pages on disc. There was a lot of path handling and so on. In the new system, I've decided to try new approach, similar to how compilers work; first create an abstract tree representation in the memory and then apply set of transformation on the tree until it has a form you want.

I've implemented kind of a virtual filesystem and a system of references instead of paths in HTML links and then I've rewritten everything to work with this new representation. Structure is now generated from how pages link to each other and also their names are taken from the headers of each page, and the code can handle duplicates.

I've even thrown in new Atom feed generator from the changelog, as the old one was a bit hacky, and I've also implemented a diff script, that can diff between old structure and new one, and output a list of nginx redirects, so old links won't get broken.

It was 3:30 in the morning, when I've finally merged the 76 commits:

And all of this because someone thought "wouldn't it be great if we put a UUID into the filenames?"

tinySelf progress

I've had this conversation with my friend on the topic of systems versus languages, and what an operating system is, and also about Steven Wolfram and his information system, that he built around himself.

Then I've watched following video and felt a need to work on something as cool.

So, after a long period of inactivity, I am slowly trying to return to 📂tinySelf development.

At the moment, language mostly works, but interpreter needs some upgrades. There are details, that are not really done. For example, you can have only specific amount of slots in object, because bytecode encodes the literals to one byte, so I need to add multi byte encoding and so on.

It would also be nice to increase performance. One big bottleneck is that each object (ehm, it's map actually) has its own {string: object} dictionary for slots, which is slow to create. Although it has fast lookups, you usually need to do lookup into all parent slots recursively through whole parent chain. This can be a lot of lookups, so there is a layer of caching on top of this, which creates additional dictionary, which is also slow. I think that I've maybe overthought the stuff and that there may be some simpler method to use, like one global dictionary for everything, where you encode information to the keys, or something like that.

I am also thinking about the alternative possibilities for implementation. I want to look into implementation of the Spry language and see how lookups work there. There is also RSqueak and RPySom, which were a source of inspiration in the past, but I've never really studied the implementation of the VM in the detail, only specific stuff, like for example bytecode crunching.

It could pay itself to actually, you know, get some education in the field of writing interpreters, before I go and do that. I think that I should read Crafting interpreters and maybe also Writing An Interpreter In Go before I return to the development and began reinventing the wheel again. I've loaded them into the iPad / ebook reader and I'll see what I'll learn.

I can't help but to wonder whether the Rpython is really the best language / toolkit for implementation of the interpreter, because there is quite a lot of undocumented dark corners and even some stuff that is documented doesn't really make much sense to me.

For example, it is possible to run the JIT translation and interpretation with pygame visualisation, but I have literally no idea what I am looking at and you can't really get a documentation for the whole process. Rpython also makes me constantly wonder what is actually doing at the lowlevel. Like for example, when I simply define and use an object, what it will actually translate into. Sometimes, I just need a simple structure, and there are ways how to do that, but the documentation is confusing and I have no idea what really happens.

I think that it may actually make sense to try to rewrite the whole thing into Rust, as I try to stay away from the C/C++. There may be sane ways how to add support for JIT. And then there is also GraalVM, which means Java. I've gone to the GraalVM lectures last year and I really liked the experience, so I should probably try it. But I didn't really touch Java in like ten years, so I need to do some tasks to reacquaint (eh, what a word) myself with it again. Sigh.

Object wiki update

Last time, I was talking about another developer, and work on transactions (see Transaction support for simple in-memory database for details). It took me almost 100 commits, but transactions are finally working, and I've also finished and tested the basic client-server architecture.

The system I've created is this monstrosity, which tracks transactions on the server, and sends lazy-loaded objects to the client. Client gets transaction with NodeTree root object, which lazily loads part of the tree of Nodes and Inputs each time he accesses some property or calls some method on any of the objects he received. And it also tracks access to it and sends updates to the server, so it can use it to compute transaction conflicts.

Everything seems to work, and I've also added a lot of basic unittests, so it shouldn't be fragile. Nevertheless, I still feel like I should add much more tests and also do some kind of performance testing, because I've done things with python you normally don't do. Reflection, hooking on all kinds of lookups and so on. For example, I serialize exceptions at the server and send them pickled to client.

Life & philosophy; trap of overspecialization

I may have fallen into the trap of the overspecialization. I mean, I make decent money for the part of the world where I live in. Problem is, that I've kinda reached the top of what I can make, which is alright, but it limits me to try new stuff.

At the moment and at least for some time, there are several people depending on the money I make. This forces me to get a lot of money, which limits what I can do.

For example, I've watched a lot of Warhorse videos about game making, and I am really interested in the system they've made, because it is relevant to my quest for structured systems (see Programmer's critique of missing structure of operating systems for details). In the past, I would simply quit my job and go work there, to get new experience and see something new. This is now a luxury I can no longer afford, because I know that as a newcomer to the field, I wouldn't even get a half of what I am now making. Maybe a third. So.. that's no longer an option.

Same situation happened with the VRginners / XTAL. They are a company making this futuristic HMD headset:

And they have dev office nearby. I've sen't them my CV and asked whether they could find some use for me. They've replied that they could, provided that I could work in C/C++ and take a low junior pay. In the past, I would love to accept the position. Now, I simply can't.

So, yeah. It's not that great problem and I certainly can't complain, because I am still incredibly lucky and do well. It just sucks that possible branches of my future are now closed. I am no longer free to work in interesting positions, because I have to trade them for more money.

I suspect that this will force me to make my own company, or maybe go work to different country. Context: we are still relatively poor country compared to the neighbors in the EU and I know I could make more there.

Book report

I kinda feel like Asher got himself into tight corner. So far, each of his books pushed the described technology a little bit further, but I can't seriously imagine where he could push after this.

⚠️

I've had so many thoughts about this, that I've had to put them in separate blog.

When I was reading the epic descriptions of the battle fought on all levels and scales, from macroscopic, with kinetic weapons and lasers, to underspace technology, information warfare and nano and pico tech at the same moment, I couldn't help but to keep playing psydnb in my head:

---

The Bosh was also great. It's a short novella from the future of the Polity, I would guess that several thousand years from Asher's other books.

---

I must say, that I've learned a lot about particle physics from this book. And it has this really optimistic feel about it, which sometimes let my skin crawl, like for example when he talks about all possible usages for the atomic technology. For example radioactive floaters in liquid containers, which you can use to measure level of liquid inside by putting Geiger counter on the top. That's certainly .. creative.

---

Blender books updated for blender 2.80 arrived. So far, I am reading through Blender for dummies.

Blender is one of those things, that more I know about, the more I am impressed. I've bought the books to get an idea what actually is there in Blender. I've watched a lot of youtube videos with tutorials, and although they are great for learning some specific task, they can't replace a good book.

Iterative explorations

Finally, after many years, I've been able to get my hands on voice code interface in form of kaldi / dragonfly. At the moment, I've only confirmed that it works. I'll definitely write about the details when I manage to create some setup beyond the simple demo.

At the moment, I'll just leave you with some inspiration:

Writing

I've published 📂Vitamins, nootropics and other boosters, Theobromine, Transaction support for simple in-memory database and the Software.

Blender

Some time ago, I've taken multiple photos of the steam room in public swimming pool, because each time I am sitting there, I am wondering about the quaternary codes in the tiles.

Now I am trying to compose all images into one scene. So far, I've mapped two fspy cameres to one surface and projected the textures to the surface:

I mean I know that there are probably no hidden messages, but still..

Improvements

The cable management cage arrived, so I've installed it. Underside of my desk went from this:

to this:

There is still room for an improvement, but I think that I can live with this for some time.

Notebook

Some time ago, I've mentioned that I am trying to use paper notebook for todo and checklists for work. So far, it worked great.

I am using this pseudo bullet journal method and there is something about it that just works. I mean I still use a ton of computer checklists and issues and mindmaps, but this just works with a different parts of the brain.

I've found that it is a great tool especially when I am programming, as it doesn't require me to switch mental context into some other app to write notes. It may seem counterintuitive, but to actually pick a pen and handwrite a note requires a lot less context switching than to change virtual desktop to notetaking app and start laying it there.

This may be because I was forced to invest massive amount of time into learning how to handwrite, and I suspect that people who weren't forced to take notes in school by hand could feel this differently.

Disadvantages are obvious: the stuff on the paper is dead. I can easily move it to the computer, but the other way is almost impossible.

Random stuff

Inspirative 3D desktop project:

---

Why there isn't more music like this?

---

Guy behind the Astartes project had an AMA on reddit.

---

💡

Yes, my English is probably horrible, but this kind of blog post doesn't qualify for (paid) grammar corrections yet (there is stuff with higher precedence in queue). If you want to change this, subscribe to my patreon. Even with a $1 / month, you can make a difference.