Chapter 3: What’s Else Is Needed? #

Script all the things! But Not Like That™ #

Computers are, generally, really good at repetitive tasks. People are, generally, really bad at telling computers what they want.

This means that developers (which, I assume, is a large portion of those still reading this) need to make systems that let users automate tasks in a more user-friendly way than they would do it.

Previously, I mentioned that I run backups with a little backup script:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

#!/bin/bash
rsync -a --delete -P /mnt/data /run/media/vega/BigOlBackup
rsync -a --delete -P /mnt/win/VegaProductions /run/media/vega/BigOlBackup
rsync -a --delete -P /etc /run/media/vega/BigOlBackup
rsync -a --delete -P --exclude={'Xilinx','intelFPGA'} /opt /run/media/vega/BigOlBackup
rsync -a --delete -P /srv /run/media/vega/BigOlBackup
rsync -a --delete -P /usr /run/media/vega/BigOlBackup
rsync -a --delete -P --exclude={'.local/share/Steam'} /home/vega /run/media/vega/BigOlBackup
rsync -a --delete -P /mnt/win/ProgramData/Ableton /run/media/vega/BigOlBackup/win
rsync -a --delete -P /mnt/win/Users/Vega/Documents /run/media/vega/BigOlBackup/win
rsync -a --delete -P /mnt/win/Users/Vega/Downloads /run/media/vega/BigOlBackup/win
rsync -a --delete -P /mnt/music /run/media/vega/BigOlBackup/
rsync -a --delete -P /run/media/vega/Archived/SD /run/media/vega/BigOlBackup
notify-send "Backup Complete"

And for those that know what they’re looking at, this is pretty basic. The problem is, a lot of users would feel totally overwhelmed by this. The idea of reading through the rysnc man page alone would make many people run away. So, one option is to just make application specific software that provides a nice GUI. And sure, that works. The problem is that prevents people from being able to do some of the more complex operations, doesn’t allow for convenient triggering (for example, I could put this script on a Systemd timer or have it trigger as soon as it detects that my removable media has been inserted).

But there’s a lot more than backups at stake. There are a surprising number of people out there whose entire job could be replaced by a 100 line python script, especially if you use OpenCV or TensorFlow. These people could be doing something much more meaningful with their lives.

The problem isn’t that the tools don’t exist to do this automation, it’s that a lot of people don’t know they exist and that they don’t have the skills to apply them.

How do I propose we fix this?

Visual programming.

Yeah. I know a lot of people hate it. But, provided it’s high enough level - so, probably higher than Pure Data or Node-RED or even the Enso language mention in the last chapter. I think the right way forward is something that generates nodes based on a small text description and some provided examples as text, images, or other data. This would let a user, for example, scan in and manually enter the data from a form a few times, then leave the computer to do it moving forward - probably with a little output indicating if it was under a threshold of confidence in a certain transcription.

The other big necessity to make this work is using the Linux philosophy of treating everything as a file (or folder). This would make it much easier for this language/interface to let users, say, open a running program’s ‘file’ and - with support - grab data out. This could be as mundane as how many browser tabs are open or as advanced as pulling the current geometry or live rotation angle out of 3D modeling software. With enough support from applications, the work flow potential is mind-blowing, hence the previous mention of this in the last chapter with tool composition.

My point here is that by making user-friendly but open tools we could dramatically reduce the amount of repetitive work done by many people and, more importantly, not just support one-off use cases with application specific software but instead empower people to make their own tools that fit their specific needs that developers could never anticipate. Will this result in some mildly horrific tools being made? Probably. But that’s the beauty of it.

Optional Security #

TempleOS is a really weird operating system. From Wikipedia

But the part that’s cool to me? TempleOS gives no, and I mean absolutely zero, fucks about security, now, that’s at least in part because it doesn’t have networking (though there is a fork with networking) but even then, there’s no concept of an admin or ’normal’ user. There’s no system-protected processes. You want to flip arbitrary bits, directly send data over IO lines, whatever. The OS will not stop you. The entire thing runs in ring 0, which means you always have total control over the hardware, but it also means there’s nothing stopping you from massively fucking everything up and crashing the system, corrupting everything, etc.

( You may want to watch TempleOS (Linus Tech Tips, YouTube) if this is interesting to you)

And, okay, yeah, that’s obviously not ideal for normal use, but I think there’s something to be said for having the option to say fuck it, slam your fist down on the manual override button, and take full control of the system. This is what I mean by optional security - no HAL 9000 in the way. No sudo, no ’no’. In this mode it should also be easy to turn off all networking and malware protection, so a super-fuckit No firewalls, no threat scanning, etc. switch.

Running as root in Linux is almost this. It could even be argued that it’s better, since toying with memory though /dev/kmem is probably easier than trying to deduce the physical memory map. That said, there are still some restrictions on the root account compared to running directly in ‘kernel mode’.

Backwards Compatibility via Compatibility Layers #

And yeah, that pretty much sums it up. Currently, backwards compatibility is often a choice of actually going to better technology and fixing bugs - like the Excel leap year problem or supporting older files without them massively breaking. There’s really no reason we couldn’t do both using comparability layers. The cost, I think, would mostly be in accepting that the transition to those comparability layers will be rough and that something will just have to be upgraded until that compatibility infrastructure is in place.

There’s no reason we shouldn’t be able to run very old software, but also no reason to use that as an excuse against moving forward.

A fairly interesting solution to some of this might be to implement hardware compatibility via FPGAs, that is, loading either full soft-core CPUs or instruction set translation logic onto a tightly integrated FPGA. Higher end retro console emulators are already using FPGA emulations of the original CPUs anyway, so this isn’t some crazy fantasy. To add even further hope, AMD owns Xilinx and Intel own Altera, so the two biggest CPU manufactures both already own the two biggest FPGA manufactures (which, is a problem in some ways). There’s nothing stopping them from making integration standard. (well, other than cost and market segmentation)

Hardware Modularity, Repairability, and Customization #

According to Wikipedia, Right to Repair is

legislation that is intended to allow consumers the ability to repair and modify their own consumer electronic devices, where otherwise the manufacturer of such devices require the consumer to use only their offered services

This is a big deal, with big player like Apple and John Deer lobbying incredibly hard against it. I can’t possibly explain even a fraction of all the nuance in the arguments, so instead I recommend reading this post from the Electronic Frontier Foundation (EFF) for a quick overview.

As for modularity and customization, I mean that hardware should be made such that components can be swapped out. For example, most desktop computers do a decent job of this, where each component - the CPU, graphics card, power supply, RAM, etc. - can be upgraded individually (1). This concept desperately needs to be more widespread. It simultaneously cuts down on eWaste by increasing longevity, allows consumers to only pay for what they need, and lets consumers buy-now-upgrade-later if they can’t afford what they really want to start with.

The only real downside for the consumer is that the products might be hair bit thicker? Yeah, I think that’s a fair trade.

Network Transparency #

Network Transparency is a neat concept, basically, it means you can use a program over the network as if it were local. I think the coolest example of this is the X Window System on Linux, which lets you run graphical programs on a remote computer, even one with no screen attached, and have the window be visible on your own computer. For example, while you could always set up a file server to access file remotely, you could also just use X’s network transparency to forward the entire file manager window to your computer and browse files like normal (not that you should)

I’d love to see more applications and backend software (like X) support this.

The Ability to go Back In Time #

A lot of software already runs on an idea of ’events’ or ’transactions’, whatever you want to call it. What I want is for these to always be undo-able, at least for a window of time or number of past events. This might mean updates, changes to a file, or just scrolling down on a screen.

A refresh should try to bring you to the previous state, dismissed notifications should be able to be un-dissmissed.

As previously mentioned, you should be able to go back to a saved copy of any web page you’ve visited. You should be able to revert your system to a restore point (Good idea from Windows, poorly implemented).

EVERY
F**IN'
TIME
💢
🦊 pic.twitter.com/5JGKXqxIwF

— FPV 🟡 眼鏡の狐 (@F0XnR0LL) January 7, 2021

Open Sensors and Generators #

Hardware is gradually becoming more general purpose, and as this happens, it would be amazing if they were opened up enough to allow for experimentation. Think using full on software defined radio chips for Wi-Fi and Bluetooth, such that they could be reconfigured on the fly and a different antenna attached to make use of GPS, cellular, or satellite internet. There’s already a mini PCI-e card SDR on the market. Like this, but it’s my understanding that many Wi-Fi chipsets are effectively just limited SDRs running locked down firmware anyway. Why not open it up?

I also want more platform to have GPIO access. On a full on desktop or laptop, this would have to be done with care to avoid people straight up frying devices, but I think the wide array of hats available for the RaspberryPi shows that having this as an option would really allow for some cool hardware to exist.

I also want senors and IOT devices to be more open and extendable. Give me a serial out, let me wire it up to my own home automation. I mentioned above that I have a CO2 meter from CO2Meter.com, but I’m looking at getting this CO2 sensor just so I can get reading out of it easier.

IOT especially needs to be open. In 2020 alone, Wink started charging customers $5/mo for app access to IOT gear they already owned, and Sonos bricked devices with ‘Recycle Mode’.

Software that breaks the mold #

There’s endless software out there, but a lot of it does mundane tasks in a mundane way. I’d like to showcase a few things that I think really found new ways of doing things.

• Along with Blockhead, Radium is a very strange Digtal Audio Workstation with its own interesting workflow
• ORCΛ is a really neat esoteric programming language music sequencer… thing. It’s so different from everything else it’s hard to describe.
  • You may want to look at the Live Coding Music Tools from the Music section of this website.
• MasterPlan by SolarLune is a task management program focused on Indie game development with a really need mind-map style workflow
• Habitica is task and habit tracking software that gamifies things
• Hexels is an art program where you can work with hexgons or triangles instead of big squares for “pixel” art.
• On the note of art, Black Ink has some really neat brushes and support for a crazy high resolution canvas, and a node based language for making brushes or working with the layers.
• … Clearly this list can’t be exhaustive, but I do hope this has provided at least some inspiration.

For even more, you may want to check out The big list of Generative Art Tools in the Design section of this website

While all of this is cool, I do want to point out I really, really wish there were more tools that made the more mundane tasks better too. There’s been very little innovation in the user interface of browsers (like, Vivaldi may have vertical tabs but Firefox has had Tree Style Tabs for much longer) text editors, and spreadsheets(3). The was, briefly, a bit of innovation in the slideshow space with Prezi but it was still pretty gimmicky and not a dramatic improvement.

Developers have gotten better tools pretty rapidly recently though, as there’s a crazy number of replacements Alternatives to coreutils and other basic tools that have come out over the last few years. This gives me hope that tools for “normies” will catch up and see innovation too.

Hardware for Open Experimentation #

Microcontrollers, like the infamous Arduino UNO (ATMEGA 328p) and the much nicer Raspberry Pi Pico (RP2040) are awesome for letting people experiment and make crazy contraptions. FPGAs are finally starting to get open source toolchains and low enough price options to allow for people to make their own soft core CPUs and custom logic. Single board computers like the Raspberry Pi 3B+ are cheap enough to be thrown at many problems. This is all awesome.

Unfortunately, I see one big thing being left behind: the PC. If we really want innovation, it’d be awesome if there was an affordable PCIe FPGA card that could have bitstreams flashed onto it for specific applications to accelerate different workloads. GpGPU programming is nice, but custom accelerators would be even better!

Furthermore, I’d really like to see high speed not-USB-based GPIO for desktops so that people could experiment with crazy ideas like using microfluidics and diode logic for computation, playing with FeRam, or even ternary logic - Even if most of these ideas are dead ends, they’re cool and I want to see more of them.

Stop wasting time on stupid shit that nobody cares about #

Microsoft has been spending a lot of time changing to the new UI, and their calculator has been updated like a dozen times for UI now but still SpeedCrunch remains 1000x more usable and tools like WolframAlpha remain superior yet. Stop spending time on shit literally nobody gives a shit about and that nobody should be using. If something is dumb for the sake of being dumb, like Harder Drive: Hard drives we didn’t want or need (Tom7, YouTube) sure, but otherwise, let’s make things actually better.

Give me a fucking physical knob #

↑ My eurorack setup, for making something vaguely resembling music

I can keep this one brief. I love VCV Rack, I’ve even developed my own modules for it yet here you can see I have a hardware modular music setup? So, why? Because physical, real, tactile knobs make a difference (2) . I’m not saying everything needs to be physical, I’m saying that even if Popular Mechanics was wrong back in 2009 when they said:

We should be careful of getting too comfortable with our touch screens and not forget the value of having a physical interface. Hell, in that same picture above is a ROLI Seaboard Block (a product I actually recommend against, for various reasons) which is Amazing for its ability to convey a boatload of information to the computer in a tactile way, like, seriously, look at the MIDI data from this little jam session:

There’s just no way I could have programmed that in manually, and despite the chaos, I can assure you the sound of this was amazing.

So as Bret Victor put it all the way back in 2011 in A Brief Rant On The Future Of Interaction Design, “Our hands feel things, and our hands manipulate things. Why aim for anything less than a dynamic medium that we can see, feel, and manipulate?”

1. Within the normal limits of compatability, you can't drop a crazy fast graphics card in without upgrading the power supply, and often the CPU, Ram, and motherboard all need to be from the same generation.
2. Well, and there's some techniques that VCV rack can't do well, like feedback, on account of every cable incurring one sample of delay. This is getting into the weeds though.
3. There has been some innovation in spreadsheets, but none of it is really crazy. There's Cubeweve which bills itslef as a "Multidimensional" spreadsheet, but is rather lame and awkwardly mouse driven from what I can see. subset.so looks promising as well; however, it seems like a pretty small improvment. Getting a bit deeper, natto.dev and Mito both use python in their data processing, albeit with radically different interfaces. Mito is probably the closest to what I'd actually consider a real improvment.