Rendered at 05:47:11 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
jmmv 7 hours ago [-]
I went through a similar "thought process" to build this: https://www.endbasic.dev/endbox.html last year. I originally wanted to "just" launch my binary right after the kernel started... but in the end settled for a full NetBSD base system to get things like network and WiFi configuration to work with ease. (That said, I still hook very early in the boot sequence to launch my own program and take over the console so that the rest of the system is invisible and initializes in the background.)
keyle 6 hours ago [-]
Awesome work. I wish you'd write the process of getting it done step by step. I too would love to DIY my own netbsd.
jmmv 6 hours ago [-]
Thanks. I haven't documented it (yet?) but I did give a talk last year on the process that got me there and the internals behind this system: https://www.youtube.com/watch?v=WZFYTInWAqc
nottorp 12 hours ago [-]
> I found “Building a tiny Linux from scratch” which does most of what I do here but in Rust and a year ago
> yeah, I know, proper C code needs to be scattered with return value checks and sensible reports of errno. I’ve left these out for clarity.
Somewhere, a LLM is trained on this code as we speak :)
But anyway, it's great that people are still interested in learning this stuff for fun.
saghm 10 hours ago [-]
> Somewhere, a LLM is trained on this code as we speak :)
If we collectively have to change the way we do things to better fit our tools rather than changing our tools to fit what we would naturally like to do, then we've failed as an industry.
nottorp 8 hours ago [-]
Hmm LLM hype aside, remember the first writing system (Grafitti) for PalmOS?
Change the way you write to obtain much better recognition.
saghm 7 hours ago [-]
Sadly, I don't remember it. I was born in the early 90s, so although I remember PalmPilots being a thing, I never had one
asalahli 10 hours ago [-]
> Somewhere, a LLM is trained on this code as we speak :)
Had a similar thought when I saw
dd if=/dev/zero of=/dev/sda bs=1M count=1
mid-kid 9 hours ago [-]
I came up with this one on my own many many years ago. Exact order and bs=1M and everything. I taught it that :)
Though I usually do the individual partitions before the main disk, to make sure fdisk doesn't complain about existing signatures as I start making partitions again. These days many disks and SD cards support "blkdiscard" to just wipe the whole thing (and temporarily gain some performance from clearing internal info), and "wipefs" became a thing to wipe the exact sequences fdisk and mkfs look for.
The "dd" method makes most sense on BIOS systems where you want to make sure to wipe the MBR boot code, and know that nothing is accidentally being preserved when you install your preferred bootloader.
jorvi 9 hours ago [-]
> .. There was a brief moment in the early 2000s where the newly introduced SSDs made booting quick ..
Anyone else reading this and getting the feeling someone is reminiscing about a past they didn't live and thus are seeing it through rose-colored glasses because it didn't exist?
In the early 2000s, SSDs had an obscene price tag of ~$1000 per gigabyte, so even an 8GB drive would cost you about $8000. There was nothing for the tech industry to ruin because only 0.01% of people owned a setup with an SSD. And neither Linux, NT or Mach were optimized for them. Hell, to this day Linux' VM system is by default still tuned for swap and slow HDDs (swappiness 60 and all that) because no one has bothered to update the defaults.
jmmv 7 hours ago [-]
They probably meant 2010s.
I remember this was the time when Google started pushing the Chromebook idea and highlighting how it could boot in "just a few seconds". One of the earliest models I got as a dogfooding device probably needed 20 seconds to boot or so. Nice, compared to the awful boot times of machines with HDDs... but not stellar.
But then, in 2011, my wife bought a MacBook Air with an SSD and I was blown away. That thing booted to a full desktop (and not the joke that ChromeOS was) in... 5, 6 seconds? It was ridiculous.
And we have lost all of those gains. I find it painful to witness how a recent Mac chews through I/O during boot or doing any sort of software update (iStat Menus is great to watch this sort of thing), and how these feel slower to that early experience of 15 years ago :-/
MichaelDickens 8 hours ago [-]
I distinctly remember buying my first SSD laptop in 2012 and being amazed at the boot time. At the time, I didn't realize it was because of the SSD, I just thought, "wow, the devs have made some fantastic optimizations in boot time!"
agolliver 7 hours ago [-]
I wrote a batch script that that opened the entire Creative Suite in parallel and couldn't believe how much faster it was than just launching just Flash would have been on the old drive.
keyringlight 6 hours ago [-]
It was also around the time of the early netbooks, when the premise was a lean laptop and you'd do tasks that'd usually be done with relatively demanding local clients by the browser, as google and others were improving their online app offerings. They'd be using things like SATA/IDE to compact flash adapters to keep power down and responsiveness up, they were an ideal candidate for early SSDs as they didn't need a lot of capacity to run a linux + lightweight DE. Then they became just cheap laptops running full windows.
keyle 6 hours ago [-]
When I swapped my disk in my mac for a SSD for the first time... It was insanely faster. Booted in 5-6 seconds. Apps like photoshop opened up cold in 2 seconds. So no, I think it was a huge leap.
I think no matter what the advance in tech is, it will get enshittified to tolerated "computer is working" slowness within months of said upgrade.
abrowne 8 hours ago [-]
Or they meant 2010s?
mrbluecoat 10 hours ago [-]
> the linux kernel configuration menu, a wonderful text menu system with a thousand options which has been baffling new users for about 30 years now.
So true.
In addition to C in the article and Rust linked to in the article, Go fans can use the similar https://gokrazy.org/ project.
helterskelter 12 hours ago [-]
It's be cool to dual boot with a Linux that has a ~1s boot time, drops you into neovim and lets you save text files to a shared partition.
megous 11 hours ago [-]
It's possible to do a similar thing with any old smartphone. :)
I do the “Linux from Scratch” challenge every few years and the last time I did it, I got as far as I needed to be able to build antirez’ LOAD81[1] directly from single mode in a few seconds.. it was so satisfying. I lost the image in some later admin task, but I’m going to do it again, and maybe see about adding threads and SQLite support to LOAD81 alongside the other goodies, just for the fun of it ..
Cool article! I'm working on a tangentially related issue requiring microVMs inside isolated infrastructure environments. Latency isn't really my main priority, but I am always tempted by any option to minimize attack surface. I wonder what it would take to replace the host block mount in this configuration with `vsock` for all communications between the host and guest microVM? Then you could avoid any files being mounted on the host at all while still enabling, e.g., one-way egress to a pre-signed S3 URL via a private VPC endpoint. Very cool article!
quesomaster9000 10 hours ago [-]
Amazon Nitro Enclave does pretty much this, the guest has one method of communication, via vsock, and it's up to you to build the pipes on either side.
It's a huge PITA in practice because whatever you want to run inside some enclave usually ends up being a 'normal program' that needs to talk TCP/IP over sockets... so your vsock I/O becomes a weird mix between a TUN proxy or a SOCKS5 local listener inside the VM that tunnels through vsock.
For example, I have the Windows NT 3.50 kernel compiling from scratch with virtio-net drivers, it's fairly straightforward for me to add a bus driver that runs over vsock inside Nitro Enclave that exposes itself (o,o) as a NIC then handle the tunneling logic in a usermode process in the host - but I don't uderstand the point of why you would do that when you already have sufficient attestation methods that don't require you to do vsock isolation.
matheusmoreira 7 hours ago [-]
Highly instructive. My long term goal with my lone lisp programming language is to do exactly this.
dwroberts 9 hours ago [-]
Tried out doing this kind of stuff a while back to create a custom user space, and I found that suckless’ init, sinit, was quite educational for how to deal with init’s responsibilities in a super minimal way: https://core.suckless.org/sinit/
M95D 12 hours ago [-]
I... fail to see the point of running just one process.
If it's just a PoC, then:
1) I remember seeing a linux firewall/gateway set up to run with just the kernel, without any userspace at all. Completely unhackable.
2) To print some text or run a simple program, I belive DOS without a memory manager would be even faster.
3) It takes 1s to boot linux, but an ordinary PC takes 10s to get to that linux. Even U-boot on ARM takes some seconds to load a kernel.
BTW, if anyone knows any current platform that can XiP a linux kernel, please share.
8 hours ago [-]
yjftsjthsd-h 12 hours ago [-]
> I remember seeing a linux firewall/gateway set up to run with just the kernel, without any userspace at all. Completely unhackable.
Do you remember any details that would let me search for it? Because that does sound cool, and even maybe useful; the thought has certainly crossed my mind that a router or VPN box doesn't really get a lot of use out of userspace... Although maybe it's worth keeping for control/configuration/debugging.
> To print some text or run a simple program, I belive DOS without a memory manager would be even faster.
Or just make your code boot directly. It's not hard to make a .efi, or use
https://github.com/jart/cosmopolitan
to make a binary that runs in many places including bare metal.
M95D 11 hours ago [-]
IIRC, it ran a script as init process that set up the network cards, set up iptables, etc. and then just exited. Kernel would panic (the "init was killed" panic), but the network would still be functional. Automatic reboot on panic was disabled.
To reconfigure, the admin would simply reset it and start the system with "init=/something/else" as kernel parameter that booted to a normal userspace.
yjftsjthsd-h 11 hours ago [-]
Oh, clever; I didn't know you could make Linux panic but keep running.
mikepurvis 10 hours ago [-]
> control/configuration/debugging
This is one of several major arguments made against unikernels in that famous Triton rant from a decade ago:
Basically, even if your application _can_ run as the kernel, and it's desirable for it to run with kernel-level permissions, do you really want production to be a world without strace and iotop and the like?
MertsA 10 hours ago [-]
That's a halted firewall setup. Normally as part of shutdown you would tear down networking in SysVinit or systemd but you don't actually have to do that. When shutting down you can choose whether to power off or just to halt. It's basically like the old Windows "It is now safe to power off your PC".
girvo 7 hours ago [-]
That beautiful orange text on a black background gives me nostalgia warm and fuzzies
kube-system 12 hours ago [-]
Wouldn't this be useful for embedded types of applications where you have a very specific task you want to do and you want to do it now.... like that firewall example?
M95D 12 hours ago [-]
Yeah, but there's a problem.
Do do something actually useful, the program would have to access some data: network, disk, some sensors, etc. Network alone means scanning PCIe for the network card and configuring it, disk access needs controller also on PCIe, then scanning the ports for the drive, reading partition table, mounting the partition, etc.
All that takes a lot more than 1s. The speedup might not even be significant compared to a kernel optimized for that system (all modules built-in, nothing redundant), but full-featured, plus busybox or sysvinit alone.
hylaride 12 hours ago [-]
Embedded devices or other SoC situations, certain limited scope situations where docker is undesirable/unnecessary, etc.
mschuster91 11 hours ago [-]
> I... fail to see the point of running just one process.
It makes sense if you got some legacy piece of hardware that has extremely limited resources, both in terms of RAM and storage. Write your code in Go and you don't even need libc any more.
SpaceNoodled 11 hours ago [-]
At that point, it would seem that an RTOS would be even more efficient - and if multithreading is not necessary, then just run it on bare metal.
ranger_danger 6 hours ago [-]
> Completely unhackable.
I don't think this could ever be true.
-warren 9 hours ago [-]
> cpio is a very weird and ancient program with a command line which makes tar look user-friendly. But let’s not worry about the details for now.
So true. Been using cpio sporadically for many (many) years. And I still have to look it up. But at least I don't have to do `info cpio` anymore.
yjftsjthsd-h 12 hours ago [-]
I do like this as an exploration.
It's possible to boot a VM noticeably faster still, though I'm unclear on whether any of that applies to hardware:
As I'm currently exploring kernel build things, the alternative to `make tinyconfig` is `make allnoconfig` which is supposedly will not disable expert options and might be a little bit safer starting point.
myaccountonhn 7 hours ago [-]
There's an option `make localyesconfig` (or similar)/which uses the kernel modules loaded on your current system.
keyle 7 hours ago [-]
I bought a couple of $60 refurbs online. This is perfect!
ahepp 10 hours ago [-]
buildroot does a nice job of tying a lot of this stuff together
megous 12 hours ago [-]
You may also want to build and run busybox for your tiny userspace.
Other things you may want to experiment with is gen_init_cpio.c from linux kernel tree. It makes creating initramfs file structure easier from scripts.
And finally if sys/isolinux is also fun to use for minimal boot images.
testycool 11 hours ago [-]
"Butt Naked Linux" is how I read it.
I know it's off topic. I accept my downvotes.
Andrex 11 hours ago [-]
I could end up stealing that for my own tinkering distro that never ends up releasing because it's awful.
hungryhobbit 9 hours ago [-]
It's so annoying how he starts out talking about boot speed, does the entire project, and then doesn't even mention how long it takes to boot up! All he ever says is "boots a bit quicker".
MrDOS 9 hours ago [-]
The photo of his monitor shows the kernel exec'ing his binary after 0.92 seconds.
Linux from scratch seems to still be doing fine at: https://www.linuxfromscratch.org. It's going on 27 years now.
> yeah, I know, proper C code needs to be scattered with return value checks and sensible reports of errno. I’ve left these out for clarity.
Somewhere, a LLM is trained on this code as we speak :)
But anyway, it's great that people are still interested in learning this stuff for fun.
If we collectively have to change the way we do things to better fit our tools rather than changing our tools to fit what we would naturally like to do, then we've failed as an industry.
Change the way you write to obtain much better recognition.
Had a similar thought when I saw
Though I usually do the individual partitions before the main disk, to make sure fdisk doesn't complain about existing signatures as I start making partitions again. These days many disks and SD cards support "blkdiscard" to just wipe the whole thing (and temporarily gain some performance from clearing internal info), and "wipefs" became a thing to wipe the exact sequences fdisk and mkfs look for.
The "dd" method makes most sense on BIOS systems where you want to make sure to wipe the MBR boot code, and know that nothing is accidentally being preserved when you install your preferred bootloader.
Anyone else reading this and getting the feeling someone is reminiscing about a past they didn't live and thus are seeing it through rose-colored glasses because it didn't exist?
In the early 2000s, SSDs had an obscene price tag of ~$1000 per gigabyte, so even an 8GB drive would cost you about $8000. There was nothing for the tech industry to ruin because only 0.01% of people owned a setup with an SSD. And neither Linux, NT or Mach were optimized for them. Hell, to this day Linux' VM system is by default still tuned for swap and slow HDDs (swappiness 60 and all that) because no one has bothered to update the defaults.
I remember this was the time when Google started pushing the Chromebook idea and highlighting how it could boot in "just a few seconds". One of the earliest models I got as a dogfooding device probably needed 20 seconds to boot or so. Nice, compared to the awful boot times of machines with HDDs... but not stellar.
But then, in 2011, my wife bought a MacBook Air with an SSD and I was blown away. That thing booted to a full desktop (and not the joke that ChromeOS was) in... 5, 6 seconds? It was ridiculous.
And we have lost all of those gains. I find it painful to witness how a recent Mac chews through I/O during boot or doing any sort of software update (iStat Menus is great to watch this sort of thing), and how these feel slower to that early experience of 15 years ago :-/
I think no matter what the advance in tech is, it will get enshittified to tolerated "computer is working" slowness within months of said upgrade.
So true.
In addition to C in the article and Rust linked to in the article, Go fans can use the similar https://gokrazy.org/ project.
https://xnux.eu/p-boot/
[1]- https://github.com/antirez/load81
It's a huge PITA in practice because whatever you want to run inside some enclave usually ends up being a 'normal program' that needs to talk TCP/IP over sockets... so your vsock I/O becomes a weird mix between a TUN proxy or a SOCKS5 local listener inside the VM that tunnels through vsock.
For example, I have the Windows NT 3.50 kernel compiling from scratch with virtio-net drivers, it's fairly straightforward for me to add a bus driver that runs over vsock inside Nitro Enclave that exposes itself (o,o) as a NIC then handle the tunneling logic in a usermode process in the host - but I don't uderstand the point of why you would do that when you already have sufficient attestation methods that don't require you to do vsock isolation.
If it's just a PoC, then:
1) I remember seeing a linux firewall/gateway set up to run with just the kernel, without any userspace at all. Completely unhackable.
2) To print some text or run a simple program, I belive DOS without a memory manager would be even faster.
3) It takes 1s to boot linux, but an ordinary PC takes 10s to get to that linux. Even U-boot on ARM takes some seconds to load a kernel.
BTW, if anyone knows any current platform that can XiP a linux kernel, please share.
Do you remember any details that would let me search for it? Because that does sound cool, and even maybe useful; the thought has certainly crossed my mind that a router or VPN box doesn't really get a lot of use out of userspace... Although maybe it's worth keeping for control/configuration/debugging.
> To print some text or run a simple program, I belive DOS without a memory manager would be even faster.
Or just make your code boot directly. It's not hard to make a .efi, or use https://github.com/jart/cosmopolitan to make a binary that runs in many places including bare metal.
To reconfigure, the admin would simply reset it and start the system with "init=/something/else" as kernel parameter that booted to a normal userspace.
This is one of several major arguments made against unikernels in that famous Triton rant from a decade ago:
https://tritondatacenter.com/blog/unikernels-are-unfit-for-p...
Basically, even if your application _can_ run as the kernel, and it's desirable for it to run with kernel-level permissions, do you really want production to be a world without strace and iotop and the like?
Do do something actually useful, the program would have to access some data: network, disk, some sensors, etc. Network alone means scanning PCIe for the network card and configuring it, disk access needs controller also on PCIe, then scanning the ports for the drive, reading partition table, mounting the partition, etc.
All that takes a lot more than 1s. The speedup might not even be significant compared to a kernel optimized for that system (all modules built-in, nothing redundant), but full-featured, plus busybox or sysvinit alone.
It makes sense if you got some legacy piece of hardware that has extremely limited resources, both in terms of RAM and storage. Write your code in Go and you don't even need libc any more.
I don't think this could ever be true.
So true. Been using cpio sporadically for many (many) years. And I still have to look it up. But at least I don't have to do `info cpio` anymore.
It's possible to boot a VM noticeably faster still, though I'm unclear on whether any of that applies to hardware:
https://jvns.ca/blog/2021/01/23/firecracker--start-a-vm-in-l...
Other things you may want to experiment with is gen_init_cpio.c from linux kernel tree. It makes creating initramfs file structure easier from scripts.
And finally if sys/isolinux is also fun to use for minimal boot images.
I know it's off topic. I accept my downvotes.