1 - The CPU #

These are CPUs or Central Processing Units. They’re the beating heart of your computer, doing the number crunching. The white, rectangular one is of one of the first CPUs in the lineage leading up to CPUs like the ones used in desktop’s today, with further successors shown in order from top left to the bottom right. The metal-capped one at the bottom right, while a tad old, is visually quite representative of a modern desktop CPU, though the CPUs in a laptop or smart phone do look quite different still.

One of the nifty things we do pretty easily in linux is get information about our hardware directly. Just as when we were installing Arch and we used lsblk to see an overview of the disks on the system, we can use some other tools to find out some other information about the system. Let’s start off basic and see what CPU you have. Go ahead and run

1
2

╭─vega@lyrae ~
╰─➤  cat /proc/cpuinfo

This is actually just using that same cat command we used before. This time we’re using it to read the system generated file that tells us about the processor in this system. I’m going to provide the output from my system for reference:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD Ryzen 7 1700 Eight-Core Processor
stepping        : 1
microcode       : 0x8001137
cpu MHz         : 2018.119
cache size      : 512 KB
physical id     : 0
siblings        : 16
core id         : 0
cpu cores       : 8
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb hw_pstate sme ssbd sev ibpb vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 xsaves clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca
bugs            : sysret_ss_attrs null_seg spectre_v1 spectre_v2 spec_store_bypass
bogomips        : 7688.44
TLB size        : 2560 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 43 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate eff_freq_ro [13] [14]

Alright, that’s a whole lot of information, let’s break it down.

First of all, almost all modern Central Processing Units (CPUs) have multiple cores, and as with most things in computers the’re counted from 0, so on a 4 core computer you’ll have cores 0, 1, 2, and 3. Multiple cores simply let your computer do things in parallel, running multiple programs or tasks at the same time

Next is the vendor ID, family, model, name, and stepping. My processor is an AMD Ryzen 7 1700. There’s a pretty good chance your CPU will be made by Intel, and as such the family, model, name, and what not will reflect that. AMD and Intel are practically the only two laptop and desktop CPU providersWith the exception of Apple’s newer hardware, which runs their own silicon, and some low end devices like chromebooks which may run processors like you’d find in phones., though in the future we’ll work with boards that use CPU’s made by other manufactures. Really, most of this doesn’t matter. Some CPUs are nicer than others, and if you follow the market or care it’s easy to get a sense of a CPU’s speed just based on it’s name, but let’s move on for now.

1
2
3

microcode       : 0x8001137
cpu MHz         : 2018.119
cache size      : 512 KB

Each of these things is very important, but I’m going to start with cpu MHz as it’s probably the easiest to understand, however, to understand this we’ll need to get even a bit lower level than this and learn about the electronic component that changed humanity: the transistor.

Clock Speed #

However, to get to transistors we’ve gotta go just a bit further down the rabbit hole to the relay. Relays are super simple to understand, they’re just a metal switch that is pulled open or closed using another input signal (usually a magnet pulling/pushing the switch closed/open) basically imagine a light switch, where the switch itself is controlled by yet another electrical signal. Relays are slow though, they require a physical metal plate to move to change the connection. Because of this they have limited reliability and worth note they’re actually loud. You can hear an audible click of the switch as they change state.

Enter the vacuum tube. Though rarely used today outside of high end audio and old radios, for a period of time the logic inside a computer used these tubes. Essentially theres three important parts of the tube, the Cathode, Plate and Grid. Put very simply the Cathode emits electrons and the Plate collects them. Where it gets interesting is the grid in between. By applying a voltage to the grid a signal can be controlled giving us the same ability to turn something on or off by a third wire as in the relay.

Though the real break though here was the fact that this was no longer a mechanical system. With relays there was a very slow limit on the rate at which they could respond reliably, but with tubes this increased exponentially. Tubes were still expensive, large, and power hungry though. However, with their invention early computers saw a massive boost in speed with a decrease in cost.

Finally, enter the transistor. The physics here isn’t that far removed from the vacuum tube, only now instead of a vacuum the electrons are moving though a semiconductor - typically silicon. Again this brought a massive shrink in physical size and increase in that rate at which it could respond. Pictured above are two discrete transistors, however, this is where things get mind blowing:

The Ryzen 1700 CPU in the computer I’m typing this on has 4,800,000,000 transistors in a package that is only 213 mm², and finally, we can refrence the number output by cat /proc/cpuinfo

At the moment I got that output the transistors where being turned on and off at a rate of 2018Mhz*. or 2Ghz. That’s 2000000000 times per second.

However, this system can run up to roughly 3.8Ghz. The faster this speed the faster your computer; however, your CPU will also use more power and run hotter. It’s for this reason that most systems adjust the speed based on load. Doing simple things like writing this document and as seen with that output my system runs at nearly half speed which is actually the slowest it can run. Because the computer is hardly doing anything right now the majority of that switching is actually just doing nothing but using power running ’no operation instructions’ the functional equivalent of just running 0+0 while it waits for something to do.

While not listed in this output, by googing the Ryzen 1700 we can also see it was made on a 14nm process. Put a bit reductively, this means each transistor is 14nm wide. For context, a human hair is about 70,000nm wide. This is relevant, because generally the smaller the process the faster a chip can run (max speed) while using less power.

But, I want to put this into perspective a bit: At the top of this document, there’s a white, rectangular chip which is a very old CPU: The Intel 8088 from 1979, it’s only 8-bit, runs at 16Mhz, was made on a 3μm process, and has ~29,000 transistors inside. So, each individual transistor is over 200 times larger, they switch much slower, and there’s a human-understandable number of them. But why bring this up? Because I broke one open!

What you’re seeing here is the actual silicon, with the individual MOSFETs (transistors) causing the different reflections of the light and the gold wires going out to the pads which eventuall go out to the external pins, this isn’t all that different from a modern CPU though. While pointing out what sections do what isn’t hugely relevant, sufice to say different sections do different jobs. One block might be cache (talked about further down this page) while another might be the chain of transistors that do the logic to add two binary numbers together. At the end of the day, these tranistors are real, they’re tiny, but they’re real. This is the “magic” behing the fancy chip. These, therefore, have real, physical limitations. The electricity running through them can only be switched so fast before it becomes hard for the circuit to differentiate a 1 from 0, on from off.

Back to power and speed in a modern system: The OS itself actually tells the processor what speed it should be running at. In Windows, when you change your power plan to ‘high performance’ one of the major things it does is not allow the processor to run at a slower speed, and in Linux you can similarly control this using some cpu speed commands. We’ll get to that later though.

Finally it’s worth note that on some systems, primarily high end desktops, you can actually run your processor outside of factor specifications by increasing the maximum clock rate of the processor. Doing this can lead to system stability issues and obviously leads to a higher power usage and heat output though. This process is known as ‘overclocking’ as your taking the internal clock of the processor beyond it’s rating. My CPU, a Ryzen 1700 at the time of writing, has actually been over clocked in order to get 3.8Ghz on all of the cores.

Finally, I want to note that clock speed is far from the single measure of performance. Two chips that both have the same clock speed and same number of cores may still perform radically differently depending on many, many things about how the processor works. Note that I didn’t say each ’tick’ of this clock is a single instruction? That’s because different processors have different Instructions Per Clock (IPC) values. It’s totally possible that a slower clocked CPU will outperform a faster one because it is able to execute more instructions per each tick of the clock. IPC can also varry depending on the workload, as some instructions will take more clock cylces than others and different tasks will require a different mix of instructions. If in doubt, look at benchmarks that directly compare the performance of CPUs in various workloads with their real, measured execution times.

Microcode #

Modern processors are very, very complicated. So complicated in fact that there is a full very tiny computer in your processor. This computer does multiple things, but the main one we’ll talk about is the translation between types of machine code. In order to understand this we’ll need to look a basic program written in the language ‘C’.

1
2
3
4
5
6
7
8

int main() {
  int a;
  int b;
  a = 8;
  b = 16;
  a = a + b;
  return 0;
}

1
2
3
4
5
6
7
8
9

        push    rbp
        mov     rbp, rsp
        mov     DWORD PTR [rbp-4], 8
        mov     DWORD PTR [rbp-8], 16
        mov     eax, DWORD PTR [rbp-8]
        add     DWORD PTR [rbp-4], eax
        mov     eax, 0
        pop     rbp
        ret

This in turn gets turned into binary as can bee seen by this screenshot generated using https://godbolt.org/

As with all things technology progressed and 64bit cpu’s came along. Many attempts were made to make 32bit programs run with backwards comparability at high speeds on these CPU’s, though in the end AMD made the method used today, dubbed the x86_64 instruction set. Overtime this instruction set was expanded with various additions. We can actually see the names of these additions that are available on the CPU in the system by looking yet again at the output of cat /proc/cpuinfo and looking at the flags section. While not all of these signify instruction set additions, many do. The most common ‘famous’ if you will is SSE, of which there have been multiple revision, the first version alone adds 𝐖 70 instructions, which are used to make math faster

Alright, so finally, enter Mircocode. At some point all of this became a lot to manage and processor designs evolved even further, getting exponentially more complicated and faster with more and more instructions, so they added this little computer which has the primary duty of turning the mess of countless possible instructions into yet even smaller instructions that the heavy duty ‘real’ processor does. Every once in a while a problem will be found in the way this is done, or a security vulnerability in the hardware itself may be found, and your CPU manufacture will release a microcode update.

That update version is what you see on that line of cat /proc/cpuinfo

Now, at the beginning of this guide I said I wouldn’t provide useless information, and I’m sure that all looks pretty useless without more context. In general, this is interesting because you can (and will need to) write some assembly code by hand every now and then. Thankfully, its usually not x86 assembly which is very, very difficult to write but instead assembly on much smaller, micro controllers (basically really tiny computers), where that assembly is necessary because the system is so limited.

Don’t worry about what x86 means here, we’ll come back to that in Chapter 3½ - Architectures

If you’re interested, check out 𝐖 Microcode and MicrocodeDecryptor

Cache #

Okay, next up is cache. Cache, just like in the real world, is a small place to store things. Most people like to think the majority of what a computer does is raw number crunching, doing hard math, but the truth is more often than not it’s just moving data around. This follows a path from slowest and cheapest storage up to the fastest but most expensive. Typically this order looks a bit like

Hard Drive → Solid State Drive → Ram → Cache → Registers , where the price for storage on a hard drive can be under $0.10Gb, Ram upwards of $10Gb, and Cache and registers, which are storage baked directly onto the CPU, cost much, much, more to implement. It’s of note that these not only are faster with each jump - able to read and write data more quickly -, but also usually physically closer to the CPU. A hard drive can be 10’s of feet of wire from the CPU while the RAM can be a few inches at best, and the cache and registers are physically part of the CPU. Most of the data above the HDD/SSD level is actually just smaller subsets of each previous pool. In fact, on modern CPU’s there are actually usually 3 levels of cache, each with a progressively smaller size but increase in proximity to the executed instruction.

We’ll come back to this idea a bit in Chapter 3.10 - The Bigger Picture .

Put simply, just as with adding more Random Access Memory (RAM) to your system, having more cache means more information can be within arm’s reach of the CPU to do work on at any moment.

Let’s take a look. install the hwloc package using yay and then run lstopo and you should get an output that looks a bit like this

The stuff on the right are connections around the system, you can ignore those for now, but see the various cache layers, designated by L3, L2, L1d and L1i, and you can see how each core has it’s own cache. Finally, you can see that each core has two processing units? Hey, wait, what’s that all about?

Hyper threading, or SMT, or whatever the new term is for it, is a way of adding paths for doing things to keep every part of the CPU active. We’ll go into this more later, but for now, suffice it to say it’s a way of getting just a bit more performance out of a system.

Interrupts #

While not listed in the /proc/cpuinfo output , it’s still relevant to bring up CPU interrupts, which are, fittingly, listed in /proc/interupts

Output of cat /proc/interupts on my system

Output of information for the first 3 cores on my system displayed graphically using Netdata

CPU interrupts are a bit confusing because the term interrupt can refer to the action itself, the code that executes, or the general concept. In general a CPU interrupt is when something needs to poke the CPU to tell it to interrupt what it’s doing. This can be when physical pin in the CPU gets a signal, a timer going off, or a program dividing by 0. We’ll cover this more in depth later, but from the hardware perspective what’s relevant is the CPU is generally so fast compared to other actions that the rest of the system mostly just triggers an interrupt to let the CPU know that a task the CPU asked to be done has completed. The most common example is requesting data from disk (Hard drive or SSD) takes so long, even at just a few milliseconds, that the CPU can get millions of things done while it waits for the request to be fulfilled. So rather than waste that time having the CPU poke the device asking “Are you done yet? Are you done yet?” the disk (or the controller it’s hooked up to, like the chipset) issues an interrupt to the CPU to let it know a task has completed. USB works in a similar way, though the really old keyboards with the circular connector (PS2) actually did interrupt the CPU directly on each key press.

We’ll come back and explore this more later, but if you’re really interested you can read ahead by going over these links and reading these man pages: Oreilly on Interupts, Oreilly cont., man trap, man syscall,

Power #

The CPU is one of the most power hungry parts of your computer, and as such it does some interesting things to try to keep it’s power usage lower. Most notably, it’ll turn down its clock speed to do less operations per second, but theres other things like entering sleep and hibernation states the processor is also capable of. The main thing you might want to concern yourself with from a performance to power consumption stand point is how aggressive this clock speed changing is. You can go as extreme as to tell the system to never let the processor get to full speed or to say ‘only run at max speed all the time’. The latter is actually quite useful as often it can take quite a few clock cycles after a heavy load is started for the processor to come up to speed, but if it’s always running full steam ahead this is a non issue. This entire topic is particularly relevent to laptops, and on Arch you may want to install a program to manage some of this for you or set sane defaults to save battery. I’ve had good luck using the tlp package. The cpupower package is helpful for looking at how arch handles cpu scaling and it lets you directly set these min and max frequencies or pick the scaling governor to handle how aggressive the switching is. We’ll get more into tuning the system to use less power in Chapter 3.7 - Power .

I very strongly recommend looking into this on any system though, as by default some CPUs will run at the minimum frequency only.

If you want to actually see how much power is used, rapl should do the trick, just be sure to actually put the system under load first- stress-ng --cpu 16 (or whatever your thread count is) should work.

If you are also on a Ryzen system, you may need to use rapl-read-ryzen / (aur) to get this output

We’ll talk about delivering power to the CPU in a bit, when we talk about the Voltage Regulation Module (VRM) on the Motherboard.

Physically, what and where is this thing? #

The CPU is on a square or rectangular board that is usually covered by a large heat sink used to keep it cool under load. It connects to the motherboard via hundreds of small, gold plated pins to send and receive signals (which well will discuss in depth later) from all around the system. Even though most modern CPUs are x86_64, generation to generation and cross manufacturer there are changes in the number of pins and the way they are arranged, meaning getting a new processor that’s not from the same generation usually won’t work. Furthermore, most laptops have soldered on processors that can not be upgraded to begin with.

Numa Nodes? Chiplets? Die Yield? #

[TODO] + CCX Groups and pinning processes to cores -

https://github.com/AdelKS/LinuxGamingGuide

Branch Prediction & Pipelining #

I don’t want to dive toooo deep into the weeds, but because of the Meltdown and Spectre vulnerabilities, I find it necessary to briefly talk about branch prediction.

1
2
3
4

if condition == True:
	doThing()
else:
	doOtherThing()

The CPU will do a good job guessing which path will actually be taken. This is done because CPUs are actually pipelined.

CPUs have to do a few things per each instruction, generally they need to FETCH the instruction, DECODE what that instruction is telling the CPU to do, EXECUTE that instruction, and then WRITE BACK the result of the computation.

This could be done in order,

but because the CPU uses different parts internally, they can be pipelined, sort of like an assembly line. While one instruction is in WRITE BACK, the next instruction is being EXECUTED, the next-next instruction is being DECODED, and the next-next-next is being FETCHED.

Now, there’s a lot more that goes into this, times where this doesn’t work -like if two instructions work on the same data, have to wait for write back before that data can be executed on- but the relevant bit here is that as much as we can, we want to keep this pipe line full. That is, we want to keep each point in the assembly line busy. To do this, the above code with an if utilizes branch prediction, and assuming it’s correct, is able to keep the pipeline full and running like normal. If it’s wrong, it has to empty the pipeline, go back, and do the correct instructions instead.

The issue with this that lead to the Spectre bug was that the assumed correct path to be executed may include a security level elevation, that is, it might be code that should only be able to be executed by the administrator (root) user, but in the prediction code there’s no check there, so if the prediction can be abused, code can be executed as an admin- a doomsday level vulnerability.

Meltdown is similar- you can read how it works on Wikipedia (actually human readable and explained well).

SOC? Microcrontrollers? #

In the blurb under the CPU picture I mentioned both SOCs and Microcontrollers [TODO]

A note on x86, vendors, and CPU politics #

[TODO]

Intel took an AMD Zen microarchitecture presentation and turned it into a patent

china trying to make their own silicon , apple to own SOC (Called it), etc.

https://www.agner.org/forum/viewtopic.php?f=1&t=6

New Intel chips won’t play Blu-ray disks due to SGX deprecation (Bleeping Computer)

Other things done by the CPU: #

The CPU has a lot of dedicated hardware inside for specific functions, for example many CPUs have a dedicated hardware random number generator, a “media engine” (encoders/decoders) (Quick Sync), Encryption, DSP

“Management Engine” #

The future #

From the past: Analog Computers #

Built-in Programmable Hardware #

Offloading Security: The TPM, Apple’s T2, Microsoft’s Pluton #

Yet More Acceleration #

TCP offloading, DSP++, Software Defined Radio?, In-Memory Processing, Tensor Cores, etc.

Further Reading #

If you want to know about the internals of the CPU itself, check out Chapter 29 - Let’s Make our own CPU which goes into the componets of the CPU, why they’re made the way they are, and what the future of CPU architecture may look like.

Putting the “You” in CPU (Lex Mattick & Hack Club)

Why you should use ’nproc’ and not grep /proc/cpuinfo

Spyware at The Hardware Level - Intel ME & AMD PSP (Mental Outlaw)

Cascade: CPU Fuzzing via Intricate Program Generation