(Note - this accompanies a previous thread on lisp machines which was inspired by the same talk)
The Register’s Liam Proven has released an article on the opportunities to redesign operating systems so as to better take advantage of non-volatile memory (which promises the best of both RAM and solid-state drive). It is entitled “Starting over: Rebooting the OS stack for fun and profit” and he added it to this serialization of his FOSDEM speech this year.
The author is motivated by a desire to eliminate the “technical debt” of today’s operating systems which he considers have degenerated into bloatware. To this end he suggests two thrusts - a modern lisp machine - possibly based on Dylan; and a Smalltalk environment built on Oberon. I’d have thought on the Framework our existing operating systems like 9Front or Genode would be a better choice to host modern Smalltalk descendants which include Squeak and Pharo.
But the other part of this discussion would be to imagine whether there are hardware implications. This is hard as Intel discontinued their Optane non-volatile memory, but The Register recently described several promising non-valatile memory technologies in development.
The non-volatile memory has flopped as a real world hardware.
It offers worst of both RAM and SSD. Like SSD and unlike DRAM it wears down, and writing single memory location repeatedly destroys it quite quickly. SSDs have wear levelling to counter that problem but nvdimm cannot afford to stop and go out with the garbage so it lacks the feature. Even with that limitation it does not offer RAM speeds.
Unless there is significant breakthrough in the technology real soon it does not make much sense planning for it. And like higher density cheaper batteries that are ‘almost there’ for decades until the device is on the market it’s hard to predict how new developments turn out.
That said, some research systems like keyos/coyotos exist that use the RAM as if it were persistent, and instead of writing filesystems they snapshot the system RAM repeatedly do disk.
Like VNC and other display compression technologies that transfer only the content that really changed through the bandwidth bottleneck rather than arbitrary other data the system memory snapshotting can provide good performance at least for some common applications.
I agree it is no silver bullet and like anything will eventually wear out but suspect that this problem is overstated, maybe even a post-hoc attempt to rationalise why it was not commercially successful. The withdrawal of Optane seems more to do with boring matters concerning allocation of fab capacity and restless shareholders rather than a flaw with the technology. Many things are not the problem initially made out to be. How often do you change your display because your desktop is burned in?
Liam Proven’s comment on the discontinuation of Optane suggests a “chicken and egg” situation here. There was a breathrough technology
which failed - according to the writer’s analysis - because nobody was willing to take advantage of it.
The information is out there, no need to trust anybody. Get some datasheet that gives the durability of the Optane nvdimm cells, and measure how often busy kernel spinlocks are written on your system with performance counters. If you need some guidance on what is a busy kernel lock you might peruse the mailinglists or ask someplace like #linux-rt on oftc.
Because nobody could make advantage of it in the form in which it was marketed. I would expect that the experiences from the research that went into Optane nvdimms would also be used for improving SSDs directly or indirectly.
Also there is the thing that it was never available as a standard DIMM, you needed a special Intel board to use the Optane memory as main memory.
There are presumably standard DIMMs that have DRAM, capacitor, and flash memory all on one DIMM that can persist the DRAM content by copying it to the flash memory on power outage. These lose the appeal of what Optane promised, though. They would be more expensive because they have more components than plain RAM, and also extra more expensive because they are a niche device. The software workaround is more appealing in this case.
I don’t see your concerns reflected in a web search for Optane endurance, its Wikipedia article, or a brief scan across Anandtech headlines. They offer 5 years warranty on products.
I concede it is not a technology I have followed - I only know about it from Liam’s articles and a little web research just discussed - but I think it integrates normal RAM and the 3D Xpoint on the same stick, so with some onboard electrickery might defy your back-of-the-envelope calculations as to how long it lasts. Reviews compare the Optane with SSD, against which it offers magnitudes more endurance and speed.
That is of course Liam’s argument, albeit he reached the conclusion that it was the job of software to embrace this new hardware. I think we can probably disagree on the viability of Optane and still discuss in the abstract how Reform could take advantage of non-volatile memory.
Not at all, it’s all 3D Xpoint. If it weren’t they could not offer one of the main advantages: ~ 4x cheaper per gigabyte, at least as far as large memory configurations are considered.
It cannot because we don’t have any available. If we had we would never have to boot the system, only install it once and be done with it. Of course, that would require a system that is somewhat reliable, and does not crash and corrupt itself on a daily basis.
We seem to be trapped in a cycle where we cannot conceive anything better than what went before, seemingly because executives in the West have become too savvy to take a chance on anything new.
Witness the Boeing 737 Max, a development of an airframe first flown in 1967. We recognise what a bad idea to continue developing that plane, rather than start with a clean sheet, turned out to be. But consider it took to the air less than 5 years earlier than the UNIX whose descendants run the world today - so much for software as a weightless expression of intellect unencumbered by the inertia of physical objects!
As for the hardware Intel’s Optane discussed overlapped the end of their Itanium line. Imagine how different things might have been if they had been magnanimous enough to have shared that ISA with AMD so the latter did not have to kludge x86 into the 64 bit era? Could that ISA be next to become open source in the footsteps of IBM’s Power, Sun SPARC and all?
It is right we should dream a bit rather than rely on streetwise “smarts”, because excessive pragmatism can lead to such “triumphs” as the Boeing 737 Max, stagnant operating system design and x86-64.
And then was gone before it reached a point when you could just pop an Optane stick into your board and have your memory persist.
Admittedly I cannot find an official specification of the durability either which means that it is either no concern at all or a very big concern. Typically when it’s not a concern at all the maker would still proudly present it. And also if it were a viable technology somebody would likely license it from Intel. Either way it’s academic, Optane is no more.
Doubly so given it’s been shown that persistence can be achieved completely in software with the hardware we have today. Of course, the experimental OS that does it runs on next to no hardware, has next to no application, and next to nobody uses it. You can’t just run Photoshop or SAP on it, not even the X server. It had compositor-based windowing system long before Wayland was a thing.
Not sure that IA64 was such a great thing, Intel is not know for good CPU design, it’s known for throwing money at it until it works. It was certainly interesting in some ways. It has this concept of register ring buffer from which registers were allocated for functions, and if the ringbuffer would overflow/underflow it would be spilled to/refilled from stack. There is a problem with register allocation in current day CPUs. What you write in assembly or machine code are virtual registers, you do not get to see the real registers. And while this mechanism does make register mapping available to the user it’s on function granularity, not instruction granularity. So what happens in practice is that compiler writers reverse-engineer how the CPU would allocate real registers and arithmetic units, and try to generate code that makes optimal use of these resources to which they do not have direct access. All in the name of making the CPU look like a Z80 with immediate execution and no pipelining.
There is another problem that is related to faking immediate execution: branching. While Sparc has delayed branching, and all the instructions that are already loaded into the pipeline at the time a branch is resolved get executed most other architectures try to pretend that branch happens at the exact place where it is in the assembly. Which means that they speculatively load and execute some following instructions, and if it turns out that the execution took a different branch they pretend it did not happen - except Spectre and Meltdown happen then. The Sparc approach of making the thing that is really happening visible and put under user control needs to be extended to more aspects of the CPU if we want to have any chance at having usable CPUs in the future.
If it was prone to failure we would, these days, doubtless be hearing its failings loudly broadcast from the rooftops as people took to the web to share their disappointment. I can find little evidence of this.
I presume you are referring to the keyos/coyotos mentioned in a previous post. As you say, it must be super obscure as I did not find many details on this.
Intel is not known for good CPU design because all the good designs have been sunk by the albatross called x86. IA-64 was merely the last, and the rest of your post seems to sing its praises!
I regret that I have not worked with registers and branching to speak from experience. But going from sources such as this paper (I found the metaphor of a carpentry workshop to describe the workings quite helpful) paints a picture of exquisite care going into all parts of this ISA. And these engineers were veterans of both Intel and HP who really knew their stuff, took their time and had a nifty new paradigm - EPIC - to guide their work.
It is also clear that, once on the market, engineers were making great strides at advancing the state of the art of compiling for the architecture, as this paper demonstrates. The authors’ experimental compilers were able to generate binaries that ran one and a half times quicker than gcc and they acknowledged there was still work to do. But of course the amount of resources going into keeping legacy architectures competitive would have dwarfed these efforts.
I would lay the blame at the feet of management. After all the previous attempts at coming up with a replacement had failed, they must have realised by then that x86 had to be killed with fire otherwise it would come back again like it did all those other times. They needed AMD to help with that, but did not reach out and ended up having to use AMD’s 64 bit x86 instead.
Even though it is out of production, I’m not sure it need be too late for IA-64 even now. Open RISC instruction sets are two-a penny these days (SPARC, RISC-V, POWER, something called J-Core, probably MIPS), but there are no open EPIC ISAs out there. The right time to have opened the ISA would have been when it was clear that it was not displacing x86 yet still in production, but the next best time is now.
I remember Optane coming out and was thinking it would change how we write databases…and then nothing happened.
includes an interesting history why Optane never made it.
every optane stick required pairing with a stick of ram.
the server motherboard that supported optane was 7k more than a non-optane board
amd came out with a faster epyc processor at the time
micron closed the factory in 2021 as it was losing 400M / year and Intel didn’t seem to have a viable path to market as their lunch was about to get eaten by AMD
Very promising technology that just never made it.
I suppose nobody who paid premium for their board would want to admit that’s wasted money.
And it’s not that Optane was useless. As far as I know it’s used in some very specific applications as very fast storage, and it pays back its cost there. CXL aims at standardizing attachment of very fast storage, and if it succeeds (which is quite likely at this point) it will obsolete Optane in this and similar applications.
On the other hand I am not aware of any successful use of Optane as RAM replacement the way you envision. Sure, the Optane-enabled boards make it possible to configure the memory in that way. If that’s usable and used is entirely different question.
