Posted by The Channel 9 Team // Tue, Aug 23, 2005 12:48 PM
MS Researchers Jim Larus and Galen Hunt lead an intriguing project where they‘ve built an OS using managed code. The project is known as Singularity. In their own words:
Singularity is a research project focused on the construction of dependable systems through innovation in the areas of systems, languages, and tools. We are building a research operating system prototype (called Singularity), extending programming languages, and developing new techniques and tools for specifying and verifying program behavior.
Besides Singularity‘s kernel being successfully written in C# (how cool is that!), there are all kinds of interesting lessons learned with respect to what a managed OS enables. Again, this is a prototype research OS, not a full fledged OS that can run the typical applications you‘ve come to expect of an OS (or even provide a user interface beyond, say, that of DOS).
Enjoy.
Download Size: 168 MB
Tags: CSharp, MS+Research
| Video Length: 00:53:32 |
  |
Replies: 151 // Views: 78,784 |
 |
|
 |
Beer28
custom linux solutions&software |
| |
 |
Beer28
custom linux solutions&software |
| |
no shared code loading into the process?
Sounds monolithic.
It sounds as if there‘s 1 process spawned by the kernel as usual, but that everything else is running in that 1 process context relying on the fact that there is no malloc or virtualalloc api to try to aquire memory outside it‘s own memory pages and no pointer to point to the memory nor to read or write to it, which it ultimately doesn‘t control anyway.
Maybe I‘m wrong, but CPU‘s aren‘t build for this type of model. And the newer generation CPU‘s like the cell don‘t seem to be going in that direction at all, but rather a direction that requires the application to control concurrent processes itself between the cores rather than the OS. With flat memory rather than registers and so forth.
Is intel working with this project to create a CPU that will accomodate this type of OS?
EDIT: Charles was also saying "what does this all matter to somebody running application level code"(not an exact quote). From how they described it, in your context of managed application level code, you lose all your PInvoke and Dll capabilities as well as the ability to create dynamic runnable code in your process.
|
|
This isn‘t the CLR. In our world, we compile entire MSIL for the kernel into x86 instructions at installation time. There is no libc at the bottom.
However, we do have around some assembly code. Like a kernel written in C, our C# kernel needs assembly code to handle the lowest part of the interrupt dispatch on the x86. But once the assembly code has finished, it dispatches directly into compiled C# (no C). BTW, there is some C code in the system, primarily for the debugger stub.
|
In Singularity, you can add new code to your application. However, instead of loading it into your own process, you load it into a child process. The OS facilitiates setting up channels between the child and its parent.
While this is still very much a work in progress, the results so far look promising. For example, we have a dynamic web server that uses child processes. Also all of our device drivers run in child processes.
|
 |
Beer28
custom linux solutions&software |
| |
|
galenh wrote:
This isn‘t the CLR. In our world, we compile entire MSIL for the kernel into x86 instructions at installation time. There is no libc at the bottom.
However, we do have around some assembly code. Like a kernel written in C, our C# kernel needs assembly code to handle the lowest part of the interrupt dispatch on the x86. But once the assembly code has finished, it dispatches directly into compiled C# (no C). BTW, there is some C code in the system, primarily for the debugger stub.
|
What are you using to compile the MSIL into x86?
how are you handling hardware interupts, and the interupt vector table and interupt service routines from C# code?
C# code has no way to interact with the bios. You can‘t drop into __asm { }; in C#.
I feel there is part of the story that is missing here.
|
|
Beer28
custom linux solutions&software |
| |
|
galenh wrote:
This isn‘t the CLR. In our world, we compile entire MSIL for the kernel into x86 instructions at installation time. There is no libc at the bottom.
|
what is your MSIL runtime linked to on the lowest level?
Are you saying your MSIL to x86 compiler generates dependancy free x86 code from MSIL that is not statically linked to any library be it kernel syscalls hardcoded as a library or compiled-in module or whatever?
Does it compile libc and the kernel functions as a static library?
EDIT: last question, where can we download the src tarball and the MSIL to x86 compiler you‘re using?
|
|
Manip
Christmas, A holiday that brings everyone together |
| |
I would kill (or pay) to get my hands on the source code. I am having such trouble learning about OS concepts using MINUX, being able to look at these ideas in C# (or original ideas) would be wonderful. But frankly that isn‘t going to happen so I‘ll just move on.
This type of system has some great potential in areas where you need to be able to rely on the system to be secure and stable. It might not be a speed demon but it is what I would like running on an ATM machine I am using or on the hospital monitor equipment.
|
|
Beer28
custom linux solutions&software |
| |
|
galenh wrote:
This isn‘t the CLR. In our world, we compile entire MSIL for the kernel into x86 instructions at installation time. There is no libc at the bottom.
|
Here‘s what I just can‘t compute.
You have a compiler that does MSIL to x86 ops right? Entirely possible.
But to not have a libc would mean that everything in the MSIL is compiled strait to the ops with no underlying shared routines in libc?
That means that console.writeline() no longer calls the function in the CLR that calls the C printf or read() or write() to the file # descriptor for stdin or out which calls the syscall,
It would mean they rewrote an implementation of the CLR libraries that call the bios functions directly for io, isr, ivt, time and other bios functions, and that the MSIL compiler that compiles the C# kernel, compiles against only this new CLR runtime library. It would also mean that the bulk of these libraries is not written in C# but rather in C or C++ with asm dropdowns or C bios calls.
And I gather the public will never get access to the compiler nor the new library it links in.
EDIT: wouldn‘t it have just been easier to make a C# native compiler and let C# manipulate memory and pointers and drop to __asm inline??
|
Dang! this sounds good... must watch and then comment.... I‘d love to see this taken further...
an OS Keneral based on code that has been verifiied and has built in to it some aspects of the managed code ala .net could be the next step in making a much more "Bullet Proof" windows OS later.
|
Can some similar system to Singularity be implemented as a device driver in the existing NT kernel? Perhaps the singularity runtime could run all of its code and "lightweight processes" in kernel mode and all calls out of managed code would be simple function calls to normal NT device drivers without changing processes or switching VM spaces.
Also, singularity has no shared memory; does it have shared type instances? Or is the best one can do more like proxy objects? Does singularity even have a JITer in the kernel (perhaps for application and upper level code)?
This project seems great!
|
|
Beer28
custom linux solutions&software |
| |
I just finished it, at the end Charles commented on the OS having a webserver asking if it "parsed html and stuff." (52:10)
A webserver reads the file off the disk(the document portion of the http request header), optionally shoots it to registered functions of interpreters loaded as modules in it‘s proc address, like mod_php or mod_perl, or even mod_mono, or does cgi piping(older style) to an interpreter process, then takes that output and throws it back down the tcp line with send(socket,,);
An http server doesn‘t do any type of document processing on it‘s own, that‘s the browser on the client side that parses it, and sets it up for drawing to the client area. That‘s why the guy came back right away and said "http".
|
|
Beer28 wrote:
comment withdrawn.
|
THis is not good practice, Beer (replacing a post‘s text with "comment withdrawn").
C
|
|
Beer28 wrote:
I just finished it, at the end Charles commented on the OS having a webserver asking if it "parsed html and stuff." (52:10)
A webserver reads the file off the disk(the document portion of the http request header), optionally shoots it to registered functions of interpreters loaded as modules in it‘s proc address, like mod_php or mod_perl, or even mod_mono, or does cgi piping(older style) to an interpreter process, then takes that output and throws it back down the tcp line with send(socket,,);
An http server doesn‘t do any type of document processing on it‘s own, that‘s the browser on the client side that parses it, and sets it up for drawing to the client area. That‘s why the guy came back right away and said "http".
|
Did you listen to the reply to my vague/misleading question, Beer? That should suffice.
C
|
|
Manip
Christmas, A holiday that brings everyone together |
| |
It was just lots of edits as he watched the video, very messy, no loss.
|
|
Beer28
custom linux solutions&software |
| |
I‘ll restate the comment then in abbreviated form; there‘s no way the kernel is completely managed because there‘s nothing to manage the code when you load it into the ram with the bootloader. This was responded to by affirming that the kernel is not managed but rather it‘s a kernel coded using C# and a strong type safe x86 compiler with MSIL as an intermediate step.
I‘m guessing it‘s linked to undisclosed low level C or ASM libraries since it‘s not libc. You can‘t write low level code with C#, it‘s impossible because it won‘t let you break free into the instruction set you need to reach the bios and or service interupts, such as faults, device interupts, or system interupts of any kind.
I think they may have written the actual library that does this in C or ASM, then only done the higher level stuff in C#, using the library to reach what they have to.
That‘s what my first post was ultimately about in not so many words.
The fact that it‘s not compiled to IL, makes me think that this kernel is just coded with C# and a strong typed x86 compiler rather than consider it managed code. They said it has a garbage collector, but it‘s clearly not for the kernel itself if the MSIL is stepped down to native binary by a last stage compiler.
|
|
Beer28
custom linux solutions&software |
| |
|
Charles wrote:
Did you listen to the reply to my vague/misleading question, Beer? That should suffice.
C |
yes, they ported the rotor httpd server to singularity.
|
|
Manip wrote:
It was just lots of edits as he watched the video, very messy, no loss.
|
I don‘t like the practice of removing what you post because you decide you don‘t want it to be there any more. Live with what you say. Have some onions and accept it when you look like a fool or whatever. Hey, I do!
C
|
|
Manip
Christmas, A holiday that brings everyone together |
| |
You know you come across a lot better in person than you do when posting online.
|
|
Beer28
custom linux solutions&software |
| |
I‘ll try to leave more of my messy responses there in the future.
Back to the original topic, when the cpu is running in a process, it needs to be interupted to handle something with pressing data or that needs to be handled, there is a table way down in the low addresses of the ram with addresses of functions that handle these interuptions, some of them for system calls, some of them for faults, some of them for hardware....
At any rate C#‘s very nature doesn‘t allow any of this type of code that would set up the environment for an OS to run and service resource requests, so I‘m sure that they have a very, very special version of C# and the library that runs it and a very special compiler that drops the IL down to x86 or x64.
The next thing I thought of but forgot to post, was that, say you have a linked list of task structs for your kernel scheduler. Wouldn‘t it slow it down alot if that linked list was managed, processes wake and sleep and are interupted thousands of times a second? Even if it was C# and MSIL compiled to x86 or x64, won‘t those extra instructions seriously impeed the kernel‘s function? Because the kernel runs those routines so often, won‘t the extra instructions aplify that?
Or does the MSIL to x86 compiler optimize all of that away?
|
|
Manip wrote:
You know you come across a lot better in person than you do when posting online. |
As I‘ve mentioned, the Charles entity that exists online is human Charles‘ attempt at an AI construct. Funny nobody believe this. I will try and adjust my autopoetic content sensing mechanism to apply more human-like, emotionally balanced contexts to my posted replies.
C
|
|
Manip
Christmas, A holiday that brings everyone together |
| |
hmm ironic you edited the above post, what were you saying about living with what you post again?
|
Beer, did you read Galen‘s reply:
"However, we do have around some assembly code. Like a kernel written in C, our C# kernel needs assembly code to handle the lowest part of the interrupt dispatch on the x86. But once the assembly code has finished, it dispatches directly into compiled C# (no C). BTW, there is some C code in the system, primarily for the debugger stub."
Clear?
C
|
|
Manip wrote:
hmm ironic you edited the above post, what were you saying about living with what you post again? |
I appended content, which is orthogonal to the notion of content subtraction. Note that the original text was decorated with additional text, not changed in meaning or removed completely. There is a difference between editing a post for clarity and removing it to protect your ego.
C
|
|
Manip
Christmas, A holiday that brings everyone together |
| |
That‘s a very fine line you walk there. And appending text can also protect your ego, and change the meaning of it. Case in point:
You jackass.
|
|
