Strictly speaking, the kernel is not a basic component of a computer system. The program can be directly transferred to the computer for execution, indicating that the designer does not want to provide any hardware abstraction and operating system support, which is very common in the design of early computer systems. Finally, some auxiliary programs, such as program loader and debugger, are designed into the machine kernel or solidified in read-only memory. After these changes, the concept of operating system kernel gradually became clear.
The more important question is who should know the kernel. In other words, how the understanding of the kernel will affect a person's work is complicated after all.
[Edit this paragraph] Kernel classification
Single-chip kernel: It provides many perfect hardware abstraction operations for potential hardware.
Microkernel only provides a small part of hardware abstraction, and most functions are completed by a special user-mode program: server.
The structure of hybrid kernel is very similar to that of microkernel, but its components run more in kernel mode to get faster execution speed.
External Kernel This kind of kernel does not provide any hardware abstraction operations, but it allows adding extra runtime to the kernel, through which applications can directly or almost directly operate the hardware.
Integral kernel
The monolithic kernel structure defines a high-level abstract interface in hardware, and uses a set of primitives (or system calls) to realize the functions of the operating system, such as process management, file system, storage management and so on. These functions are completed by multiple modules running in the core state.
Although each module serves these operations separately, the kernel code is highly integrated and difficult to write correctly. Because all modules run in the same kernel space, a small bug will crash the whole system. However, if the development is successful, the monolithic kernel structure will benefit from the running efficiency.
Many modern monolithic kernel structures, such as Linux and FreeBSD kernels, can be loaded and executed by modules at runtime, which can expand the functions of the kernel more easily and make the core part of the kernel more concise.
Example of the overall kernel structure:
1. Traditional UNIX kernel, such as the version released by Berkeley University.
2.Linux kernel
micro kernel
Microkernel structure consists of a very simple hardware abstraction layer and a set of key primitives or system calls, including only a few parts necessary to build a system, such as thread management, address space and inter-process communication.
The goal of micronucleus is to separate the realization of system services from the basic operating rules of the system. For example, the input/output locking service of a process can be provided by a service component running outside the micronucleus. These very modular user-mode servers are used to complete more advanced operations in the operating system, which makes the design of the core part of the kernel simpler. The failure of a service component will not lead to the collapse of the whole system. What the kernel needs to do is to restart this component without affecting other parts.
Microkernel puts many OS services into separate processes, such as file system and device driver, and processes call OS services through message passing. The micronucleus structure must be multithreaded. The first generation microkernel provides more services at the core, so it is called' fat microkernel'. Its typical representative is MACH, which is not only the core of GNU HURD, but also the core of Apple server OS. It can be said that it is thriving. The second generation microkernel only provides the most basic OS.
Examples of microkernels:
1.[ meter] Advanced Interactive Executive program (advanced interactive executive)
2.BeOS
3.L4 micronucleus series
4.Mach, for GNU Hurd and MAC OS X.
5.Minix
6. the big butterfly
7.QNX
8. Radio
9.VSTa
Comparison between monolithic kernel and microkernel
The monolithic kernel structure is a very attractive design, because it is more efficient to control the code complexity of all low-level operations in the same address space than in different address spaces.
In the early 1990s, the monolithic kernel structure was considered obsolete. Designing Linux as a single kernel structure instead of a microkernel has caused a lot of controversy.
Now, the single-core structure tends to be easy to design correctly, so it will develop faster than the micronucleus structure. Both camps have successful cases. Micronucleus is often used in the embedded design of robots and medical equipment, because the key parts of its system are in a separate protected storage space. This is impossible for single-core design, even if the module is loaded at runtime.
Although Mach is a well-known multipurpose microkernel, besides it, several microkernels have been developed. L3 is a demo kernel, just to prove that micronucleus design is not always slow. Its subsequent version L4 can even run the Linux kernel as a process in a separate address space.
QNX is a microkernel system designed since 1980s. It is closer to the concept of micronucleus than Mach. It is used in some special fields, in which system failures caused by software errors are not allowed. For example, the manipulator on the space shuttle and the machine for grinding telescope lenses, a little mistake will lead to thousands of dollars in losses.
Many people think that micronucleus technology is useless because Mach can't solve some problems when he puts forward micronucleus theory. Mach's fans show that this is a very narrow view. Unfortunately, it seems that everyone is beginning to accept this view.
Mixed kernel
The hybrid kernel is actually a microkernel, which only allows some code with microkernel structure running in user space to run in kernel space, making the kernel run more efficiently. This is a compromise, and the designer refers to the theory of poor running speed of the micro-kernel structure system. But later experiments proved that the pure micronucleus system can actually be efficient. Most modern operating systems follow this design category, and Microsoft Windows is a good example. And XNU, a kernel running on Apple's Mac OS X, is also a mixed kernel.
Examples of mixed kernels:
BeOS kernel
Dragonfly BSD
ReactOS kernel
Windows NT, Windows 2000, Windows XP, Windows Server 2003, Windows Vista and other operating systems based on NT technology.
XNU
Some people think that there is no difference between a single-core system that can load modules at runtime and a hybrid kernel system. This is not correct. Mixing means that it absorbs some design patterns of single-core and micronucleus systems, for example, some non-critical codes run in user space, while others run in kernel space, just to improve efficiency.
External kernel
Outer core system, also known as vertical structure operating system, is a more extreme design method.
Its design concept is to let the designer of user program decide the design of hardware interface. The outer kernel itself is very small, and it is usually only responsible for services related to system protection and system resource reuse.
Traditional kernel design (including single core and micronucleus) abstracts the hardware and hides the hardware resources or device drivers under the hardware abstraction layer. For example, in these systems, if a piece of physical storage is allocated, the application does not know its actual location.
The goal of the outer core is to let the application directly request specific physical space, specific disk blocks and so on. The system itself only guarantees that the requested resource is currently idle, and the application allows direct access to it. Because the outer core system only provides the bottom hardware operation, and does not provide advanced hardware abstraction like other systems, it needs to add additional runtime support. These runtimes run on the external kernel and provide complete functions for user programs.
Theoretically, this design can make various operating systems run on an external kernel, such as Windows and Unix. Designers can adjust the functions of each part of the system according to the operating efficiency.
At present, the external core design is still in the research stage, and no commercial system has adopted this design. Several conceptual operating systems are being developed, such as Nemesis of Cambridge University, Citrix of Glasgow University and a system of Swiss Institute of Computer Science. The Massachusetts Institute of Technology is also conducting this kind of research.
[Edit this paragraph] Nuclear-free
Both TUNES Project and UnununiumOS try to have no kernel. Coreless systems are not limited to a single centralized entry point.
[Edit this paragraph] Reference
operating system
[Edit this paragraph] Improvement of kernel development
Red Flag 5 is a 2.6 kernel, so I'm looking forward to it.
Turn: The long-awaited 2.6 kernel has finally arrived. PaulLarson of IBMLinuxTechnologyCenter secretly pays attention to the tools, tests and technologies that make 2.6 the best kernel ever-from correction control and regression testing to defect tracking and list saving.
After three years of active development, the new 2.6Linux kernel was released recently. During this period, some interesting changes have taken place in the development and testing methods of Linux kernel. The current method of developing kernel is no different from that of three years ago in many aspects. However, some key changes have improved the overall stability and quality.
Source code management
Historically, there has never been a formal Linux kernel source code management or version control system. In fact, many developers have implemented their own calibration controllers, but there is no official LinuxCVS file for LinusTorvalds to check the added code and let others get the code from it. The lack of calibration controllers often leads to a "generation gap" between release versions. No one really knows what changes have been added, whether these changes can be well integrated, or what new content is worth looking forward to in the upcoming version. Usually, if more developers can understand these changes and their own changes, some problems can be avoided.
Due to the lack of formal error correction controller and source code management tools, many people propose to use a product called BitKeeper. BitKeeper is a source code control management system, and many kernel developers have successfully applied it to their own kernel development work. Shortly after the release of the original 2.5 kernel, LinusTorvalds began to try BitKeeper to determine whether it could meet his needs. Now the Linux kernel source code of the main 2.4 and 2.5 kernels is managed by BitKeeper. For most users who may care little or nothing about kernel development, this seems irrelevant. However, in some cases, users can benefit from the change in the method of developing Linux kernel brought by using BitKeeper.
One of the biggest benefits of using BitKeeper is the fusion of patches. When multiple patches are applied to the same basic code, some of which will affect the same part, there may be fusion problems. A good source code management system can automatically complete some complicated work, so that patches can be merged more quickly and more patches can be added to the kernel. With the expansion of the Linux kernel developer community, it is necessary to modify the controller to help track all changes. Since everyone can integrate these changes into the main Linux kernel, tools such as BitKeeper are necessary to ensure that patches are not forgotten and can be easily integrated and managed.
It is necessary to use real-time and centralized archiving to save the latest updates of Linux kernel. Every change or patch accepted by the kernel is tracked as a change set. End users and developers can save their own source files and update them with the latest change set with a simple command as needed. For developers, this means that you can always use the latest code copy. Testers can use these logical change sets to determine which changes caused the problem and shorten the time required for debugging. Even those users who want to use the latest kernel can directly use real-time and centralized archiving, because now they can update the components or defect fixes they need as soon as they are added to the kernel. When the code is integrated into the kernel, any user can provide immediate feedback and defect report about the code.
Parallel development
With the growth of Linux kernel, it has become more complex and attracted more developers to focus on the special development of specific aspects of the kernel, and another interesting change has taken place in the method of developing Linux. During the development of the 2.3 kernel version, besides the main kernel tree published by LinusTorvalds, there are some other kernel trees.
During the development of 2.5, the kernel tree exploded. Because the synchronization and parallelism of development can be maintained by using source code management tools, partial parallelism of development can be realized. In order for others to test their changes before accepting them, some development needs to be parallelized. Kernel maintainers who maintain their own trees are dedicated to specific components and goals, such as memory management, NUMA components, code to improve scalability and specific architecture. Some trees collect and track the correction of many small defects.
Figure 1. Linux2.5 development tree
The advantage of this parallel development mode is that it allows developers who need to make major changes, or developers who make a lot of similar changes for a specific goal, to develop freely in a controlled environment without affecting the stability of the kernel used by others. When developers have finished their work, they can release patches for the current version of the Linux kernel to realize the changes they have made so far. In this way, testers in the community can easily test these changes and provide feedback. When each part is proved to be stable, these parts can be integrated into the main Linux kernel separately, or even simultaneously.
Testing in practical application
In the past, Linux kernel testing methods revolved around the open source development model. Since the code is open to other developers for review once it is released, there has never been a formal verification cycle similar to other forms of software development. The theoretical basis behind this method is the so-called "Linus rule" in "The Land and Market of the Cathedral" (please refer to the Resources for relevant references). The content of this rule is "Everyone's eyes are sharp". In other words, intensive review can find out most of the real big problems.
However, in fact, the kernel has many complex interconnections. Even if the review is strong enough, many serious defects will still be missed. In addition, once the latest kernel is released, end users can (and often) download and use it. When 2.4.0 was released, many people in the community proposed more organized testing to ensure the intensity of specific testing and code review. Organized testing includes the use of test plans, repeatability in the testing process, and so on. Using these three methods will bring higher code quality than using only two methods at first.
Linux test project
The earliest contributor to Linux organized testing is LinuxTestProject (LTP). The purpose of this project is to improve the quality of Linux through more organized testing methods. Part of this test project is the development of automatic test suite. The main test suite developed by LTP is also called Linux test project. When the 2.4.0 kernel was released, the LTP test suite had only about 100 tests. With the development and maturity of Linux and version 2.5, the LTP test suite is also developing and maturing. At present, Linux test projects include more than 2000 tests, and this number is still growing!
Code coverage analysis
The new tools used now provide the kernel with the function of code coverage analysis. Coverage analysis tells us which lines of code in the kernel are executed during a given test run. More importantly, coverage analysis reveals which parts of the kernel are not tested at all. This data is important because it points out what new tests need to be written to test these parts of the kernel in order to test the kernel more completely.
Multi-day nuclear regression test
In the development cycle of 2.5, another project adopted by Linux test project is to use LTP test suite to perform multi-day regression test on Linux kernel. People use BitKeeper to create a real-time, centralized archive so as to get a snapshot of the Linux kernel at any time. When BitKeeper and snapshot are not used, testers must wait until the kernel is released before starting testing. Now, as long as the kernel changes, testers can test it.
Another advantage of using automated tools to perform regression tests that last for many days is that there is little change compared with the last test. If a new regression defect is found, it is usually easy to detect which change caused the defect.
Similarly, because it is the latest change, it is still impressive in the minds of developers-I hope it will make it easier for them to remember and modify the corresponding code. Perhaps Linus's rules should draw the conclusion that some defects are easier to find than others, because those defects have been found and dealt with by kernel regression testing for many days. These tests can be carried out every day before the development cycle and the actual release, which enables testers who only focus on the full release version to pay attention to more serious and time-consuming defects.
Extensible test platform
Another team is called OpenSourceDevelopmentLabs, and OSDL (OSDL) has also made important contributions to Linux testing. 2.4 Shortly after the release of the kernel, OSDL created a system called ScalableTestPlatform (STP). STP is an automated test platform, which allows developers and testers to run tests provided by systems based on OSDL hardware. Developers can even use this system to test their own kernel patches. Extensible test platform simplifies the steps of testing, because STP can build the kernel, set up tests, run tests and collect results. Then the results are compared in depth. Many people have no access to large-scale systems, such as SMP machines with 8 processors. Through STP, anyone can run tests on such large-scale systems, which is another advantage of this system (STP).
Micro-defect
Since the release of 2.4, one of the biggest improvements in organized testing of Linux kernel is defect tracking. In the past, defects found in the Linux kernel were reported to the Linux kernel mailing list, the mailing list of a specific component or system, or directly to the individual who maintained the part of the code where defects were found. With more and more people developing and testing Linux, the shortcomings of this system are quickly exposed. In the past, defects were often ignored, forgotten or ignored unless people's reports on defects were surprisingly maintained.
Now, OSDL has installed a bug tracking system (see Resources for a link) to report and track bugs in the Linux kernel. The system is configured such that when a defect of a component is reported, the maintainer of the component will be notified. The maintainer can either accept and fix the defect, redistribute it (if it is finally determined that it is actually a defect in another part of the kernel), or eliminate it (if it is finally determined that it is not a real defect, such as a misconfigured system). Defects reported to the mailing list are still in danger of being lost, because more and more emails are pouring into the list. However, in the defect tracking system, there is always a record of each defect and its current state.
A lot of information
In the process of opening up for the future 2.6Linux kernel, in addition to these automated information management methods, different members of the open source community also collected and tracked an amazing amount of information.
For example, a status list is created on the KernelNewbies site to track the new kernel components that have been proposed. This list contains items sorted by status. If they have been completed, it indicates which kernel they have been included in, and if not, it indicates how long it will take. Many items in the list have links to large project websites, or when the items are small, these links point to copies of e-mail messages explaining the corresponding parts.
Kernel version history
Up to now, many of us are familiar with the version numbering system of Linux kernel, but AndriesBrouwer reminds us how irregular it really is.
The first public version of Linux is version 0.02 of 199 1+0. Two months later, in February of 199 1, Linus released version 0. 1 1, which is the first independent kernel that can be used independently of Minix.
One month after the release of version 0. 12, in March, the version number jumped to 0.95, reflecting that the system is becoming more and more mature. Moreover, the landmark 1.0.0 was not completed until two years later, that is,1March 994.
At about this time, we began to use the method of bidirectional numbering to mark the development of the kernel. Even numbered cores (such as 1.0, 2.2, 2.4 and now 2.6) are stable "product" models. Meanwhile, the odd kernel version (1. 1, 2.3) is a frontier or "developing" kernel. Recently, a few months after the release of a stable kernel, the development of a new kernel began. However, the development of 2.5 did not begin until several months after the completion of 2.4.
So when can we expect 2.7? It's hard to say, but there are already clues to discuss in KernelTrap.
Before that, you can read RagibHasan's article to learn more about the history of Linux.
At the same time, the Post-Halloween Document tells users what to expect from the upcoming 2.6 kernel (see Resources for a link). Most of the discussions in the post-Halloween documents are about the main changes that users need to pay attention to and the system tools that need to be updated (for use). People who care about this information are mainly Linux publishers and end users who want to know what's in the 2.6 kernel in advance, so that they can determine whether any programs need to be upgraded to take advantage of new components.
The KernelJanitors project keeps (and actually still keeps) a list of small defects and solutions that need to be fixed. Most of the solutions to these defects are caused by the need to modify many parts of the code when patching the kernel, for example, some places will affect the device driver. People who are just engaged in kernel development can choose the items in the list from the beginning, so that they can learn how to write kernel code through small projects and have the opportunity to contribute to the community.
In addition, in another pre-release project, JohnCherry tracked the errors and warnings found when compiling each released kernel version. As time goes by, these compiled statistics have been declining, and publishing these results in a systematic way makes the progress clear. In many cases, you can use these warning and error messages just like using the KernelJanitors list, because compilation errors are usually caused by small defects and need some efforts to fix.
Finally, there is AndrewMorton's "compulsory" list. Since he has been selected as the maintainer after the release of the 2.6 kernel, before the final release of the 2.6 kernel, he used his privilege to outline the most urgent problems he thought needed to be solved. The list that must be fixed contains the defects of the kernel Bugzilla system, the components that need to be completed, and other known problems, which will hinder the release of 2.6 if not solved. This information can help indicate what steps are needed before the new kernel is released; It can also provide valuable information for those who care about when the much-anticipated 2.6 kernel release will be completed.
Since the release of the 2.6 kernel at the end of last year, some of these materials have obviously not been maintained. Other related work has not been completed after the release of the main version, and the later update will continue. Interestingly, what have we seen and what innovations have we made? We are close to a major release.
Concluding remarks
When most people consider a new stable version of the kernel, the first question is usually "Is there anything new in this version?" In fact, in addition to some new features and fixes, there is a process of continuous improvement over time behind the scenes.
In the Linux community, open source development is booming. The connection between the coders working in Linux kernel and other aspects is loose, which makes the team adapt to the change successfully. In many aspects, compared with many individual improvements and bug fixes that have been completed, the development and testing methods of Linux-especially these methods have been improved over time-have a far-reaching impact on the reliability of the new kernel.
[Edit this paragraph] The core in the astronomical sense
Astronomically speaking, a planet (or star) has two cores: an inner core and an outer core, and the temperature of the inner core of a star can reach several thousand degrees Celsius.