Professor, Faculty of Environmental Information, Keio University
Interviewer: Yoshihiro Nakajima, Editor-in-Chief, IPv6Style
The objectives of IPv6 protocol stack development
It is said that the Internet has its roots in two events that originally started in 1969. One is that UNIX development started; the other is that the ARPANET experiment started. However, amazingly, these two technologies did not come to meet during the 1970s.
In the 1980s, people finally started thinking about how these two, UNIX and ARPANET, could be connected. Ethernet, which was proposed in a paper in 1976, played a major role in this relationship. Up until then, ARPANET was just a remote access over packet switched network and was used to access mainframes remotely. However, once Ethernet emerged at the beginning of the 1980s and the idea of local area networks was born, the combination of remote access networks with the slightly faster (though much faster at the time) local area networks to perform distributed processing became the focus of research.
As a result, people discussed the necessity of rethinking the operating system’s architecture. This led to a movement to rebuild the UNIX 4.1 BSD, which was used in DEC’s VAX, as a “network operating system”.
In 1982, when I was a doctoral student at Keio University, we developed our network, S&T NET, and was conducting experiments. At that time, Ethernet had not yet been standardized. University of California at Berkeley (hereafter UCB), was also conducting similar experiments for 4.2 BSD. In other words, operating system (OS) researchers started focusing on networks.
This 4.2 BSD project was a research project that was done by UCB’s CSRG (Computer Science Research Group) around the beginning of the 1980s. I was researching together with people at the CSRG, primarily having discussions about redesigning to a accomodate a protocol stock. 4.1 BSD only added UNIX support for VAX’s virtual memory. However, for 4.2 BSD, for the purpose of network, basic components such as process scheduling and memory management were redesigned, which was a fundamental change for the OS. The reason for these changes was because the CPU had to process not only user processes, but also protocols coming in from the network interface at the same time, and, moreover, it had to do so efficiently, since it had to support networks.
The memory management changes were also done due to protocol layering. Up to that point, data was copied in memory each time the data was passed from a driver to an IP layer, the IP layer to a TCP, and from the TCP layer to a program. However, since copying the memory takes a long time, I suggested not to use that method.
So, using the hardware’s memory management, and the header change would be done logically, keeping the network data in the same place in memory. Unless you make it like this so that memory copying to be as minimum, the protocol process becomes slow. With this, the protocol stack started running fast. This solves made something that had been considered difficult and I felt proud as an engineer about it.
After these OS architecture changes were made, the members who were working on the TCP/IP protocol design implemented the protocol. In other words, the TCP/IP protocol stack was added into 4.2 BSD.
The Internet deployed because of open source
The outcome of these developments was distributed as source code-based software. When reading the protocol specifications, software developers would have difficulty in understanding what exactly they mean. However, if they take a look at the source code implementing the protocol, they will immediately know what it means. A packet arrives. You look at its address, and if it is not for you, all you need to do is to send it to another person. You can also immediately figure out that how it works is very simple. Then, improvements start happening rapidly. In other words, you learn about TCP/IP while looking at the source code and then extend it.
Over the course of the spread of the Internet, various simple accidents and incidents happened. For example, there was a time when networks just about everywhere stopped working. This was because no one had actually implemented ICMP correctly in their products, even though they listed ICMP in their products’ features. This problem was exposed when new applicationsthat expected those products to have implemented ICMP, such as traceroute, appeared. In other words, since vendors develop various products after studying the BSD source code, if there are bugs in BSD itself, other products developed based on BSD are affected and all kinds of computers worldwide will fail one after another for the same reason. In short, it was proof that the influence of BSD’s all the source code is tremendous.
When we adopted the new concept of subnet, we fixed BSD. Since we showed it to vendors around the world and asked them to fix each implementation, a new concept was spread. In other words, if there had been no source code that everyone could reference, the protocol would not have spread. When you think this way, you can say that the Internet spread because there was BSD source code.
Shortly before the IPv6 specs were decided
In the 1990s, the idea that computers had to be connected to each other became increasingly widespread, and an international standard, OSI, emerged. In government procurement, the US adopted OSI, and, also in Japan, the cabinet approval was given twice for OSI adoption. I also heard that Sun Microsystems even built a separate factory to implement OSI. Once the decision is made that it is an “international standard”, you have to follow it at all times and places. In 1991, an organization that promotes the Internet, called IAB, decided to change IP to CLNP, which is the OSI’s IP.
However, when engineers around the world heard the news, they got angry, asking why the protocol had to be changed to one that was not certain to work and did not have any track record. This led to a big revolution later. That revolution happened in Kobe in 1992. It was when the WIDE Project was hosting the first international conference for Internet society, INET 92, in Kobe. Engineers opposed to the idea, and the revolution happened. It led to a reorganization of the IAB, a decision-making organization for engineers, as well. Up until then, the IAB, known as the Internet Activities Board then, was a group of people who played major roles in Internet activities; in other words, it was a group of patriarchs, such as Vinton Cerf, who created the Internet.
The IAB has to oversee the overall compatibility. However, as for what will be used for each protocol, proposals would be submitted after being evaluated from a purely engineering viewpoint within the organization called the IETF, and good ones that people have confidence in and that is useful will be adopted. Then, a group that ensures that there is no discrepancy between the society’s demands for the Internet and the architecture should be separated. These matters were discussed at the INET 92. As a result, decisions on protocols were to be made at the IETF and the IAB’s name was changed to Internet Architecture Board. Moreover, there was an opinion that it was not good to have matters decided primarily by the US, and I was chosen as an IAB member. With that, everything went back to where we started and we took back the decision to go with OSI.
Certainly, there would be a shortage of IP addresses in the next generation. Furthermore, there were new demands such as mobility and security. So, we decided to hurriedly work on the next generation Internet protocol called IPNG (Internet Protocol Next Generation), and the IAB told the IETF to make suggestions if they had good ideas on IPNG. Of course, this includes the OSI idea that was originally suggested. Among the suggestions, we narrowed them down to two as a result. One was SIP (simple-IP), which had a simpler IP header but a longer address; the other was a suggestion that used OSI’s CLNP. The OSI suggestion was a very well made one called TUBA, which was created by talented engineers, including Peter Ford, at Los Alamos research lab.
SIP uses a fixed-length address. CLNP uses a variable-length address. The difference between a fixed-length address and a variable-length address is that the variable-length address has the length information at the beginning of the address. If you look at the information, you will know up to how many bits you need to read the address. You can also make the address length bigger using this.
However, there is a difference of 2-3 steps, in terms of program code, between retrieving the address after checking the length and retrieving the pre-determined bit length. We made our decision by considering the gap that would be created among programmers around the world by requiring them to figure out these 2-3 steps.
How many programmers out there who know how to process fixed-length addresses but not variable-length addresses? We may possibly lose many programmers who can write software for the Internet because of this just 2-3-step difference. This was the tiny but important factor in deciding on the two types of addresses. The criteria we used to make our decision was the idea that more and more people should participate to create software for the future of the Internet.
Can IPv6 reference code be created?
One other thing on our minds was that when IPv6 deployed, who was going to play the role that UCB played by implementing the V4 TCP/IP protocol stack in the old BSD. UCB’s source code was written by all of us working together and UCB widely distributed the outcome. Of course this source code was of the highest quality of TCP/IP. That is to say, the community’s ability was concentrated into BSD.
We had to be able to do the same for IPv6, too. So, we looked around but the CSRG did not exist at UCB any longer. The Internet bubble had already started in the US. Rick Adams had founded UUNET. Some members of the CSRG had founded BSDI. Bill Joy was at Sun Microsystems. All the dependable people I knew had started businesses. I felt like “Oh? Am I the only one in a university?”
So, I felt that I had to take the lead and do it. I gathered the WIDE Project members and gave the command, saying “We will implement the IPv6 protocol stack for BSD ourselves!” First, we created three groups in order to develop three totally different implementations under the rule that we were not to look at and discuss each other’s code. The goal was to try and see if they could really be connected. Once we confirmed that they could be connected, we discarded the entire source code temporarily.
Next, we asked the companies and organizations that the members, who stayed until the end, worked for saying, “Please lend them to us for the WIDE Project for two years,” although it ended up lasting eight years. These top gun engineers were cooped up in an office and were not allowed to do anything else. They started creating the new, strongest, IPv6 protocol stack from scratch using their past experience. This was the KAME Project’s original form. There are various stories that explain the name of KAME. My official explanation is that the office that we developed in happened to be in Karigome, where SFC is at, and we took the first and the last sounds of that name.
There were two other IPv6 protocol stack implementations in other countries, so there were three in total. When the three groups competed technically, people thought that the KAME project should be the united result. So we succeeded in integrating them.
After that, various companies asked us to let them use the protocol stack that the KAME Project developed; however, it was when Apple told us that they wanted to use it for Mac OS X that we started clearly feeling that our objectives were achieved. Also, in China, all the universities use BSD and they all use the protocol stack that the KAME Project developed. Considering all these things, I feel that our project played the same role that UCB’s CSRG’s TCP/IP code once played in propagating the Internet all over the world.
The technical role of the KAME Project
The KAME Project’s goal at the beginning was that the source code we write would help engineers deepen their understanding of the protocol while looking at the source code. We wanted that reference implementation. The KAME Project members have verified the protocol that was mere theory up until then by implementing the protocol stack on their own, and then they fed the results back to the standardization efforts.
So, since the reference implementation was done while standardization was in progress, it is fair to say that there are no technical concerns for IPv6 propagation. What remains are market concerns. For example, the security business. Or, how to transition to IPv6. When both IPv4 and IPv6 exist, how can we keep costs low while using both and migrating to IPv6? These areas are referred to as transition engineering; it is not protocol development any longer. So, it is important to make the KAME Project’s role and outcome clear now.
The world’s highest quality reference code is now available to anyone in the world. Our goal was to remove the obstacles that people around the world face when they study IPv6, adopt its technology, use it in businesses, etc. The KAME Project has achieved that goal splendidly.
The outcome of the KAME Project
Since each company has been letting us use their talented engineers as KAME Project members, there is a time limit. Of course, if there are some people who want to continue the project, I would like them to do so, and the WIDE Project will accept it as well. Now, I would like to express my deepest appreciation to those companies and organizations that let their talented engineers concentrate on this project. I should not be the only one who appreciates those many people who supported this project; I think the entire Internet community should appreciate them.
Also, I think the engineers who worked on this development were able to gain great confidence. I think they can say with confidence that the IPv6 protocol stack that is used around the world, including on Mac, is the result of their development efforts. I hope they will keep that in mind and be proud about it for the rest of their lives.
The protocol works in UNIX. People can connect to the Internet worldwide. Since I as well as other international people did roles in the development, it feels a little strange when people say, “The Internet is US-made and it is something that you imported, right?” I did not want the KAME Project members and other next generation engineers to experience the same strange feeling.
However, since we had such great results, I don’t think people will ever say, “IPv6 is foreign-made, isn’t it?” This is something that engineers around the world know. That was the most important thing to me.