IPv6 Support in FocusShare, an IPv6 Multicast Audio/Video Communication Tool

Noritaka Osawa (Media Education Development Center)
Norio Takase, Fiatlux


Introduction

We have been developing educational research assist systems using IP technology, based on satellite communication network. FocusShare was developed as a prototype for educational research assistance system, utilizing the reach and multicasting capability of satellite communications.

FocusShare won the Implementation Category Grand Prix at IPv6 Appli Contest 2003. This is a sort of multipoint video conference tool sending video, audio and "eye focus area" information over IPv6 multicast. A part of video image can be designated as eye focus area, which can be given a linear or non-linear zooming. An image with different quality can also be overlaid in this area, while mouse pointers can be shared.

This article explains how FocusShare was developed with IPv6 support.


Design

Media Education Development Center (NIME) develops and maintains Space Collaboration System (SCS), a remote video conference system over satellite, which consists of about 150 stations, mainly universities and research institutions. Satellite communication is uni-lateral, but SCS utilizes plural satellite communication paths, so that video transmission can be switched among participating stations. Participating stations can receive all transmitted video streams.

FocusShare is designed for the use in SCS. We had already developed and used a filter for Microsoft DirectShow for image adjustment purpose. FocusShare project focused on developing additional filters and user interface.

In the design process, we made sure that we clearly separated user interface from communication module. We also attempted to ease the implementation of remote control feature by sending user interface events over multicast, so that receivers of these events could process them.

FocusShare had to be able to conduct basic transmission of video and audio over unilateral communication path, for use in SCS. Therefore, we could not base this software on TCP. We chose to use UDP transmission. We also decided to use multicast, in order to utilize the reach and multicasting capability of satellite communications. General multipoint video conferencing systems require Multi Control Units (MCUs) or servers, but our system made them unnecessary by using multicast. Incidentally, FocusShare can be used over any multicast-capable networks in addition to satellite networks.

FocusShare has to send the following information over multicast, in addition to video and audio.

1. Video data (compressed/uncompressed)
   
2. Audio data (compressed/uncompressed)
   
3. Mouse pointer information
   
4. Focus display information
   
5. Other control information

We enabled use of various codecs for participants to choose for various purposes. Transmitted video, for example, needs to support multiple resolutions and formats.

Therefore, we developed a general-purpose filter for sending data on various formats over IPv4/IPv6 multicast. This filter conducts the following functions.


Processing at Transmission Filter

• A process to integrate all required data to one data unit
  
• A process to divide the data units to packets, and add transmission speed adjustment and duplicate transmission control
  
• A process to transmit data in packets.

Figure 1 shows transmission process flow.


Figure 1 Transmission process flow

Processing at Receiving Filter

• A process to receive data in packets
  
• A process to reassemble data unit from multiple packets
  
• A process to get necessary data from various data units

Implementation of Tx/Rx Filters

Transmission and Receiving filters are developed so that they don't rely on address family. That was because we wanted the software to support IPv4 and IPv6 dual stack environment, while we wanted to reduce code rewrite for supporting future protocols. The latest Microsoft Windows socket API has non-address dependent functions such as getaddrinfo(). We decided to utilize them.

The first Tx/Rx filters we developed only supported IPv4 multicast. We made this IPv6-enabled, but the only necessary job was in packet-based transmission and receiving of packets.

The following are the changes we made from IPv4 version. Emphasized parts in the code are address-family dependent.

1. Changes in Address Structure

SOCKADDR_IN _sockAddrIn;

was replaced by:

SOCKADDR_STORAGE _sockAddr;

SOCKADDR_STORAGE structure has enough length to support IPv4 and IPv6 address.

2. Changes in Socket Creation

In creating sockets, we avoided using AF_INET and other address-family dependent macros directly.

[Before IPv6 Support] // Create Socket _socket = WSASocket( AF_INET, SOCK_DGRAM, 0, NULL, 0, WSA_FLAG_MULTIPOINT_C_LEAF | WSA_FLAG_MULTIPOINT_D_LEAF | WSA_FLAG_OVERLAPPED);

Instead, we used getaddrinfo() to get address information. getaddrinfo() automatically judges address family from given host names, and returns appropriate address information.

[After IPv6 Support] // Enumeration Address info addrinfo hints, *res; memset(&hints, 0, sizeof(addrinfo)); hints.ai_family = AF_UNSPEC; hints.ai_socktype = SOCK_DGRAM; hints.ai_flags = AI_NUMERICHOST; sprintf(str, "%d", port); ret = getaddrinfo(address.c_str(), str, &hints, &res); if (ret != 0 || sizeof(_sockAddr) < res->ai_addrlen) return false; _sockAddrLen = res->ai_addrlen; _destSockAddrLen = res->ai_addrlen; memcpy(&_sockAddr, res->ai_addr, _sockAddrLen); freeaddrinfo(res); // Create Socket _socket = WSASocket( _sockAddr.ss_family, SOCK_DGRAM, 0, NULL, 0, WSA_FLAG_MULTIPOINT_C_LEAF | WSA_FLAG_MULTIPOINT_D_LEAF | WSA_FLAG_OVERLAPPED);

3. Changes in Bind Expression [Before IPv6 Support] // Bind Socket SOCKADDR_IN ownSockAddrIn; memset(&ownSockAddrIn, 0, sizeof(SOCKADDR_IN)); ownSockAddrIn.sin_family = AF_INET; ownSockAddrIn.sin_port = htons(port); ownSockAddrIn.sin_addr.S_un.S_addr = htonl(INADDR_ANY); ret = bind(_socket, (SOCKADDR *)&ownSockAddrIn, sizeof(SOCKADDR_IN)); if (ret == SOCKET_ERROR) return false;

AI_PASSIVE is specified for ai_flags in addrinfo hints, and NULL is specified as host name given to getaddrinfo() function. This way, wild card address (INADDR_ANY in IPv4 and IN6ADDR_ANY_INIT in IPv6) is automatically specified.

[After IPv6 Support] // Bind Socket SOCKADDR_STORAGE ownSockAddr; memset(&hints, 0, sizeof(addrinfo)); hints.ai_family = _sockAddr.ss_family; hints.ai_socktype = SOCK_DGRAM; hints.ai_flags = AI_PASSIVE; sprintf(str, "%d", port); ret = getaddrinfo(NULL, str, &hints, &res); if (ret != 0 || sizeof(ownSockAddr) < res->ai_addrlen) { closesocket(_socket); return false; } assert(_sockAddrLen == res->ai_addrlen); memcpy(&ownSockAddr, res->ai_addr, _sockAddrLen); freeaddrinfo(res); ret = bind(_socket, (const sockaddr *)&ownSockAddr,ÅÀ_sockAddrLen); if (ret == SOCKET_ERROR) { closesocket(_socket); return false; }

We can get rid of all address-family dependency to support IPv6 (coexistence with IPv4). The work required for IPv6 support was completed in a short time, partly because we had been careful in separating socket communications and other communications. In actual implementation, we referenced IPv6 Network Programming (Junichiro Hagino, ASCII), a book on IPv6 programming published in Japan. Using FocusShare In March 2004, we conducted remote tutorial, connecting a farm at Chiba University and a studio at NIME, using car-mounted SCS. We experienced no trouble. We confirmed that we could send high-resolution image of microscope in low-compression rate. We were also successful in conducting simultaneous transmission from stereo camera, using lower frame rate (transmission rate) par camera image. Figure 2 through 4 are photos taken at the trial. Figure 2 Microscope camera at the farm Figure 3 Stereo image transmission (notebook PC) Figure 4 Stereo image receipt at the studio Conclusion In applying for IPv6 Appli Contest, we added a feature to assign multicast IP address for eye focus area sharing as well as support for more input devices such as USB Web cameras and for more codecs. We also developed an application to receive and display multiple videos simultaneously. For details, see our Web If you are interested in this software, please download and try it out. We are still improving the features of FocusShare, while adding tools for online surveys and remote control, in an attempt to make it a better live remote tutorial tool set. Eye focus sharing is a major feature of this software. We will try to introduce more various eye focus technology, while adding 3D video support. We would like to say that IPv6-enab led software is few steps away from IPv4 software. Source code for communication modules and Tx/Rx filters we developed are available on "Open.NIME", a public NIME Educational assist system research site. Please feel free to reference them.