Kernel Summit: Customer Panel

Thursday's first kernel summit session was a customer panel, organized by James Bottomley and yours truly. The presenters were Sean Kamath from Dreamworks, Head Bubba from Credit Suisse, and Markus Rex, the new CTO for The Linux Foundation. Sean covered the basics of their configuration and workloads, then pointed out some of the key problems that they see in their IT environment around Linux. These problems centered around memory management and a recent shift in semantics regarding RO/RW mounts of local and NFS filesystems. Some of the memory management problems were reportedly fixed (or at least possibly fixed) since the latest enterprise distribution that most customer are using. However, a number of the questions centered around information that was very difficult to for a corporate IT department to determine definitively and, with some reticence, the group present recognized that as a possible problem. Specifically, understanding what capabilities are available for handling out of memory conditions, how to limit the resident set of a process (or group of processes), and how to determine how much memory is being used by a process (or group of processes) were considered difficult for end users to determine. Further, some of these behaviors are different between versions of mainline and the various distro releases.

Head Bubba began with an intro to Credit Suisse's business and IT environment. He then followed with an explanation of some of the key problems that he sees in his environment. The first observation was that nearly every change to the CPU scheduler has an impact on the behavior of many applications. While those impacts can be easily contained when an application writer knows they are coming, but often those changes are made either silently, or only with notification to LKML, which isn't really sufficient for end users (complementing lots of discussion about the signal to noise ratio on LKML is leading to many fewer readers of LKML lately).

Head Bubba's next mentioned the value of Linux Real Time which is currently in development, and some measurements based on his workload and the benefits seen by using RT. One large area of pain is general access to Linux diagnostic tools. SystemTap was mentioned in particular as was utrace and PAPI. Bubba then reviewed some problems with jitter in TCP/IP. The last area where Bubba focused was iWarp and OpenFabrics and the performance, and, more importantly, the latency of packets received in these high bandwidth interconnect technologies. There are some interesting charts in the slides that describe these problems.

The last speaker was Markus Rex, covering the consolidated list of priorities from the Linux Foundations User and Vendor Advisory Councils. The list started with a high level set of areas that are of interest to these groups, including such obvious things as virtualization, power management, scalability, IPv6 readiness, etc. Markus then drilled into some of the specifics in each area, although the attendees would have preferred to see more specifics in some areas, such as scalability. Within virtualization, for instance, it was most important to be able to run any OS as a guest on any major distribution. Right now the paravirt_ops is generally moving in that direction, but the major distros are still not quite there. The hope is that a server or workstation could support running a plurality of distributions as guests simultaneously. Today we aren't quite there with out of the box distributions, although that will likely be present in the next versions of the enterprise distros and most concurrent distros in that time frame. And, with all that flexibliity, end users are going to want to have a single, standardized management interface to all that capability, e.g. CIM interfaces and a unified management application.

Other requested features include power management capabilities for servers, and increased support for device drivers, in conjunction with the community plans for enabling open source device drivers.

The other significant request was for full IPv6 compliance. The US DoD has recently increased the requirements in the IPv6 arena for RFP's and RFQ's (Request for Pricing/Quotes) to include as mandatory many more IPv6 related standards, many of which are not yet implemented by any operating system, including Linux. As a result, there is a full court press to identify the gaps, distribute the work, and achieve compliance in time for the DoD's required dates.

The remainder of the requirements will be able to be found in the slides from the three presenters, which I'll have a pointer to as soon as they are uploaded, probably onto the Linux Foundation site.