Tue, 03 May 2005
Linux kernel.org infrastructure
This interesting article talks about the kernel.org infrastructure used to maintain the Linux kernel. Overall it's a fascinating little bit of history. It's also intriguing because it gives an example of running an extremely bandwidth and processor intensive site. This quote is especially interesting regarding an earlier verision of kernel.org hosted on a dual PIII.
Serving data with http and ftp is is not very CPU intensive, but over time the amount of rsync traffic being fed by the kernel.org server continued to increase, and rsync is CPU intensive. "That's what rsync does" Peter said, "it trades bandwidth for CPU horsepower. We were getting to the point where we had all the bandwidth, but the Dual PIII 1.1's couldn't really keep up." He noted that the load average kept growing, well into triple digits. Referring to 32-bit systems, Peter noted, "we learned that the Linux load average rolls over at 1024. And we actually found this out empirically."
That's fairly amazing. Also noteworthy is the bare number of software optimizations they've thrown at the problem, which basically consisted of mounting their filesystems with the noatime attribute. Have to double check that one on some of my busier http boxes.
Slashdot has an article in their FAQ, detailing their hardware and software mix as well. It probably hasn't been updated in a while but the basic config described probably remains as detailed in the FAQ entry.
| Sun | Mon | Tue | Wed | Thu | Fri | Sat |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | 3 | 4 | 5 | 6 | 7 | 8 |
| 9 | 10 | 11 | 12 | 13 | 14 | 15 |
| 16 | 17 | 18 | 19 | 20 | 21 | 22 |
| 23 | 24 | 25 | 26 | 27 | 28 | 29 |
| 30 | 31 |
About
Nix Bits, Notes about life, *nix, and other stuff.
Scott Harney (GPG key)
<scott_harney@yahoo.com>
Wedding
I got married on 9/4/2004. So click for details, already.
Resume
An online copy of my resume. This resume is suitable for printing
but also contains clickable links providing more detail.
Just mouse over the bullet items.
Photo Album
Lots and lots of family pictures.
Archives
Search
Bookmarks
My del.icio.us bookmarks
Subscribe
Subscribe to a syndicated feed of my weblog,
brought to you by the wonders of RSS or Atom.
Old stuff
Links and writings from the older version of this site
Old stuff