What are Student Cluster Competitions, anyway? A Field Guide
So here’s what all the fuss is about: university students build their own supercomputers – that’s the “cluster” part – for a live face-off to see whose is the fastest. That’s the “competition” part. They’re given real, scientific workloads to run and a power limit they can’t exceed, and the team with the highest-performing system wins.
These are great programs – chock full of technical challenges, both “book learning” and practical learning, and quite a bit of fun too. There are rules and traditions, just as there are in cricket, soccer, and the Air Guitar World Championships. Whether you’ve been following the competitions obsessively (sponsoring office betting pools, making action figures, etc.) or this is the first time you’ve ever heard of them, you probably have some questions… so here’s everything you need to know. Listen up; we’re only explaining this once.
Student Clustering Triple Crown
In 2013, there will be three major world-wide student cluster events. The United States-based Supercomputing Conference (SC) held the first Student Cluster Competition (SCC) in November 2007. The contest has been included at every subsequent SC conference, usually featuring eight university teams from the US, Europe, and Asia. As the first organization to hold a cluster competition, SC pretty much established the template on which the other competitions are based.
The other large HPC conference, the imaginatively named ISC (International Supercomputing Conference), held its first SCC at the June 2012 event in Hamburg. This contest, jointly sponsored by the HPC Advisory Council, attracted teams from the US, home country Germany, and China. It was a big hit with conference organizers and attendees – big enough to justify expanding the field to nine teams in 2013.
The third entry is the newly formed Asia Student Supercomputer Challenge (ASC). The finals of their inaugural SCC will be held this April in Shanghai, China (yeah, the Shanghai in China). At least 30 teams from China, India, South Korea, Russia, and other countries have submitted applications seeking to compete. This competition is sponsored by Inspur and Intel.
How Do These Things Work?
All three organizations use roughly the same process. The first step is to form a team of six undergraduate students (from any discipline) and at least one faculty advisor. Each team submits an application to the event managers in which they answer questions about why they want to participate, their university’s HPC/computer science curriculum, team skills, etc. A few weeks later, the selection committee decides which teams make the cut and which need to wait another year.
The teams who get the nod have several months of work ahead. They’ll need to find a sponsor (usually a hardware vendor) and make sure they have their budgetary bases covered. Sponsors usually provide the latest and greatest gear along with a bit of financial support for travel and other logistical costs. Incidentally, getting a sponsor isn’t all that difficult. Conference organizers (and other well-wishers, like me) can usually help teams and sponsors connect.
The rest of the time prior to the competition is spent designing, configuring, testing, and tuning the clusters. Then the teams take these systems to the event and compete against each another in a live benchmark face-off. The competition takes place in cramped booths right on the tradeshow floor.
All three 2013 events require competitors to run the HPCC benchmark, with an independent HPL (LINPACK) run, plus a set of real-world scientific applications. Teams receive points for system performance (usually “getting the most done” on scientific apps) and, in some cases, the quality/precision of their results. In addition to the benchmarks and app runs, teams are usually interviewed by subject matter experts to gauge how well they understand their systems and the scientific tasks they’ve been running.
The team that amasses the most points is dubbed the “Overall Winner.” There is usually an additional award for the highest LINPACK score and sometimes a “Fan Favorite” award for the team that does the best job of displaying and explaining their work.
While many of the rules and procedures are common between competition hosts, there are some differences:
SC competitions are grueling, 46-hour marathons. The students begin their HPCC and separate LINPACK runs on Monday morning, and the results are due around 5:00 p.m. that day. This usually isn’t very stressful; most teams have run these benchmarks many times and could do it in their sleep. The action really picks up Monday evening when the datasets for the scientific applications are released.
The apps and accompanying datasets are complex enough that it’s pretty much impossible for a team to complete every task. So from Monday evening until their final results are due on Wednesday afternoon, the students are pushing to get as much done as possible. Teams that can efficiently allocate processing resources have a big advantage.
ISC competitions are a set of three day-long sprints. Students run HPCC and LINPACK the afternoon of day one but don’t receive their application datasets until the next morning. On days two and three, they’ll run through the list of workloads for that day and turn in the results later that afternoon.
The datasets usually aren’t so large that they’ll take a huge amount of time to run, meaning that students will have plenty of time to optimize the app to achieve max performance. However, there’s another wrinkle: the organizers spring a daily “surprise” application on the students. The teams don’t know what the app will be, so they can’t prepare for it; this puts a premium on teamwork and general HPC/science knowledge.
Hardware Dreams Spawn Electrical Nightmares
When it comes to hardware, the sky’s the limit. Over the past few years, we’ve seen traditional CPU-only systems supplanted by hybrid CPU+GPU-based clusters. We’ve also seen some ambitious teams experiment with cooling, using liquid immersion cooling for their nodes. Last year at SC12, one team planned to combine liquid immersion with overclocking in an attempt to clean the clocks of their competitors. While their effort was foiled by logistics (their system was trapped in another country), we’re sure to see more creative efforts along these lines.
There’s no limit on how much gear, or what type of hardware, teams can bring to the competition. But there’s a catch: whatever they run can’t consume more than 3,000 watts at whatever volts and amps are customary in that location. In the US, the limit is 26 amps (26*115 volts = 3000 watts.) At the ISC’13 competition in Germany, the limit will be 13 amps (13*230 volts = 3,000 watts.) The same 3,000-watt limit also applies to the upcoming ASC competition in Shanghai.
This is the power limit for their compute nodes, file servers, switches, storage and everything else with the exception of PCs monitoring the system power usage. There aren’t any loopholes to exploit, either – the entire system must remain powered on and operational during the entire three-day competition. This means that students can’t use hibernation or suspension modes to power down parts of the cluster to reduce electric load. They can modify BIOS settings before the competition begins but typically aren’t allowed to make any mods after kickoff. In fact, reboots are allowed only if the system fails or hangs up.
Each system is attached to a PDU (Power Distribution Unit) that will track second-by-second power usage. When teams go over the power limit, they’ll be alerted by the PDU – as will the competition managers. Going over the limit will result in a warning to the team and possible point deduction. According to the HPAC-ISC Student Cluster Challenge FAQ, “if power consumption gets well beyond the 13A limit, Bad Things™ will happen…” meaning that the team will trip a circuit breaker and lose lots of time rebooting and recovering their jobs.
On the software side, teams can use any o/s, clustering, or management software they desire as long as the configuration will run the required workloads. The vast majority of teams run some flavor of Linux, although there were Russian teams in 2010 and 2011 who competed with a Microsoft-based software stack, and they won highest LINPACK at SC11 in (appropriately enough) Seattle.
It’s amazing how much hardware can fit under that 3 kW limbo bar. At SC12 in Salt Lake City, the Overall Winner Texas Longhorn team used a 10-node cluster that sported 160 CPU cores plus two NVIDIA m2090 GPU boards. Texas had the highest node count, but other teams used as few as six nodes and as many as eight GPU cards.
Look at the performance ramp from November 2009 to November 2012. In three years, student LINPACK scores went from .692 Teraflop/s to 3.014 TFLOP/s. In absolute terms, that’s an increase of 335%. You can also compare it to a loose interpretation of Moore’s Law and assume that compute performance doubles every 18 months.
Using this measure, 2009’s 0.692 score would double to 1.38 in a year and a half, then double again to 2.77 at the three-year mark. So our student clusterers are doing a pretty good job from that perspective. They also compare favorably to the Top500 list during the same period.
As can be seen on the accompanying table, the LINPACK score of the top system on the Top500 went up by a factor of ten over the last three years vs. the student clusterers, who managed to wring 4.36x more LINPACK bang out of their best hardware.
While the kids are humbled by the results at the top of the list, they’re eking out slightly better performance improvements than systems at the median (#250) and end (#500) of the list.
Speaking for myself (and probably untold millions of maniacal fans worldwide), these competitions are highly compelling affairs. The one thing I hear time and time again from students is, “I learned sooo much from this…” They’re not just referring to what they’ve learned about systems and clusters, but what they’ve learned about science and research. And they’re so eager and enthusiastic when talking about this new knowledge and what they can do with it – it’s almost contagious.
For some participants, the Student Cluster Competition is a life-changing event. It’s prompted some students to embrace or change their career plans – sometimes radically. These events have led to internships and even full-time, career-track jobs. For many of the students, this is their first exposure to the world of supercomputing and the careers paths that are available in industry and research. Watching them begin to realize the range of opportunities open to them is very gratifying; it even penetrates a few layers of my own dispirited cynicism.
The schools sending the teams also realize great value from the contests. Several universities have used the SCC as a springboard to build a more robust computer science and HPC curriculum – sometimes designing classes around the competition to help prepare their teams. The contests also give the schools an opportunity to highlight student achievement, regardless of whether or not they win.
Just being chosen to compete is an achievement. As these competitions receive more attention, the number of schools applying for a slot has increased. Interest is so high in China that annual ‘play-in’ cluster competitions are held to select the university teams that will represent the country at ISC and SC.
With all that said, there’s another reason I find these competitions so compelling: they’re just plain fun. The kids are almost all friendly and personable, even when there’s a language barrier hindering full-bandwidth communications. They’re eager and full of energy. They definitely want to win, but it’s a good-spirited brand of competition. Almost every year we’ve seen teams donate hardware to teams in need when there are shipping problems or when something breaks.
It’s that spirit, coupled with their eagerness to learn and their obvious enjoyment, that really defines these events. And it’s quite a combination.
— Dan Olds