The Tokyo Insti­tute of Tech­nol­ogy announced the details of the “Tsub­ame 2.0,” the next-generation super­com­puter sys­tem for the uni­ver­sity that will start oper­a­tion in the fall of 2010, at a press meet­ing. The com­pu­ta­tion capac­ity of the sys­tem is 2.39 PFLOPS (petaflops, double-precision value), which ranks sec­ond in the “Top500,” a rank­ing of super­com­put­ers, as of June 2010. “It will be the first petaflops com­puter in Japan,” said Satoshi Mat­suoka, pro­fes­sor at the Global Sci­en­tific Infor­ma­tion and Com­put­ing Cen­ter (GSIC) of the uni­ver­sity. “And it will be the first world-class super­com­puter sys­tem for our university.”

How­ever, the actual con­struc­tion of the sys­tem, which will be con­ducted by NEC Corp and Hewlett-Packard Co, has yet to be done. The sys­tem has the “vector-scalar mix­ture archi­tec­ture,” Mat­suoka said. But the com­pu­ta­tion capac­ity of its graph­ics pro­cess­ing units (GPUs) accounts for 90% of the total com­pu­ta­tion capac­ity, mak­ing the sys­tem more like a vec­tor com­puter. There­fore, the per­for­mance of the sys­tem slightly dif­fers depend­ing on the type of cal­cu­la­tion. Specif­i­cally, the per­for­mance tar­get in terms of the Lin­pack bench­mark is 1.0−1.4 PFLOPS, which ranks third or fourth in the Top500 as of June 2010. On the other hand, for cal­cu­la­tions that are suited for vec­tor com­put­ers such as weather pre­dic­tion, the per­for­mance can be more than 150 TFLOPS (ter­aflops), which is much higher than the world record (50 TFLOPS).

The back­bone of the super­com­puter sys­tem con­sists of 2,816 of six cores 2.93 GHz Intel Xeon 5600 micro­proces­sor (Westmere-EP), and 4,224 Nvidia Tesla M2050 GPUs. The dou­ble pre­ci­sion arith­metic per­for­mance of the Tesla M2050 is much higher than that of the exist­ing Tesla GPUs, which are devel­oped mainly for sin­gle pre­ci­sion arith­metic. A unit of the Tesla M2050 has a per­for­mance of 515 GFLOPS (double-precision). The per­for­mance per node is 1.6 TFLOPS or 51.2 TFLOPS per rack.


The uni­ver­sity made two major improve­ments for enhanc­ing the per­for­mance of the sys­tem. First, it improved the mem­ory band­width. Specif­i­cally the net­work bisec­tion band­width (the min­i­mum com­mu­ni­ca­tion capac­ity of the cross sec­tion at a ran­dom part of the sys­tem) is about 200 Tbps, which is 33 times higher than that of the Tsub­ame 1.0, a super­com­puter sys­tem con­structed by the uni­ver­sity in 2006.

The other improve­ment was made to the mem­ory and its com­po­si­tion. The uni­ver­sity struc­tured a mul­ti­level stor­age using not only DRAMs such as DDR3 but also SSDs (solid state dri­ves) com­posed of flash mem­o­ries. While the total mem­ory capac­ity of the back­bone system’s DRAMs is 80.6 Tbytes for micro­proces­sors and 12.7 Tbytes for GPUs, the total mem­ory capac­ity of the SSDs is 173.9 Tbytes. SSDs have a high per­for­mance in inputting and out­putting data.

The new super­com­puter sys­tem has one more note­wor­thy fea­ture: low power con­sump­tion. While the power con­sump­tion of the Tsub­ame 1.0 includ­ing its cool­ing sys­tem is 0.85MW, that of the Tsub­ame 2.0, which has a 30 time higher com­pu­ta­tion capac­ity, is only 1MW. So, the power con­sump­tion per com­pu­ta­tion capac­ity was reduced to about 1/25. The per­for­mance value per watt (in terms of the Lin­pack bench­mark) is expected to exceed 1,000 MFLOPS (megaflops) per watt and will pos­si­bly be ranked first in the Green500, a rank­ing of supercomputer’s energy sav­ing per­for­mance, the uni­ver­sity said.

The dras­tic decrease in the power con­sump­tion per com­pu­ta­tion capac­ity is also an advan­tage in terms of cost. The cost for the entire sys­tem and the basic main­te­nance cost for four years amount to ¥3.2 bil­lion (US$35 mil­lion), which is low. While the nor­mal cost to intro­duce a super­com­puter is about ¥10 mil­lion per 1 TFLOPS, the cost to intro­duce the Tsub­ame 2.0 is about ¥3 mil­lion per 1 TFLOPS. The cost does not include elec­tric­ity costs, which are about ¥100 mil­lion per year. If the elec­tric­ity costs increased in the same ratio as the com­pu­ta­tion capac­ity, they could be up to ¥2.5 bil­lion per year.

21st June, 2010