I spent my early years in IT nursing a large Supercomputer. I say nursing because these machines are at the bleeding
edge of the technology curve and, while they were designed for extreme performance, they often lacked reliability.
Many times I would be phoned in the middle of the night by the operations staff because the Supercomputer was
unwell. So I was less a Systems Admin, more a Doctor diagnosing and curing a sick patient. However, when they were
working, the volumes of data they produced were enormous. Many 10’s of TB per day required storage and post
processing. This was often where the real science was done.
This week I was pleased to read the story about the challenges faced by the Poznan Supercomputing facility in
Poland supporting international research projects. Thankfully Supercomputers are much more reliable these days
and it was interesting to note that the data challenges they produce haven’t changed for those that work with
Perhaps unsurprisingly data volumes, growth, backup and latency feature highly in the article. Making simple
tasks like scanning and locating files more complex. As I had experienced, these limitations quickly prevent you
from scaling capacity and performance. Poznan had the additional challenges of processing millions of large and
small files, separating data from metadata and efficiently supporting different virtualisation technologies and
operating systems. With the complexity of their operation they also needed a single administrative interface to
simplify storage management tasks.
storage for the metadata. Here are just a few of the benefits this approach gave Poznan:-
&bul; The right performance and density for the right data type.
&bul; Multiple OS – such as RedHat, Enterprise Linux, CentOS and Ubuntu.
&bul; Multiple Virtualisation environments such as – VMWare, KVM and Hyper-V
&bul; Multiple Access protocols such as FC-SAN and InfiniBand
&bul; Emerging technologies such as CloudStack
In my experience you may not be a HPC customer today but it is well worth watching the challenges and
dynamics in this segment because one day soon you will probably have the same ,or similar, challenges.
These ‘bleeding Edge’ organisations, such as Poznan, are out there solving these issues today.
As Maciej Brzezniak, storage systems specialist at PSNC said:
“With the NetApp EF-Series flash array we can use SSD for workloads that benefit from higher IOPS,while
achieving maximum storage density with the NetApp E-Series.”
The only thing I would add is ‘under a single administrative interface that simplifies management‘ called
SANtricity Storage Manager’
Here is a link to the Story – well worth reading !