Ruminations on tech, the digital media, and some golf thrown in for good measure.

Superfast Analytics

leave a comment »

This week flew by.

As pretty much has this whole year.  November 19th, you say?

Speaking of speed, the Supercomputing 2010 conference has been going on down in the great city of New Orleans this week.

At the event, IBM earlier today unveiled details behind a new storage architecture design that will convert terabytes of pure information into actionable insights twice as fast as was previously possible.

This new capability is ideally suited for cloud computing apps and data-intensive workloads like digital media, data mining, financial analytics, and the new architecture is expected to shave hours off of complex computations without requiring heavy infrastructure investment.

Created at IBM Research Lab in Almaden, this new General Parallel File System-Shared Nothing Cluster (GPFS-SNC) architecture is designed to provide higher availability through advanced clustering technologies, dynamic file system management and advanced data replication techniques.

By “sharing nothing,” new levels of availability, performance and scaling are achievable. GPFS-SNC is a distributed computing architecture in which each node is self-sufficient; tasks are then divided up between these independent computers and no one waits on the other.

IBM’s current GPFS technology offering is the core technology for IBM’s High Performance Computing Systems, IBM’s Information Archive, IBM Scale-Out NAS (SONAS), and the IBM Smart Business Compute Cloud.

These research lab innovations enable future expansion of those offerings to further tackle tough big data problems.

As an example of how such a capability might be used in the “real” world, large financial institutions run complex algorithms to analyze risk based on petabytes of data.

With billions of files spread across multiple computing platforms and stored across the world, these mission-critical calculations require significant IT resource and cost because of their complexity.

Using this GPFS-SNC design, running this complex analytics workload could become much more efficient, as the design provides a common file system and namespace across disparate computing platforms, streamlining the process and reducing disk space.

You can learn more about the basic GPFS capability here.

Written by turbotodd

November 19, 2010 at 2:53 pm

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: