Massively Parallel Processing (MPP)

Simply put, Massively Parallel Processing is the use of many processors.  Traditional MPP machines are distributed memory machines that use multiple processors (versus SMP's, which employ a shared memory architecture).  This much everyone can agree on.  However, there seems to be a blurred line separating distributed computing, MPP, clusters and Networks of Workstations (NOW's).  All four architectures are distributed memory architectures.  Distributed Computing usually is the process of stealing idle cycles from ordinary computers.  Most NOW's fall into this category, although sometime people classify a dedicated group of machines with different hardware architectures as a NOW (which would actually seem to fall into the category of a cluster).  While Distributed Computers use many processors and distributed memory, they are usually not dedicated to the parallel applications.  Clusters are typically dedicated to the parallel application.  These systems are usually built from commercial hardware and software joined by a network and employing some type of message passing.  True MPP's for the most part employ proprietary OS's and at least some special hardware.  They are typically very large, very powerful machines.  Most MPP's are used for computationally intensive research in the engineering and sciences fields.  The Cray 3TE and Janus (links available below) are MPP's both run a proprietary version of UNIX and are not something you can build with just 15 Pentium PC's, a hub, a few patch cables and MPICH.  In the book High Performance Cluster Computing, parallel architectures were broken down into five categories: MPP, SMP, cc-NUMA, Distributed Systems, and Clusters.  The previous discussion highlights the main differences in the classification of Distributed Systems, Clusters, and MPP.  SMP's as mentioned previously are shared memory machines (an example would be the SUN E10000).  CC-NUMA (cache-coherent non-uniform memory access) employs a different type of memory access.  While the memory is physically distributed a CC-NUMA machine still retains some of the benefits of an SMP, because the memory is logically considered shared (an example of a CC-NUMA machine is the SGI Orgin 2000).  Links to some CC-NUMA information are included below.  While most of the links I included are focused toward MPP's, I also included some links for distributed computing using Internet hosts, NOWs and CC-NUMA.  SMP is covered in some of the general Parallel Processing links at the bottom of the page.  I did not include anything on clusters, you can refer to the Beowulf page for information on those.

Massively Parallel Processing (MPP) Definitions

General Articles on Massively Parallel Processing

Massively Parallel Computers

Janus at Sandia National Laboratory

Janus is powered by 9000 processors and can do one trillion operations per second

Is Parallel Processing Dead -- A 1996 Opinion that lists summaries of some of the MPP companies such as Cray, MasPar, etc.  References to computers made by these companies are made frequently on the web, however most of the companies seem to be no longer with us.  Dr. Dietz was a professor at Purdue, he is now at the University of Kentucky. 

The Cray 3TE

Cray has had quite a colorful history.  It merged with SGI, who then created Cray Research.  In March 2000, Cray Research was sold to Tera Computer Company.  It is now once again Cray Inc.  The Cray 3D was it's first MPP.  Now the Cray 3TE is it's most popular MPP.

Hitachi SR2201 Massively Parallel Processor

Distributed Computing Using Internet Hosts

Networks of Workstations

ccNUMA

Origin 2000 Server line from SGI employs this technology.  cc-NUMA combines the benefits of SMP and MPP systems by physically distributing memory that can logically be considered shared (Distributed Shared Memory).  The Origin systems are scalable up to 512 processors.

General Parallel Processing Sites