PSC Neocortex
Pittsburgh Supercomputing Center
Resource Type: Compute
User Guide: https://portal.neocortex.psc.edu/docs/system-specifications.html
Recommended Use: Neocortex, a system that captures promising innovative hardware technologies, is designed to accelerate Deep Learning (DL) and High Performance Computing (HPC) research in pursuit of science, discovery, and societal good. Currently recommended DL projects focus on foundation and large language models such as BERT, GPT-J, and Transformer, or combine supported TensorFlow or PyTorch layers. DL codes can also be developed “from scratch” using the Cerebras Software Development Toolkit (SDK). The SDK can be used to develop HPC codes, such as structured grid based PDE and ODE solvers and particle methods with regular communication. Interested researchers are encouraged to contact PSC via email at neocortex@psc.edu to address comments and questions.
Latitude: 40.440624
Longitude: -79.995888
Production Dates: 12/01/2023 -
Public URL: https://cider.access-ci.org/public/resources/RDR_002055
Description: Neocortex is a highly innovative advanced computing system ideal for foundation and large language models. Neocortex, which captures promising specialized innovative hardware technologies, is designed to vastly accelerate large deep learning (DL) models and high- performance computing (HPC) research in pursuit of science, discovery, and societal good. Neocortex features two Cerebras CS-2 systems, provisioned by an HPE Superdome Flex HPC server and the Bridges-2 filesystems. Each CS-2 system features a Cerebras WSE-2 (Wafer Scale Engine 2), the largest chip ever built, with 850,000 Sparse Linear Algebra Compute cores, 40 GB SRAM on-chip memory, 20 PB/s aggregate memory bandwidth and 220 Pb/s interconnect bandwidth. The HPE Superdome Flex (SDF) features 32 Intel Xeon Platinum 8280L CPUs with 28 cores (56 threads) each, 2.70-4.0 GHz, 38.5 MB cache, 24 TiB RAM, aggregate memory bandwidth of 4.5 TB/s, and 204.6 TB aggregate local storage capacity with 150 GB/s read bandwidth. The SDF can provide 1.2 Tb/s to each CS-2 system and 1.6 Tb/s from the Bridges-2 filesystems. Jobs are submitted via SLURM. The CS-2 systems can run customized TensorFlow and Pytorch containers, as well as programs written using the Cerebras SDK or the WSE Field Equation API.