PSC Neocortex CS
Pittsburgh Supercomputing Center
Resource Type: Compute
User Guide: https://portal.neocortex.psc.edu/docs/
Recommended Use: Neocortex, a system that captures promising innovative hardware technologies, is designed to accelerate Deep Learning (DL) and High Performance Computing (HPC) research in pursuit of science, discovery, and societal good. Currently recommended DL projects focus on NLP, Vision and Multimodal models, or combine supported PyTorch layers. DL codes can also be developed “from scratch” using the Cerebras Software Development Toolkit (SDK). The SDK can be used to develop HPC codes, such as structured grid based PDE and ODE solvers and particle methods with regular communication. Interested researchers are encouraged to contact PSC via email at neocortex@psc.edu to address comments and questions.
Latitude: 40.440624
Longitude: -79.995888
Production Dates: 12/01/2023 -
Public URL: https://cider.access-ci.org/public/resources/RDR_002055
Description: Neocortex is a highly innovative advanced computing system ideal for foundation and large language models. Neocortex, which captures promising specialized innovative hardware technologies, is designed to vastly accelerate large deep learning (DL) models and high- performance computing (HPC) research in pursuit of science, discovery, and societal good. Neocortex CS provides access to a Cerebras Wafer-Scale Cluster powered by a CS-3 system. The CS-3 system features a Cerebras WSE-3 (Wafer Scale Engine 3), the largest chip ever built, with 900,000 Sparse Linear Algebra Compute cores, 44 GB SRAM on-chip memory, 24 PB/s aggregate memory bandwidth and 245 Pb/s interconnect bandwidth. Hosted in the Cerebras Developer Cloud, the Neocortex cluster includes a MemoryX external memory node which provides virtually unlimited model weight capacity as well as a SwarmX Fabric interconnect that enables highly efficient scaling to additional CS3 units which can be added on demand. NLP, Vision and Multimodal deep learning models optimized to run on Cerebras hardware are available, supported by data processors, utilities, layers and losses. An LLM assistant can provide information and execute commands. The Cerebras SDK enables the development of new DL and HPC applications.