User Stories

How the University at Buffalo Modernized Its Research Cloud with OpenStack

An academic computing center’s journey from managing OpenStack alone to a professionally supported private cloud serving over 100 research workloads.

August 10, 2026
image

The Challenge: When HPC Alone Isn’t Enough

The University at Buffalo (UB), part of the State University of New York (SUNY) system, is a major public research university with campuses in Buffalo and Amherst, New York. Its Center for Computational Research (CCR) has served the university’s research community since 1998 as a High-Performance Computing (HPC) and data analytics provider. CCR complements departmental IT, stepping in when researchers need large-scale computing resources beyond what individual departments can provide.

CCR’s primary portfolio is HPC. Its Slurm-managed Linux Beowulf clusters provide over 26,000 CPU cores and more than 3 PFlop/s of peak compute capacity, backed by 4.6PB of Vast Data storage. Researchers across Physics, Chemistry, Engineering, and Bioinformatics, and increasingly, driven by AI adoption, departments like Linguistics, Business Management, and Geology, rely on these clusters for batch-scheduled number crunching and data analytics.

But as demand grew, the limitations of HPC alone became clear. Not all research fits the batch-scheduling model. Researchers began requesting capabilities that HPC couldn’t deliver: self-service access to long-running virtual machines, flexible software environments, and on-demand infrastructure for workloads that need to stay online continuously.

To address this, CCR deployed an on-premises research cloud using OpenStack – branded internally as Lake Effect – to complement its HPC capabilities with the scalability, flexibility, and multi-tenancy of a private cloud model.

What Researchers Run on Lake Effect

Lake Effect serves a fundamentally different purpose than CCR’s HPC clusters. Where HPC handles number crunching through batch job scheduling, Lake Effect supports long-running research infrastructure that needs to stay online continuously.In practice, this means websites fronting research projects, databases providing backend support for calculations running on HPC clusters, and portals where researchers submit data for processing. Real-world examples include:

  • REDfly – a knowledgebase of insect gene regulatory sequences, with an emphasis on experimentally validated regulatory elements curated from the published literature
  • TAMALE – Interactive web server that identifies functionally important residues across the human proteome
  • SUNYCell – Digital Slide Archive, a software designed for digital pathology upload, review, annotation, and analysis

Any research activity that is longer-term and not number crunching or data analytics runs in the cloud. HPC and OpenStack are completely separate environments serving completely different needs and together they give CCR’s researchers a comprehensive set of tools.

Deploying OpenStack: Early Wins and Growing Pains

The initial deployment of OpenStack gave researchers what they needed: self-service access to virtual machines, better resource utilization, and support for workloads that didn’t fit neatly into traditional batch-scheduling models. The cloud also allowed the university to maintain the security and compliance posture required in an academic setting, with greater control over data access and governance.

However, managing and maintaining the OpenStack environment in-house quickly became a challenge. As Salvatore Guercio, Systems Administrator at UB’s Center for Computational Research, put it:

“For the first few years we managed and maintained the Research Cloud by ourselves, and it was an incredibly challenging task that required more staff time than we had available.”

The team lacked the depth of OpenStack expertise needed to enable advanced features, and critical operational tasks – particularly upgrades – were not even a consideration. Over time, the environment fell behind on releases and began showing signs of instability: slowdowns, minor but persistent issues, and a growing gap between the version in production and the current upstream releases. CCR had almost a full-time person dedicated solely to maintaining the OpenStack deployment.

The Turning Point: Seeking Community-Backed Expertise

Rather than abandon the investment in OpenStack, UB’s CCR sought out professional support from the OpenStack ecosystem. After evaluating providers, the university partnered with VEXXHOST, an OpenInfra Foundation member and active upstream contributor.The first priority was stabilization: bringing the environment up to date by upgrading from the Wallaby release to the Zed release of OpenStack. This was a non-trivial leap across multiple release cycles and required careful planning to minimize disruption to active research workloads.

The upgrade addressed key concerns around high availability and performance, while also unlocking features from several intermediate releases that the team had previously been unable to adopt. The environment was subsequently converted to run on Atmosphere, VEXXHOST’s open-source deployment and lifecycle management tooling. CCR currently runs Atmosphere 7.6.

The Infrastructure Behind Lake Effect

Lake Effect’s physical deployment consists of 3 controller nodes, 24 compute hosts, and 18 Ceph OSD storage hosts – all Dell PowerEdge servers. Controllers are Dell PowerEdge R440s, each with a 20-core Intel Xeon Gold 6230 processor and 160GB RAM. The compute tier is a mix of configurations: the majority are Dell PowerEdge R650s with dual 56-core Intel Xeon Gold 6330 processors and 1TB RAM. Five Dell PowerEdge R7525 nodes provide GPU capability with AMD EPYC 7452 processors, 512GB RAM, and three NVIDIA A40 GPUs per host. Additional R740 nodes carry dual NVIDIA V100 GPUs.

Storage is backed by 216 Ceph OSDs distributed across three server racks on Dell PowerEdge R740xd hosts, each with 12x 8TB SATA drives and 4x 1.75TB SSDs for database journals. CRUSH maps are configured for full rack-level redundancy. The environment provides 1.7PB of Ceph block storage, backing both Cinder and Glance.

Deployed OpenStack services include Nova, Neutron, Cinder, Glance, Swift, Magnum, Octavia, and Manila.

Networking

Every host has bonded network connections, with each host connected to two separate network switches for full physical redundancy. A provider network tied to the campus external network allows provisioning of VMs with public-facing IP addresses, while a separate physical network handles cloud management traffic.

All hosts are connected at 10G with 40G uplinks to core switches. CCR encourages users to leverage private networks wherever possible and limit external-facing VMs to maintain a strong security posture.

Operational Simplicity: Day-2 Operations and Upgrades

A recurring theme in OpenStack operator discussions is the burden of day-2 operations –monitoring, patching, upgrading, and incident response. UB’s experience underscores this challenge.After the conversion to Atmosphere, the operational model changed dramatically. Host OS and firmware upgrades are now handled quarterly, live-migrating VMs during the process so that most users never notice. Atmosphere itself is upgraded every six months unless a security issue requires an earlier update. Thanks to this shift, upgrades are super easy now, often completed in just a few hours.

Monitoring is handled through Prometheus and Grafana, supplemented by custom scripts for overall status and failure checks. All physical servers are enrolled in Dell OpenManage Enterprise for hardware failure monitoring and firmware updates. User access is provided through the Horizon dashboard and OpenStack CLI, with single sign-on integration through the university’s identity management system via the OpenID protocol.

This experience echoes a broader lesson for the OpenInfra community: the sustainability of an OpenStack deployment depends not just on the initial setup, but on having a reliable, repeatable path for ongoing operations.

The Kubernetes Opportunity

While CCR has Magnum deployed and available as a service within Lake Effect, Kubernetes adoption among the research community has not yet taken hold, primarily due to the learning curve for users more accustomed to traditional VM-based workflows. The capability is in place and, as the research community’s familiarity with cloud-native tooling grows, Magnum provides a ready path for Kubernetes cluster provisioning and lifecycle management without requiring additional infrastructure investment. This is a pattern increasingly relevant across the OpenInfra community, bridging OpenStack and cloud-native tooling using upstream-aligned projects.

Usage Metrics and Cost Model

An additional capability unlocked during the modernization was detailed usage reporting. For a university environment where cloud resources are shared across departments, research groups, and grant-funded projects, visibility into who is consuming what is essential. CCR operates Lake Effect on a cost-recovery basis, billing researchers by CPU hours, similar to a public cloud model, but only to recoup hardware costs. For researchers, the savings over bare-metal alternatives are significant: spinning up cloud infrastructure takes a day or two, compared to weeks or months for hardware procurement, and with physical server costs rising 3–4x in the past year alone, the cloud model has become even more compelling.

Impact

The most significant outcome of the transition is operational efficiency. Before the partnership, CCR had almost a full-time person dedicated to maintaining OpenStack. Today, Guerciomanages the entire cloud environment himself, spending only a fraction of his time on maintenance. The environment currently runs approximately 110 VMs on average, with some instances operating without major service interruption for years.

Lessons for the OpenInfra Community

  • OpenStack and HPC are complementary, not competing. CCR’s Lake Effect cloud handles what HPC can’t – long-running services, databases, and web portals – and together they cover the full spectrum of research computing needs.
  • Staying current is now achievable. What was previously not even a consideration is now routine: quarterly host upgrades and biannual Atmosphere upgrades, completed in hours with zero user-facing disruption.
  • Kubernetes readiness doesn’t require immediate adoption. Having Magnum available means the infrastructure is ready when users are.
  • Operational sustainability matters more than initial deployment. Going from a full-time maintenance burden to a fraction of one person’s time determines whether an OpenStack deployment survives long-term.
  • Cost-recovery billing works in academic settings. CCR’s CPU-hour billing model gives researchers affordable access while keeping the infrastructure financially sustainable.

Looking Ahead

With a stable, up-to-date OpenStack environment in place, CCR is positioned to continue expanding Lake Effect’s capabilities. The team plans to integrate its Vast storage platform as an additional Cinder provider when Atmosphere 8.x is released, further bridging the HPC and cloud storage environments.

Superuser
Latest posts by Superuser (see all)