Mayflower: Business Background

Unlike traditional web applications, our platform is built around constant user-to-user interactions. Every action performed by one user can immediately affect the experience of many others, making low latency, high availability, and predictable performance critical business requirements rather than technical preferences.

As the platform continued to grow, we needed an infrastructure capable of scaling efficiently while maintaining reliability under heavy, constantly changing workloads. OpenStack became a key part of our strategy, providing the flexibility, automation, and operational control required to support our business growth and deliver a consistent experience to millions of active users every day.

Mayflower: A Hybrid Private Cloud as an AI Factory

AI projects require compute resources here and now: teams need to quickly allocate CPU and GPU capacity, launch environments for model training, deploy inference services, scale Kubernetes workloads, and move experimental pipelines into production without waiting weeks for infrastructure to be prepared manually.

At Mayflower, we moved away from the traditional choice between two extremes: virtualization or physical hardware. Instead, we built a hybrid private cloud based on OpenStack, where virtual machines, bare-metal servers, GPU resources, and OKD clusters are managed as parts of a single infrastructure platform.

The key idea behind this architecture is not to force workloads into a predefined infrastructure model. The platform should provide exactly the type of resource that matches the workload profile: virtualized, physical, GPU-enabled, or Kubernetes-native.

Architecture: Three Resource Consumption Models

We believe there is no universal infrastructure layer that fits every task. That is why Mayflower supports several models for consuming compute resources.

1. Virtual Machines

Virtual machines are used for standard business applications, internal services, test and staging environments, and workloads that do not require direct access to physical hardware.

This layer provides fast resource allocation, environment isolation, managed VM lifecycle, and a familiar operational model for services that run well in a virtualized environment.

2. Bare Metal

Bare-metal resources are used when maximum performance, predictable behavior, and the absence of hypervisor overhead are critical.

This approach is suitable for workloads that are sensitive to latency, throughput, NUMA topology, disk I/O, or direct access to hardware resources. In these scenarios, a physical server is provided directly to the workload, without an intermediate virtualization layer.

3. OKD as the PaaS Layer

OKD acts as the platform layer for cloud-native applications. Teams deploy services in a Kubernetes-compatible environment and benefit from orchestration, automated workload scheduling, rolling updates, self-healing, and declarative application infrastructure management.

Within OKD, we use two types of worker nodes:

VM-based workers — for scenarios where flexibility, fast scaling, and convenient capacity management are important.

Bare-metal workers — for heavy workloads that require high performance, direct hardware access, or GPU usage.

This model allows us to run both lightweight microservices and resource-intensive AI workloads on the same platform.

The Key Scaling Mechanism: OpenStack Ironic

One of the central components of our architecture is OpenStack Ironic. It allows us to manage physical servers as cloud resources: discover hardware, prepare it, install the required operating system image, and automatically bring the server into service.

We have automated this process so that the path from a resource request to a ready-to-use node takes less than 60 minutes.

Depending on the scenario, the same physical server can be:

• provided as a standalone bare-metal resource;
• added to OpenStack as a hypervisor host for running virtual machines;
• joined to OKD as a bare-metal worker;
• prepared as a GPU-enabled worker for AI and ML workloads.

For a private cloud of this scale, this is critically important. Resource demand, especially in AI projects, often grows in bursts: a new model, a new experiment, a new production service, or a sharp increase in inference traffic. Infrastructure must be able to respond to these changes in hours, not weeks.

Ironic allows us to turn physical hardware from a static resource that requires manual preparation into a managed capacity pool that can be quickly introduced into the required operational environment.

GPU on Demand

At Mayflower, GPUs are not treated as a separate isolated resource. They are integrated into the overall hybrid cloud model and are available in different modes depending on the workload type.

PCI Passthrough

For heavy model training and tasks that require maximum GPU performance, we use PCI passthrough. In this mode, a virtual machine gets direct access to the GPU device, minimizing overhead and allowing the hardware resource to be used as efficiently as possible.

This approach is suitable for training workloads where performance, stable latency, and predictable behavior of the compute environment are important.

Time-Slicing

For tasks that do not require full exclusive access to a GPU, we use time-slicing. This allows multiple workloads to share a single GPU resource and improves the overall utilization of expensive hardware.

This mode is especially useful for experiments, smaller inference workloads, model development, testing, and scenarios where an entire GPU would otherwise remain underutilized most of the time.

GPU-Enabled OKD Workers

For cloud-native scenarios, we provide OKD workers with GPU support. Developers deploy applications into a Kubernetes-compatible environment, while the platform handles node preparation, GPU integration, resource publication into the cluster, and GPU availability management for workloads.

For the development team, this looks like a standard Kubernetes model: the application requests the required resource, the scheduler places the workload on a suitable node, and the infrastructure complexity remains hidden inside the platform.

ECMP Load Balancing: A Distributed Entry Layer for the Hybrid Platform

Providing compute resources quickly is not enough. In a hybrid cloud, it is important not only to run the workload, but also to deliver traffic to it in a predictable, scalable, and resilient way.

At Mayflower, a single application may consist of components placed across different infrastructure layers: some services may run on bare metal, some in virtual machines, and others in OKD. At the same time, for the user or an external system, this should look like a single service with a stable entry point.

A classic design with one central load balancer does not scale well for this model. As the platform grows, the load balancer becomes a critical element of its own: it must be scaled, protected, made highly available, and constantly monitored so that it does not become a bottleneck or a single point of failure.

To address this, Mayflower uses ECMP — Equal-Cost Multi-Path — based load balancing.

ECMP allows traffic to be distributed across multiple equivalent routes. If several next hops with the same route cost are available for a service, the network equipment can use them in parallel. Traffic distribution happens at the routing layer rather than through a single centralized load balancer.

In practical terms, this means that incoming traffic can be distributed across multiple active nodes, sites, or network paths. Path selection is typically based on flow hashing, taking into account connection parameters such as source and destination IP addresses, ports, and protocol. As a result, a single flow remains stable, while new connections are distributed across the available routes.

At Mayflower, ECMP is used as an infrastructure-level load balancing mechanism for different types of workloads:

• bare-metal services;
• virtual machines;
• Kubernetes/OKD workloads;
• internal platform services;
• high-load APIs;
• services distributed across multiple zones or data centers;
• scenarios requiring a unified traffic delivery model across LAN, WAN, and cloud segments.

The entry layer is built as a geographically distributed active-active model. HA BGP routers announce routes from different sites and exchange routing information between zones. This allows multiple entry points to be used simultaneously, instead of keeping one active site and another one waiting in standby mode.

If a route, node, or site becomes unavailable, the corresponding route can be withdrawn, and new connections start going through the remaining available paths. This approach reduces dependency on a single entry point and improves the platform’s resilience to failures.

ECMP fits especially well into Mayflower’s hybrid model because it is not tied to a specific infrastructure type. From the network layer’s perspective, it does not matter whether the backend is running on bare metal, inside a VM, or within OKD. What matters is that there is a correctly announced route and an available next hop.

For example, one component of an AI service may perform inference in OKD on GPU-enabled workers, another may store data or perform preprocessing on bare metal, while supporting APIs may run in virtual machines. ECMP allows these components to be connected into a single network model without requiring a separate load balancing design for each infrastructure layer.

At the same time, ECMP does not replace L7 load balancers, ingress controllers, or service mesh. It does not handle HTTP routing, TLS termination, canary deployments, or header-based routing. Its role is different: to provide scalable and resilient traffic delivery at the infrastructure level, on top of which application-level balancing mechanisms can operate.

This approach gives Mayflower an important advantage: not only compute capacity scales, but also the entry layer. The platform can increase the number of services, zones, clusters, and backends without constantly redesigning the underlying network architecture.

Why This Model Works

Mayflower is not just a set of servers and not a traditional hosting platform. It is an internal infrastructure platform designed to quickly launch and scale services according to business requirements.

The hybrid model of VM + bare metal + OKD, brought together by OpenStack and Ironic, allows us to use one infrastructure foundation for different types of workloads:

• standard enterprise applications;
• dev/test environments;
• cloud-native services;
• high-load backend components;
• GPU-intensive AI and ML workloads;
• production inference services;
• experimental research environments.

ECMP and HA BGP complement this model at the network layer. They make it possible to build a distributed active-active entry layer that connects different infrastructure environments into a single network platform and reduces dependency on centralized load balancing points.

As a result, infrastructure becomes not a limitation, but an accelerator. Teams receive resources faster, GPUs are used more efficiently, bare-metal capacity is brought into service automatically, OKD provides a convenient layer for developing and operating cloud-native applications, and ECMP ensures scalable traffic delivery to these workloads.

We do not adapt the business to the cloud. We build the cloud to match business requirements: performance, scaling speed, workload type, network resilience, and automation level.

Mayflower is a hybrid private cloud operating as an AI factory: manageable, scalable, automated, network-resilient, and ready for the next generation of workloads.

Tags: AI, ECMP, Ironic, okd containers, OpenStack, Private Cloud