Open Compute for Morpheus

Vistara aims to enhance the Morpheus ecosystem by leveraging its advanced virtualization and orchestration technologies using its two core concepts Hypercore and Spacecore.

Hypercore is a Hardware Abstraction Interface that provides a unified API to implement various hypervisors that can be purpose built for specific use cases.

Spacecore is a modular, atomic unit of the Vistara network that packages all application code, dependencies, and environment into a containerd image following the CRI (Container Runtime Interface) guidelines.

Hypercore is used to provide a secure, efficient, and scalable compute provider infrastructure. And spacecore is used to quickly deploy and orchestrate containerized Smart Agents and AI models on Morpheus.

Features:

  1. Open Compute Infrastructure:

    • Hypercore in Vistara is a Hypervisor Abstraction Interface that implements various KVM hypervisors like cloud-hypervisor that can be used to instantly provision hardware resources needed for providing computing, storage, and bandwidth to the Morpheus network. The simplicity of Hypercore's unified interface makes it easy to use and faster to deploy. Utilizing the hypercore reduces the time to market as it is faster to provision and deploy the resources needed. It also creates isolated execution environments leveraging Intel SGX, TDX, and NVIDIA MIG. This ensures hardware-level security and efficient resource utilization for running Morpheus Smart Agents and AI workloads.

  2. GPU Virtualization and Acceleration:

    • Implementing NVIDIA’s vGPU and MIG technologies within Hypercore to optimize GPU resource sharing and acceleration, is crucial for AI training and inference tasks within the Morpheus ecosystem.

    • Performance Efficiency: Ensure that Compute Providers can offer scalable and cost-efficient high-performance computing capabilities, adapting dynamically to workload demands.

  3. Containerized Deployment and Orchestration:

    • Deploy Morpheus Smart Agents and AI models using Spacecores, which packages these components, their dependencies, and the environment in secure, containerized units. This simplifies the deployment of smart agents process and ensures consistent operations across diverse hardware setups.

    • Develop tools and workflows that integrate Spacecore with the Morpheus ecosystem, enabling easy provisioning and management of Smart Agent instances by Compute Providers.

Specifications:

  • Hardware Resources: Detail the requirements and specifications for computing resources within the Morpheus network, including minimum hardware capabilities, supported hypervisor technologies, and security features. Could be detailed per LLM that needs to be served.

  • API and SDK Specifications: Define and standardize the APIs and SDKs for integrating with Morpheus Smart Agents, ensuring compatibility and ease of use for developers.

  • Abstraction for trust assumptions in security protocols implemented by compute providers, including encryption standards, authentication mechanisms, etc.

Components:

  1. Hypercore:

    1. Acts as the decentralized compute provider, offering virtualized and secure execution environments.

    2. Based on hardware utilization, history, bandwidth, and demands, it can efficiently route inferences to the right virtual machine running Vistara.

    3. Implements a KVM hypervisor for serving LLMs on GPUs using GPU-passthrough optimizing for number of inferences at scale.

    4. Implements an NVIDIA MIG (Multi-Instance GPU) to isolate different LLMs serving different AI workloads.

  2. Spacecore:

    1. Manages the faster deployment and orchestration of Morpheus smart agents with one command.

    2. Smart agent developers build a Hardware-as-Code hac.toml file specifying the hardware resource requirements for a given LLM.

  3. Morpheus Router:

    1. Coordinates compute requests and facilitates interactions between users and providers.

Observability spacecore:

  • to be built for monitoring, logging, and tracing capabilities and deployed as a sidecar container.

  • It gathers operational metrics and logs and can be standardized across different Morpheus smart agents deployments.

  • It monitors the resource utilization and performance of the provisioned compute instances and deployed smart agent spacecores.

  • Based on predefined scaling policies or real-time demand, Hypercore dynamically scales the hypervisor infrastructure to ensure optimal performance and resource allocation for serving inferences at scale.

Vistara integrated workflow:

  1. Initial setup:

    • Both Vistara and Morpheus entities generate and manage their MOR public keys to ensure secure and authenticated interactions.

    • Vistara's Hypercores and Spacecores are registered within the Morpheus network, using these keys for all communications.

  2. Compute Request Flow:

    • A user initiates a compute request via the Morpheus network, specifying the required LLM and associated parameters.

    • The request is submitted to the Morpheus Router, which performs a liveness check on available Vistara Hypercores based on the bid per inferences per second (IPS) and current load.

  3. Resource Allocation:

    • Upon passing the liveness check, the Morpheus Router directs the compute request to the optimal Vistara Hypercore.

    • The Hypercore, utilizing its secure virtual environments, begins processing the request using the specified Smart Agents deployed via Spacecores.

  4. Result and Verification:

    • Once the computation is complete, the Hypercore sends the result back to the user through the Morpheus Router.

    • The Router verifies the successful execution and performance metrics (e.g., IPS received, time taken) reported by the user.

  5. Payment and Incentives:

    • Following successful verification, the Morpheus Router triggers a payment transaction from the Compute Contract to the Vistara Hypercore, compensating for the compute resources utilized.

    • All transactions are secured and logged using the smart contracts deployed on Arbitrum and Ethereum, ensuring transparency and trust. More info here.

graph TD
    A[User sends compute request] -->|MOR Public Key| B[Router]
    B --> C{Router evaluates provider}
    C -->|Select best bid| D[Hypercore]
    C -->|Fallback| E[Other Providers]
    D --> F{Hypercore VM provisioning}
    F -->|Deploy Spacecore| G[Spacecore Orchestration]
    G --> H[Run Smart Agents/AI Models]
    H --> I{Check completion}
    I -->|Success| J[Router connects the request to hypercore]
    I -->|Failure| K[Log Error and retry]
    J --> K[Providers compute query and send results to User]
    K --> L[Observability spacecore sends metrics IFS to router]
    L --> M[Router to Compute contract and MOR to compute provider]

Technical Benefits and Enhancements:

  • Enhanced Security: Vistara's use of advanced virtualization technologies like Intel SGX and NVIDIA MIG ensures that all computations are performed in secure, isolated environments, protecting sensitive data and processes.

  • Efficiency and Scalability: Spacecores provide a streamlined mechanism for deploying and managing Morpheus Smart Agents, allowing for rapid scaling and dynamic resource allocation in response to network demands.

  • Developer Tools and Integration: Vistara will offer comprehensive SDKs and APIs to facilitate the development and integration of new smart agents and services within the Morpheus ecosystem.

Shortened:

We would like to contribute to the Morpheus ecosystem. Vistara's Hypercore capabilities provide compute provider infrastructure.

1. Decentralized Compute Provider Infrastructure with Secure Execution Environments:

  • Propose integrating Vistara's Hypercore as a trusted and secure Compute Provider infrastructure for Morpheus.

  • Leverage Hypercore's ability to provision isolated execution environments using technologies like Intel SGX and NVIDIA MIG for running Morpheus Smart Agents and AI models with hardware-level security and data protection.

  • Ensure that Compute Providers on the Morpheus network can offer secure and trusted compute resources, addressing potential concerns around data privacy and integrity.

2. GPU Virtualization and Acceleration for AI Workloads:

  • Integrate Vistara's Hypercore with NVIDIA GPU virtualization technologies (vGPU, MIG) to enable efficient sharing and acceleration of GPU resources for Morpheus AI workloads.

  • Allow Compute Providers to maximize their GPU utilization and offer high-performance computing capabilities for training and inference tasks, ensuring optimal resource allocation and cost-efficiency.

3. Containerized Deployment and Orchestration of Smart Agents:

  • Leverage Vistara's Spacecore to package and deploy Morpheus Smart Agents and AI models as secure containerized units, simplifying the deployment process for Compute Providers and ensuring consistent and reproducible deployments across different hardware environments.

  • Contribute to the development of tools and workflows for seamless integration of Spacecore with the Morpheus ecosystem, enabling Compute Providers to easily provision and manage Smart Agent instances.

4. Developer Tools, SDKs, and API Integrations (Optional):

  • Contribute to the development of developer tools, SDKs, and APIs that enable seamless integration of Morpheus Smart Agents and AI models with various applications and platforms, including Vistara's virtualization and cloud solutions.

  • Foster the adoption of Morpheus by developers and encourage the creation of specialized Smart Agents tailored to specific use cases by providing robust integration capabilities.

PreviousSecure EnclavesNextAI Inference Service

Last updated