PVM Service Management
1. Overview
The PVM service management architecture is designed to bring a hypervisor node from initial OS boot to a fully operational state — a state in which cluster services, shared storage, and the PVM infrastructure are ready for use.
This system is built around two distinct phases:
Phase 1 — Infrastructure Startup
- Configuring the node's network settings
- Establishing cluster membership and reaching quorum
- Starting the Distributed Lock Manager (DLM)
- Mounting the shared GFS filesystem
Phase 2 — PVM Service Startup
- Starting the services that provide PVM's core infrastructure
- Starting the hypervisor manager and supporting services
- Providing a single point of management via a meta-service named
pvm
The dependencies defined in systemd between these services are entirely deliberate. Services are not simply run in an arbitrary order; each critical service is prevented from starting until the infrastructure it needs has reached the required state.
The dependency chain is as follows:
INFRASTRUCTURE PHASE
=====================
+---------+
| sbone |
| Network |
+----+----+
|
v
+-------------+
| corosync |
| Cluster |
| /Quorum |
+------+------+
|
v
+------------------+
| pvm-wait-for- |
| quorum |
+--------+---------+
|
v
+-----------+
| dlm |
| Lock Mgmt |
+-----+-----+
|
v
+------------------+
| pvm-gfs-manager |
| Mount GFS |
+--------+---------+
|
|
Infrastructure is ready
|
v
====================
PVM SERVICES PHASE
====================
|
v
+-----------+
| pvm |
| Meta Unit |
+-----+-----+
|
+-------------------+-------------------+
| | |
v v v
+--------+ +---------+ +----------+
| sballd | | plogger | |pvm-engine|
+----+---+ +---------+ +----------+
|
v
+--------+
| sabad |
| API GW |
+--------+
Other PVM services may also be managed by pvm
2. The Two-Phase Startup Model
The most important concept in PVM service management is that PVM services are not the first thing that should start after boot.
The node must first establish a valid cluster environment and storage space.
Phase 1 — Infrastructure
The first phase establishes:
- Network configuration
- Cluster communication
- Cluster quorum
- Distributed locking
- Access to the shared filesystem
Only after these prerequisites are met is the infrastructure considered ready.
Phase 2 — PVM Services
Once the infrastructure is ready, the meta-service named pvm can be started. This service is responsible for bringing up the PVM services that depend on the base infrastructure.
This separation matters because services such as sballd ultimately depend on the shared storage being stable and available.
3. Infrastructure Services
3.1 The sbone Service
sbone is responsible for establishing the node's base cluster configuration. One of its most important tasks at boot time is configuring the network.
PVM systems do not rely on NetworkManager to configure the node's network addresses. Instead, sbone itself performs the required network configuration, including IP assignment.
sbone is therefore a critical prerequisite for the cluster stack.
sbone
|
+-- Cluster configuration
+-- Network configuration
|
v
corosync
4. The corosync Service — Cluster Membership and Quorum
After the node's base configuration is established, corosync provides the cluster's communication and membership layer. Corosync allows the cluster to determine which nodes are currently participating in it and whether the cluster has reached quorum.
Quorum is critical for preventing split-brain.
A split-brain condition occurs when different parts of a cluster independently believe they hold authority. For a cluster with shared storage, this can be extremely dangerous, since multiple nodes might attempt to access or modify shared resources without a consistent view of cluster membership.
sbone
|
v
corosync
|
v
cluster reaches quorum
Corosync reaching an active state does not guarantee that the cluster has reached quorum. This distinction is the reason the next service exists.
5. The pvm-wait-for-quorum Service
5.1 Purpose
pvm-wait-for-quorum exists to bridge the gap between two states:
- The cluster service having started
- The cluster actually being ready for safe use
corosync.service being active alone is not enough to guarantee that the cluster has reached quorum. This matters particularly for DLM.
5.2 Why DLM Cannot Start Before Quorum
DLM provides the distributed locking that the cluster filesystem needs. GFS relies on DLM to coordinate access to the shared filesystem across cluster nodes.
Cluster / Quorum
|
v
DLM
|
v
GFS
|
v
Shared Storage
Starting DLM before the cluster reaches quorum is dangerous. If DLM starts while the cluster has not yet reached quorum, the node may get fenced.
This behavior is intentional from the cluster's perspective: fencing is a safety mechanism designed to prevent the cluster from operating unsafely. But during normal boot, allowing DLM to start prematurely can trigger unnecessary fencing.
5.3 The Quorum Gate
pvm-wait-for-quorum waits until the cluster actually reaches quorum. DLM runs after this service via systemd's After= relationship.
corosync starts
|
v
cluster membership is established
|
v
pvm-wait-for-quorum
|
| waits
v
cluster reaches quorum
|
v
pvm-wait-for-quorum becomes active
|
v
DLM can start
The underlying goal is:
Do not start DLM until the cluster has reached quorum.
pvm-wait-for-quorum states this requirement explicitly.
6. The dlm Service — Distributed Lock Manager
Once quorum is established, DLM can be started safely. DLM provides the distributed locking capability that the cluster filesystem needs. DLM is therefore an infrastructure dependency, not a PVM application service.
corosync
|
v
quorum
|
v
pvm-wait-for-quorum
|
v
dlm
|
v
GFS
Important distinction:
- DLM provides distributed locking.
- GFS provides the cluster filesystem.
pvm-gfs-managermanages the actual GFS mount.
7. The pvm-gfs-manager Service — GFS Management
The purpose of pvm-gfs-manager is clearly defined:
After DLM starts, automatically mount the shared GFS storage.
Rather than requiring the system administrator to confirm DLM and then manually mount the filesystem, GFS Manager makes the mount process part of the systemd dependency chain.
Quorum
|
v
pvm-wait-for-quorum
|
v
DLM
|
v
pvm-gfs-manager
|
v
GFS mounted
8. The Full Infrastructure Chain
The complete dependency chain for Phase 1 is as follows:
+---------+
| sbone |
| Network |
+----+----+
|
v
+-----------+
| corosync |
+-----+-----+
|
v
+------------------+
| Cluster Quorum |
| Required |
+--------+---------+
|
v
+-----------------------+
| pvm-wait-for-quorum |
+-----------+-----------+
|
v
+-------+
| dlm |
+---+---+
|
v
+----------------+
|pvm-gfs-manager |
+-------+--------+
|
v
+---------+
| GFS |
| Mounted |
+----+----+
|
v
Infrastructure is ready
The key feature of this architecture is that each layer establishes a prerequisite for the next layer. The system administrator therefore does not need to manually coordinate these operations.
9. Phase 2 — PVM Services
Once the cluster has reached quorum and the shared GFS storage is available, the node's infrastructure is ready to host the actual PVM services.
At this point, Phase 2 begins.
Rather than managing each PVM service individually, PVM provides a meta-service:
pvm.service
The recommended management interface is the pvm service itself:
systemctl enable pvm
systemctl start pvm
systemctl restart pvm
systemctl stop pvm
Instead of managing the following services separately:
sballd
plogger
pvm-engine
sabad
...
The system administrator typically interacts with a single service:
pvm
10. The pvm Meta-Service
The pvm service should be thought of as a service group or meta-service, not as a service with a specific function of its own. Its purpose is to represent the operational status of the entire PVM services stack.
Starting pvm brings up the required PVM services. Likewise, stopping or restarting pvm provides a centralized way to manage the PVM services stack.
The intended operational model is:
Start
pvm; the service dependency graph handles the prerequisites.
11. Key PVM Services
11.1 The sballd Service
sballd is the core hypervisor management service. It is one of the most important services in the PVM stack and ultimately depends on the base infrastructure being ready. In particular, sballd requires a stable, available GFS environment.
sballd
|
v
GFS required
|
v
pvm-gfs-manager
|
v
DLM
|
v
Quorum
|
v
Corosync
11.2 The plogger Service
plogger provides PVM's logging capability. System administrators typically do not need to manage plogger independently; it starts as part of the pvm service stack.
11.3 The pvm-engine Service
pvm-engine provides PVM's microservice engine. Like other PVM components, it is managed through the pvm meta-service.
11.4 The sabad Service
sabad is responsible for running the API gateway and provides the entry point for API-driven interaction with the PVM infrastructure.
PVM Services Stack Diagram
+---------------+
| pvm.service |
| Meta-Service |
+-------+-------+
|
+------------------+------------------+
| | |
v v v
+---------+ +---------+ +------------+
| sballd | | plogger | | pvm-engine |
|Hypervisor| | Logging | | Microservice|
+---------+ +---------+ +------------+
|
v
+---------+
| sabad |
| API GW |
+---------+
12. Important Change in PVM Version 7.1
Safe Service Restarts
An important operational change was introduced in PVM version 7.1.
Versions prior to 7.1
In earlier versions, restarting PVM-related services could cause virtual machines to be unloaded, and consequently powered off. This made careless service restarts potentially dangerous.
In versions prior to 7.1, restarting the PVM service stack should be treated as a potentially destructive operation. Administrators should avoid restarting related services, especially on nodes with active virtual machines.
Version 7.1 and later
From PVM 7.1 onward, this behavior has changed. Restarting the PVM service stack is safe with respect to virtual machines — restarting PVM services no longer causes virtual machines to be unloaded or powered off.
Therefore, in PVM 7.1 and later, the service stack can be managed normally:
systemctl restart pvm
| PVM Version | Restarting PVM Services |
|---|---|
< 7.1 | Potentially destructive; virtual machines may be unloaded or powered off |
>= 7.1 | Safe; restarting services does not unload or power off virtual machines |
13. Operational Model in PVM 7.1 and Later
After boot, infrastructure services are established in the following order:
Network → Cluster → Quorum → DLM → GFS
Once these prerequisites are met, the PVM service stack starts:
pvm
|
+-- sballd
+-- plogger
+-- pvm-engine
+-- sabad
+-- other PVM services
The system administrator does not need to manually start each component or repeatedly check the infrastructure stack. This knowledge is encoded into the dependency chain within systemd.
14. Recommended Management
For PVM version 7.1 and later, system administrators should generally use the pvm meta-service to manage the service stack.
Common operations:
systemctl enable pvm
systemctl start pvm
systemctl restart pvm
systemctl stop pvm
If a node requires shared GFS storage, GFS Manager should also be enabled:
systemctl enable pvm-gfs-manager
Individual services should only be managed directly for troubleshooting, debugging, or specific maintenance operations. In normal operation, pvm should be treated as the primary management interface for the PVM service stack.
15. Summary
The PVM services architecture is divided into two main layers.
The infrastructure layer establishes the environment PVM requires:
sbone → corosync → quorum → pvm-wait-for-quorum → dlm → pvm-gfs-manager → GFS
The PVM services layer is managed through the pvm meta-service:
pvm
|
+-- sballd
+-- plogger
+-- pvm-engine
+-- sabad
+-- other PVM services
Despite these dependencies, system administrators typically only need to enable the pvm service. If a node requires shared GFS storage, pvm-gfs-manager should also be enabled. The service dependency chain ensures that the required infrastructure starts up in the correct order and that PVM services only begin running once their prerequisites are ready.
In PVM version 7.1 and later, restarting services is also safe with respect to running virtual machines, and systemctl restart pvm is considered a normal, safe operation.