diff --git a/new-docs/404.html b/new-docs/404.html index 4df5c880f8..08572acff7 100644 --- a/new-docs/404.html +++ b/new-docs/404.html @@ -1,2 +1,2 @@ 404 Page not found :: XAPI Toolstack Developer Documentation -

44

Not found

Whoops. Looks like this page doesn't exist ¯\_(ツ)_/¯.

Go to homepage

\ No newline at end of file +

44

Not found

Whoops. Looks like this page doesn't exist ¯\_(ツ)_/¯.

Go to homepage

\ No newline at end of file diff --git a/new-docs/categories/index.html b/new-docs/categories/index.html index bb5421ab79..edca987e49 100644 --- a/new-docs/categories/index.html +++ b/new-docs/categories/index.html @@ -1,10 +1,10 @@ Categories :: XAPI Toolstack Developer Documentation -

Categories

\ No newline at end of file diff --git a/new-docs/design/RDP/index.html b/new-docs/design/RDP/index.html index 8b35fa867c..0ecf386808 100644 --- a/new-docs/design/RDP/index.html +++ b/new-docs/design/RDP/index.html @@ -11,13 +11,13 @@ First it is necessary to turn on the RDP service in the guest. This can be controlled from XenCenter. Several layers are involved. This description starts in the guest and works up the stack to XenCenter. This feature was completed in the first quarter of 2015, and released in Service Pack 1 for XenServer 6.">RDP control :: XAPI Toolstack Developer Documentation -
Design document
Revisionv2
Statusreleased (xenserver 6.5 sp1)
Review#12

RDP control

Purpose

To administer guest VMs it can be useful to connect to them over Remote Desktop Protocol (RDP). XenCenter supports this; it has an integrated RDP client.

First it is necessary to turn on the RDP service in the guest.

This can be controlled from XenCenter. Several layers are involved. This description starts in the guest and works up the stack to XenCenter.

This feature was completed in the first quarter of 2015, and released in Service Pack 1 for XenServer 6.5.

The guest agent

The XenServer guest agent installed in Windows VMs can turn the RDP service on and off, and can report whether it is running.

The guest agent is at https://github.com/xenserver/win-xenguestagent

Interaction with the agent is done through some Xenstore keys:

The guest agent running in domain N writes two xenstore nodes when it starts up:

  • /local/domain/N/control/feature-ts = 1
  • /local/domain/N/control/feature-ts2 = 1

This indicates support for the rest of the functionality described below.

(The “…ts2” flag is new for this feature; older versions of the guest agent wrote the “…ts” flag and had support for only a subset of the functionality (no firewall modification), and had a bug in updating .../data/ts.)

To indicate whether RDP is running, the guest agent writes the string “1” (running) or “0” (disabled) to xenstore node

/local/domain/N/data/ts.

It does this on start-up, and also in response to the deletion of that node.

The guest agent also watches xenstore node /local/domain/N/control/ts and it turns RDP on and off in response to “1” or “0” (respectively) being written to that node. The agent acknowledges the request by deleting the node, and afterwards it deletes local/domain/N/data/ts, thus triggering itself to update that node as described above.

When the guest agent turns the RDP service on/off, it also modifies the standard Windows firewall to allow/forbid incoming connections to the RDP port. This is the same as the firewall change that happens automatically when the RDP service is turned on/off through the standard Windows GUI.

XAPI etc.

xenopsd sets up watches on xenstore nodes including the control tree and data/ts, and prompts xapi to react by updating the relevant VM guest metrics record, which is available through a XenAPI call.

XenAPI includes a new message (function call) which can be used to ask the guest agent to turn RDP on and off.

This is VM.call_plugin (analogous to Host.call_plugin) in the hope that it can be used for other purposes in the future, even though for now it does not really call a plugin.

To use it, supply plugin="guest-agent-operation" and either fn="request_rdp_on" or fn="request_rdp_off".

See http://xapi-project.github.io/xen-api/classes/vm.html

The function strings are named with “request” (rather than, say, “enable_rdp” or “turn_rdp_on”) to make it clear that xapi only makes a request of the guest: when one of these calls returns successfully this means only that the appropriate string (1 or 0) was written to the control/ts node and it is up to the guest whether it responds.

XenCenter

Behaviour on older XenServer versions that do not support RDP control

Note that the current behaviour depends on some global options: “Enable Remote Desktop console scanning” and “Automatically switch to the Remote Desktop console when it becomes available”.

  1. When tools are not installed:
    • As of XenCenter 6.5, the RDP button is absent.
  2. When tools are installed but RDP is not switched on in the guest:
    1. If “Enable Remote Desktop console scanning” is on:
      • The RDP button is present but greyed out. (It seems to sometimes read “Switch to Remote Desktop” and sometimes read “Looking for guest console…”: I haven’t yet worked out the difference).
      • We scan the RDP port to detect when RDP is turned on
    2. If “Enable Remote Desktop console scanning” is off:
      • The RDP button is enabled and reads “Switch to Remote Desktop”
  3. When tools are installed and RDP is switched on in the guest:
    1. If “Enable Remote Desktop console scanning” is on:
      • The RDP button is enabled and reads “Switch to Remote Desktop”
      • If “Automatically switch” is on, we switch to RDP immediately we detect it
    2. If “Enable Remote Desktop console scanning” is off:
      • As above, the RDP button is enabled and reads “Switch to Remote Desktop”

New behaviour on XenServer versions that support RDP control

  1. This new XenCenter behaviour is only for XenServer versions that support RDP control, with guests with the new guest agent: behaviour must be unchanged if the server or guest-agent is older.
  2. There should be no change in the behaviour for Linux guests, either PV or HVM varieties: this must be tested.
  3. We should never scan the RDP port; instead we should watch for a change in the relevant variable in guest_metrics.
  4. The XenCenter option “Enable Remote Desktop console scanning” should change to read “Enable Remote Desktop console scanning (XenServer 6.5 and earlier)”
  5. The XenCenter option “Automatically switch to the Remote Desktop console when it becomes available” should be enabled even when “Enable Remote Desktop console scanning” is off.
  6. When tools are not installed:
    • As above, the RDP button should be absent.
  7. When tools are installed but RDP is not switched on in the guest:
    • The RDP button should be enabled and read “Turn on Remote Desktop”
    • If pressed, it should launch a dialog with the following wording: “Would you like to turn on Remote Desktop in this VM, and then connect to it over Remote Desktop? [Yes] [No]”
    • That button should turn on RDP, wait for RDP to become enabled, and switch to an RDP connection. It should do this even if “Automatically switch” is off.
  8. When tools are installed and RDP is switched on in the guest:
    • The RDP button should be enabled and read “Switch to Remote Desktop”
    • If “Automatically switch” is on, we should switch to RDP immediately
    • There is no need for us to provide UI to switch RDP off again
  9. We should also test the case where RDP has been switched on in the guest before the tools are installed.
\ No newline at end of file diff --git a/new-docs/design/add-qcow-tool-for-vdi-import-export/index.html b/new-docs/design/add-qcow-tool-for-vdi-import-export/index.html index 5d16dde36d..46f3f1d8d1 100644 --- a/new-docs/design/add-qcow-tool-for-vdi-import-export/index.html +++ b/new-docs/design/add-qcow-tool-for-vdi-import-export/index.html @@ -7,7 +7,7 @@ However, we currently cannot export a VDI to a Qcow2 file, nor import one.">Add qcow tool to allow VDI import/export :: XAPI Toolstack Developer Documentation -
Design document
Revisionv1
Status
Design document
Revisionv1
Statusproposed

Add qcow tool to allow VDI import/export

Introduction

At XCP-ng, we are working on overcoming the 2TiB limitation for VM disks while preserving essential features such as snapshots, copy-on-write capabilities, and live migration.

To achieve this, we are introducing Qcow2 support in SMAPI and the blktap driver. @@ -64,12 +64,12 @@ not support different formats we just to handle data copy from socket into file and from file to socket. Sockets and files will be managed into the qcow_tool_wrapper. The forkhelpers.ml manages the list of file descriptors -and we will mimic what the vhd tool wrapper does to link a UUID to socket.

\ No newline at end of file diff --git a/new-docs/design/aggr-storage-reboots/index.html b/new-docs/design/aggr-storage-reboots/index.html index cba944b8b3..5d3920eacc 100644 --- a/new-docs/design/aggr-storage-reboots/index.html +++ b/new-docs/design/aggr-storage-reboots/index.html @@ -1,11 +1,11 @@ Aggregated Local Storage and Host Reboots :: XAPI Toolstack Developer Documentation -
Design document
Revisionv3
Statusproposed
Review#144
Revision history
v1Initial version
v2Included some open questions under Xapi point 2
v3Added new error, task, and assumptions

Aggregated Local Storage and Host Reboots

Introduction

When hosts use an aggregated local storage SR, then disks are going to be mirrored to several different hosts in the pool (RAID). This ensures that if a host goes down (e.g. due to a reboot after installing a hotfix or upgrade, or when “fenced” by the HA feature), all disk contents in the SR are still accessible. This also means that if all disks are mirrored to just two hosts (worst-case scenario), just one host may be down at any point in time to keep the SR fully available.

When a node comes back up after a reboot, it will resynchronise all its disks with the related mirrors on the other hosts in the pool. This syncing takes some time, and only after this is done, we may consider the host “up” again, and allow another host to be shut down.

Therefore, when installing a hotfix to a pool that uses aggregated local storage, or doing a rolling pool upgrade, we need to make sure that we do hosts one-by-one, and we wait for the storage syncing to finish before doing the next.

This design aims to provide guidance and protection around this by blocking hosts to be shut down or rebooted from the XenAPI except when safe, and setting the host.allowed_operations field accordingly.

XenAPI

If an aggregated local storage SR is in use, and one of the hosts is rebooting or down (for whatever reason), or resynchronising its storage, the operations reboot and shutdown will be removed from the host.allowed_operations field of all hosts in the pool that have a PBD for the SR.

This is a conservative approach in that assumes that this kind of SR tolerates only one node “failure”, and assumes no knowledge about how the SR distributes its mirrors. We may refine this in future, in order to allow some hosts to be down simultaneously.

The presence of the reboot operation in host.allowed_operations indicates whether the host.reboot XenAPI call is allowed or not (similarly for shutdown and host.shutdown). It will not, of course, prevent anyone from rebooting a host from the dom0 console or power switch.

Clients, such as XenCenter, can use host.allowed_operations, when applying an update to a pool, to guide them when it is safe to update and reboot the next host in the sequence.

In case host.reboot or host.shutdown is called while the storage is busy resyncing mirrors, the call will fail with a new error MIRROR_REBUILD_IN_PROGRESS.

Xapi

Xapi needs to be able to:

  1. Determine whether aggregated local storage is in use; this just means that a PBD for such an SR present.
    • TBD: To avoid SR-specific code in xapi, the storage backend should tell us whether it is an aggregated local storage SR.
  2. Determine whether the storage system is resynchronising its mirrors; it will need to be able to query the storage backend for this kind of information.
    • Xapi will poll for this and will reflect that a resync is happening by creating a Task for it (in the DB). This task can be used to track progress, if available.
    • The exact way to get the syncing information from the storage backend is SR specific. The check may be implemented in a separate script or binary that xapi calls from the polling thread. Ideally this would be integrated with the storage backend.
  3. Update host.allowed_operations for all hosts in the pool according to the rules described above. This comes down to updating the function valid_operations in xapi_host_helpers.ml, and will need to use a combination of the functionality from the two points above, plus and indication of host liveness from host_metrics.live.
  4. Trigger an update of the allowed operations when a host shuts down or reboots (due to a XenAPI call or otherwise), and when it has finished resynchronising when back up. Triggers must be in the following places (some may already be present, but are listed for completeness, and to confirm this):
    • Wherever host_metrics.live is updated to detect pool slaves going up and down (probably at least in Db_gc.check_host_liveness and Xapi_ha).
    • Immediately when a host.reboot or host.shutdown call is executed: Message_forwarding.Host.{reboot,shutdown,with_host_operation}.
    • When a storage resync is starting or finishing.

All of the above runs on the pool master (= SR master) only.

Assumptions

The above will be safe if the storage cluster is equal to the XenServer pool. In general, however, it may be desirable to have a storage cluster that is larger than the pool, have multiple XS pools on a single cluster, or even share the cluster with other kinds of nodes.

To ensure that the storage is “safe” in these scenarios, xapi needs to be able to ask the storage backend:

  1. if a mirror is being rebuilt “somewhere” in the cluster, AND
  2. if “some node” in the cluster is offline (even if the node is not in the XS pool).

If the cluster is equal to the pool, then xapi can do point 2 without asking the storage backend, which will simplify things. For the moment, we assume that the storage cluster is equal to the XS pool, to avoid making things too complicated (while still need to keep in mind that we may change this in future).

\ No newline at end of file diff --git a/new-docs/design/archival-redesign/index.html b/new-docs/design/archival-redesign/index.html index 8e7a5b217a..6672014fe2 100644 --- a/new-docs/design/archival-redesign/index.html +++ b/new-docs/design/archival-redesign/index.html @@ -3,7 +3,7 @@ rrdd stores knowledge about whether it is running on a master or a slave This determines the host to which rrdd will archive a VM’s rrd when the VM’s domain disappears - rrdd will always try to archive to the master. However, when a host joins a pool as a slave rrdd is not restarted so this knowledge is out of date. When a VM shuts down on the slave rrdd will archive the rrd locally.">RRDD archival redesign :: XAPI Toolstack Developer Documentation -
Design document
Revisionv1
Status
Design document
Revisionv1
Statusreleased (7,0)

RRDD archival redesign

Introduction

Current problems with rrdd:

  • rrdd stores knowledge about whether it is running on a master or a slave

This determines the host to which rrdd will archive a VM’s rrd when the VM’s domain disappears - rrdd will always try to archive to the master. However, when a host joins a pool as a slave rrdd is not restarted so this knowledge is @@ -42,12 +42,12 @@ VM.suspend followed by a VM.resume.

VM.pool_migrate and VM.migrate_send

The source host’s xapi makes a migrate_rrd call to the local rrd, with a destination address and an optional session ID. The session ID is only required for cross-pool migration. The local rrdd sends the rrd for that VM to the -destination host’s rrdd as an HTTP PUT. This behaviour will remain unchanged.

\ No newline at end of file diff --git a/new-docs/design/backtraces/index.html b/new-docs/design/backtraces/index.html index 2b5f5e091b..d663cc2d74 100644 --- a/new-docs/design/backtraces/index.html +++ b/new-docs/design/backtraces/index.html @@ -11,7 +11,7 @@ reliable cross-process (e.g. xapi to xenopsd) cross-language cross-host (e.g. master to slave) We therefore need to ensure that backtraces are captured in our OCaml and python code a marshalling format for backtraces conventions for storing and retrieving backtraces Backtraces in OCaml OCaml has fast exceptions which can be used for both control flow i.e. fast jumps from inner scopes to outer scopes reporting errors to users (e.">Backtrace support :: XAPI Toolstack Developer Documentation -
Design document
Revisionv1
Status
Design document
Revisionv1
Statusconfirmed

Backtrace support

We want to make debugging easier by recording exception backtraces which are

  • reliable
  • cross-process (e.g. xapi to xenopsd)
  • cross-language
  • cross-host (e.g. master to slave)

We therefore need

  • to ensure that backtraces are captured in our OCaml and python code
  • a marshalling format for backtraces
  • conventions for storing and retrieving backtraces

Backtraces in OCaml

OCaml has fast exceptions which can be used for both

  • control flow i.e. fast jumps from inner scopes to outer scopes
  • reporting errors to users (e.g. the toplevel or an API user)

To keep the exceptions fast, exceptions and backtraces are decoupled: there is a single active backtrace per-thread at any one time. If you have caught an exception and then throw another exception, the backtrace @@ -156,12 +156,12 @@ the backtrace is appended to the current Task backtrace. From the client’s point of view the final exception should look the same, but Xapi will have a chance to see and log the whole backtrace.

As a side-effect, it is possible for SM plugins to throw XenAPI errors directly, -without interpretation by Xapi.

\ No newline at end of file diff --git a/new-docs/design/bonding-improvements/index.html b/new-docs/design/bonding-improvements/index.html index d1f41e0dbb..8b11dc9243 100644 --- a/new-docs/design/bonding-improvements/index.html +++ b/new-docs/design/bonding-improvements/index.html @@ -7,7 +7,7 @@ Shutdown all VMs with VIFs using the interfaces that will be bonded, in order to unplug those VIFs. Create a Network to be used by the bond: Network.create Call Bond.create with a ref to this Network, a list of refs of slave PIFs, and a MAC address to use.">Bonding Improvements design :: XAPI Toolstack Developer Documentation -
Design document
Revisionv1
Status
Design document
Revisionv1
Statusreleased (6.0)

Bonding Improvements design

This document describes design details for the PR-1006 requirements.

XAPI and XenAPI

Creating a Bond

Current Behaviour on Bond creation

Steps for a user to create a bond:

  1. Shutdown all VMs with VIFs using the interfaces that will be bonded, in order to unplug those VIFs.
  2. Create a Network to be used by the bond: Network.create
  3. Call Bond.create with a ref to this Network, a list of refs of @@ -107,12 +107,12 @@ drivers) on it.
  4. In a pool of two hosts, having VIFs/VLANs/management on the interfaces of the pool slave, create a bond on the pool master, and restart XAPI on the slave.
  5. Restart XAPI on a host with a networking configuration that has -become illegal due to these requirements.
\ No newline at end of file diff --git a/new-docs/design/coverage/index.html b/new-docs/design/coverage/index.html index 26eaa6e80f..2804f88a03 100644 --- a/new-docs/design/coverage/index.html +++ b/new-docs/design/coverage/index.html @@ -7,7 +7,7 @@ TL;DR To build binaries with coverage profiling, do:">Code Coverage Profiling :: XAPI Toolstack Developer Documentation -
Design document
Revisionv2
Status
Design document
Revisionv2
Statusproposed

Code Coverage Profiling

We would like to add optional coverage profiling to existing OCaml projects in the context of XenServer and XenAPI. This article presents how we do it.

Binaries instrumented for coverage profiling in the XenServer project @@ -149,12 +149,12 @@ coverage: _tags _tags.coverage test ! -f _tags.orig && mv _tags _tags.orig || true cat _tags.coverage _tags.orig > _tags -

\ No newline at end of file diff --git a/new-docs/design/cpu-levelling-v2/index.html b/new-docs/design/cpu-levelling-v2/index.html index 033d58398e..d03727c563 100644 --- a/new-docs/design/cpu-levelling-v2/index.html +++ b/new-docs/design/cpu-levelling-v2/index.html @@ -3,17 +3,17 @@ History Original XS 5.6 design: heterogeneous-pools Changes made in XS 5.6 FP1 for the DR feature (added CPUID checks upon migration) XS 6.1: migration checks extended for cross-pool scenario High-level Interfaces and Behaviour A VM can only be migrated safely from one host to another if both hosts offer the set of CPU features which the VM expects.">CPU feature levelling 2.0 :: XAPI Toolstack Developer Documentation -
Design document
Revisionv7
Status
Design document
Revisionv7
Statusreleased (7.0)
Revision history
v1Initial version
v2Add details about VM migration and import
v3Included and excluded use cases
v4Rolling Pool Upgrade use cases
v5Lots of changes to simplify the design
v6Use case refresh based on simplified design
v7RPU refresh based on simplified design

CPU feature levelling 2.0

Executive Summary

The old XS 5.6-style Heterogeneous Pool feature that is based around hardware-level CPUID masking will be replaced by a safer and more flexible software-based levelling mechanism.

History

  • Original XS 5.6 design: heterogeneous-pools
  • Changes made in XS 5.6 FP1 for the DR feature (added CPUID checks upon migration)
  • XS 6.1: migration checks extended for cross-pool scenario

High-level Interfaces and Behaviour

A VM can only be migrated safely from one host to another if both hosts offer the set of CPU features which the VM expects. If this is not the case, CPU features may appear or disappear as the VM is migrated, causing it to crash. The purpose of feature levelling is to hide features which the hosts do not have in common from the VM, so that it does not see any change in CPU capabilities when it is migrated.

Most pools start off with homogenous hardware, but over time it may become impossible to source new hosts with the same specifications as the ones already in the pool. The main use of feature levelling is to allow such newer, more capable hosts to be added to an existing pool while preserving the ability to migrate existing VMs to any host in the pool.

Principles for Migration

The CPU levelling feature aims to both:

  1. Make VM migrations safe by ensuring that a VM will see the same CPU features before and after a migration.
  2. Make VMs as mobile as possible, so that it can be freely migrated around in a XenServer pool.

To make migrations safe:

  • A migration request will be blocked if the destination host does not offer the some of the CPU features that the VM currently sees.
  • Any additional CPU features that the destination host is able to offer will be hidden from the VM.

Note: Due to the limitations of the old Heterogeneous Pools feature, we are not able to guarantee the safety of VMs that are migrated to a Levelling-v2 host from an older host, during a rolling pool upgrade. This is because such VMs may be using CPU features that were not captured in the old feature sets, of which we are therefore unaware. However, migrations between the same two hosts, but before the upgrade, may have already been unsafe. The promise is that we will not make migrations more unsafe during a rolling pool upgrade.

To make VMs mobile:

  • A VM that is started in a XenServer pool will be able to see only CPU features that are common to all hosts in the pool. The set of common CPU features is referred to in this document as the pool CPU feature level, or simply the pool level.

Use Cases for Pools

  1. A user wants to add a new host to an existing XenServer pool. The new host has all the features of the existing hosts, plus extra features which the existing hosts do not. The new host will be allowed to join the pool, but its extra features will be hidden from VMs that are started on the host or migrated to it. The join does not require any host reboots.

  2. A user wants to add a new host to an existing XenServer pool. The new host does not have all the features of the existing ones. XenCenter warns the user that adding the host to the pool is possible, but it would lower the pool’s CPU feature level. The user accepts this and continues the join. The join does not require any host reboots. VMs that are started anywhere on the pool, from now on, will only see the features of the new host (the lowest common denominator), such that they are migratable to any host in the pool, including the new one. VMs that were running before the pool join will not be migratable to the new host, because these VMs may be using features that the new host does not have. However, after a reboot, such VMs will be fully mobile.

  3. A user wants to add a new host to an existing XenServer pool. The new host does not have all the features of the existing ones, and at the same time, it has certain features that the pool does not have (the feature sets overlap). This is essentially a combination of the two use cases above, where the pool’s CPU feature level will be downgraded to the intersection of the feature sets of the pool and the new host. The join does not require any host reboots.

  4. A user wants to upgrade or repair the hardware of a host in an existing XenServer pool. After upgrade the host has all the features it used to have, plus extra features which other hosts in the pool do not have. The extra features are masked out and the host resumes its place in the pool when it is booted up again.

  5. A user wants to upgrade or repair the hardware of a host in an existing XenServer pool. After upgrade the host has fewer features than it used to have. When the host is booted up again, the pool CPU’s feature level will be automatically lowered, and the user will be alerted of this fact (through the usual alerting mechanism).

  6. A user wants to remove a host from an existing XenServer pool. The host will be removed as normal after any VMs on it have been migrated away. The feature set offered by the pool will be automatically re-levelled upwards in case the host which was removed was the least capable in the pool, and additional features common to the remaining hosts will be unmasked.

Rolling Pool Upgrade

  • A VM which was running on the pool before the upgrade is expected to continue to run afterwards. However, when the VM is migrated to an upgraded host, some of the CPU features it had been using might disappear, either because they are not offered by the host or because the new feature-levelling mechanism hides them. To have the best chance for such a VM to successfully migrate (see the note under “Principles for Migration”), it will be given a temporary VM-level feature set providing all of the destination’s CPU features that were unknown to XenServer before the upgrade. When the VM is rebooted it will inherit the pool-level feature set.

  • A VM which is started during the upgrade will be given the current pool-level feature set. The pool-level feature set may drop after the VM is started, as more hosts are upgraded and re-join the pool, however the VM is guaranteed to be able to migrate to any host which has already been upgraded. If the VM is started on the master, there is a risk that it may only be able to run on that host.

  • To allow the VMs with grandfathered-in flags to be migrated around in the pool, the intra pool VM migration pre-checks will compare the VM’s feature flags to the target host’s flags, not the pool flags. This will maximise the chance that a VM can be migrated somewhere in a heterogeneous pool, particularly in the case where only a few hosts in the pool do not have features which the VMs require.

  • To allow cross-pool migration, including to pool of a higher XenServer version, we will still check the VM’s requirements against the pool-level features of the target pool. This is to avoid the possibility that we migrate a VM to an ‘island’ in the other pool, from which it cannot be migrated any further.

XenAPI Changes

Fields

  • host.cpu_info is a field of type (string -> string) map that contains information about the CPUs in a host. It contains the following keys: cpu_count, socket_count, vendor, speed, modelname, family, model, stepping, flags, features, features_after_reboot, physical_features and maskable.
    • The following keys are specific to hardware-based CPU masking and will be removed: features_after_reboot, physical_features and maskable.
    • The features key will continue to hold the current CPU features that the host is able to use. In practise, these features will be available to Xen itself and dom0; guests may only see a subset. The current format is a string of four 32-bit words represented as four groups of 8 hexadecimal digits, separated by dashes. This will change to an arbitrary number of 32-bit words. Each bit at a particular position (starting from the left) still refers to a distinct CPU feature (1: feature is present; 0: feature is absent), and feature strings may be compared between hosts. The old format simply becomes a special (4 word) case of the new format, and bits in the same position may be compared between old and new feature strings.
    • The new key features_pv will be added, representing the subset of features that the host is able to offer to a PV guest.
    • The new key features_hvm will be added, representing the subset of features that the host is able to offer to an HVM guest.
  • A new field pool.cpu_info of type (string -> string) map (read only) will be added. It will contain:
    • vendor: The common CPU vendor across all hosts in the pool.
    • features_pv: The intersection of features_pv across all hosts in the pool, representing the feature set that a PV guest will see when started on the pool.
    • features_hvm: The intersection of features_hvm across all hosts in the pool, representing the feature set that an HVM guest will see when started on the pool.
    • cpu_count: the total number of CPU cores in the pool.
    • socket_count: the total number of CPU sockets in the pool.
  • The pool.other_config:cpuid_feature_mask override key will no longer have any effect on pool join or VM migration.
  • The field VM.last_boot_CPU_flags will be updated to the new format (see host.cpu_info:features). It will still contain the feature set that the VM was started with as well as the vendor (under the features and vendor keys respectively).

Messages

  • pool.join currently requires that the CPU vendor and feature set (according to host.cpu_info:vendor and host.cpu_info:features) of the joining host are equal to those of the pool master. This requirement will be loosened to mandate only equality in CPU vendor:
    • The join will be allowed if host.cpu_info:vendor equals pool.cpu_info:vendor.
    • This means that xapi will additionally allow hosts that have a more extensive feature set than the pool (as long as the CPU vendor is common). Such hosts are transparently down-levelled to the pool level (without needing reboots).
    • This further means that xapi will additionally allow hosts that have a less extensive feature set than the pool (as long as the CPU vendor is common). In this case, the pool is transparently down-levelled to the new host’s level (without needing reboots). Note that this does not affect any running VMs in any way; the mobility of running VMs will not be restricted, which can still migrate to any host they could migrate to before. It does mean that those running VMs will not be migratable to the new host.
    • The current error raised in case of a CPU mismatch is POOL_HOSTS_NOT_HOMOGENEOUS with reason argument "CPUs differ". This will remain the error that is raised if the pool join fails due to incompatible CPU vendors.
    • The pool.other_config:cpuid_feature_mask override key will no longer have any effect.
  • host.set_cpu_features and host.reset_cpu_features will be removed: it is no longer to use the old method of CPU feature masking (CPU feature sets are controlled automatically by xapi). Calls will fail with MESSAGE_REMOVED.
  • VM lifecycle operations will be updated internally to use the new feature fields, to ensure that:
    • Newly started VMs will be given CPU features according to the pool level for maximal mobility.
    • For safety, running VMs will maintain their feature set across migrations and suspend/resume cycles. CPU features will transparently be hidden from VMs.
    • Furthermore, migrate and resume will only be allowed in case the target host’s CPUs are capable enough, i.e. host.cpu_info:vendor = VM.last_boot_CPU_flags:vendor and host.cpu_info:features_{pv,hvm} ⊇ VM.last_boot_CPU_flags:features. A VM_INCOMPATIBLE_WITH_THIS_HOST error will be returned otherwise (as happens today).
    • For cross pool migrations, to ensure maximal mobility in the target pool, a stricter condition will apply: the VM must satisfy the pool CPU level rather than just the target host’s level: pool.cpu_info:vendor = VM.last_boot_CPU_flags:vendor and pool.cpu_info:features_{pv,hvm} ⊇ VM.last_boot_CPU_flags:features

CLI Changes

The following changes to the xe CLI will be made:

  • xe host-cpu-info (as well as xe host-param-list and friends) will return the fields of host.cpu_info as described above.
  • xe host-set-cpu-features and xe host-reset-cpu-features will be removed.
  • xe host-get-cpu-features will still return the value of host.cpu_info:features for a given host.

Low-level implementation

Xenctrl

The old xc_get_boot_cpufeatures hypercall will be removed, and replaced by two new functions, which are available to xenopsd through the Xenctrl module:

external get_levelling_caps : handle -> int64 = "stub_xc_get_levelling_caps"
 
 type featureset_index = Featureset_host | Featureset_pv | Featureset_hvm
 external get_featureset : handle -> featureset_index -> int64 array = "stub_xc_get_featureset"
-

In particular, the get_featureset function will be used by xapi/xenopsd to ask Xen which are the widest sets of CPU features that it can offer to a VM (PV or HVM). I don’t think there is a use for get_levelling_caps yet.

Xenopsd

  • Update the type Host.cpu_info, which contains all the fields that need to go into the host.cpu_info field in the xapi DB. The type already exists but is unused. Add the function HOST.get_cpu_info to obtain an instance of the type. Some code from xapi and the cpuid.ml from xen-api-libs can be reused.
  • Add a platform key featureset (Vm.t.platformdata), which xenopsd will write to xenstore along with the other platform keys (no code change needed in xenopsd). Xenguest will pick this up when a domain is created, and will apply the CPUID policy to the domain. This has the effect of masking out features that the host may have, but which have a 0 in the feature set bitmap.
  • Review current cpuid-related functions in xc/domain.ml.

Xapi

Xapi startup

  • Update Create_misc.create_host_cpu function to use the new xenopsd call.
  • If the host features fall below pool level, e.g. due to a change in hardware: down-level the pool by updating pool.cpu_info.features_{pv,hvm}. Newly started VMs will inherit the new level; already running VMs will not be affected, but will not be able to migrate to this host.
  • To notify the admin of this event, an API alert (message) will be set: pool_cpu_features_downgraded.

VM start

  • Inherit feature set from pool (pool.cpu_info.features_{pv,hvm}) and set VM.last_boot_CPU_flags (cpuid_helpers.ml).
  • The domain will be started with this CPU feature set enabled, by writing the feature set string to platformdata (see above).

VM migrate and resume

  • There are already CPU compatiblity checks on migration, both in-pool and cross-pool, as well as resume. Xapi compares VM.last_boot_CPU_flags of the VM to-migrate with host.cpu_info of the receiving host. Migration is only allowed if the CPU vendors and the same, and host.cpu_info:features ⊇ VM.last_boot_CPU_flags:features. The check can be overridden by setting the force argument to true.
  • For in-pool migrations, these checks will be updated to use the appropriate features_pv or features_hvm field.
  • For cross-pool migrations. These checks will be updated to use pool.cpu_info (features_pv or features_hvm depending on how the VM was booted) rather than host.cpu_info.
  • If the above checks pass, then the VM.last_boot_CPU_flags will be maintained, and the new domain will be started with the same CPU feature set enabled, by writing the feature set string to platformdata (see above).
  • In case the VM is migrated to a host with a higher xapi software version (e.g. a migration from a host that does not have CPU levelling v2), the feature string may be longer. This may happen during a rolling pool upgrade or a cross-pool migration, or when a suspended VM is resume after an upgrade. In this case, the following safety rules apply:
    • Only the existing (shorter) feature string will be used to determine whether the migration will be allowed. This is the best we can do, because we are unaware of the state of the extended feature set on the older host.
    • The existing feature set in VM.last_boot_CPU_flags will be extended with the extra bits in host.cpu_info:features_{pv,hvm}, i.e. the widest feature set that can possibly be granted to the VM (just in case the VM was using any of these features before the migration).
    • Strictly speaking, a migration of a VM from host A to B that was allowed before B was upgraded, may no longer be allowed after the upgrade, due to stricter feature sets in the new implementation (from the xc_get_featureset hypercall). However, the CPU features that are switched off by the new implementation are features that a VM would not have been able to actually use. We therefore need a don’t-care feature set (similar to the old pool.other_config:cpuid_feature_mask key) with bits that we may ignore in migration checks, and switch off after the migration. This will be a xapi config file option.
    • XXX: Can we actually block a cross-pool migration at the receiver end??

VM import

The VM.last_boot_CPU_flags field must be upgraded to the new format (only really needed for VMs that were suspended while exported; preserve_power_state=true), as described above.

Pool join

Update pool join checks according to the rules above (see pool.join), i.e. remove the CPU features constraints.

Upgrade

  • The pool level (pool.cpu_info) will be initialised when the pool master upgrades, and automatically adjusted if needed (downwards) when slaves are upgraded, by each upgraded host’s started sequence (as above under “Xapi startup”).
  • The VM.last_boot_CPU_flags fields of running and suspended VMs will be “upgraded” to the new format on demand, when a VM is migrated to or resume on an upgraded host, as described above.

XenCenter integration

  • Don’t explicitly down-level upon join anymore
  • Become aware of new pool join rule
  • Update Rolling Pool Upgrade
\ No newline at end of file diff --git a/new-docs/design/distributed-database/index.html b/new-docs/design/distributed-database/index.html index 07b9029c37..15b518201c 100644 --- a/new-docs/design/distributed-database/index.html +++ b/new-docs/design/distributed-database/index.html @@ -7,7 +7,7 @@ Both problems can be addressed by observing that the database objects correspond to distinct physical objects where eventual consistency is perfectly ok.">Distributed database :: XAPI Toolstack Developer Documentation -
Design document
Revisionv1
Status
Design document
Revisionv1
Statusproposed

Distributed database

All hosts in a pool use the shared database by sending queries to the pool master. This creates

  • a performance bottleneck as the pool size increases
  • a reliability problem when the master fails.

The reliability problem can be ameliorated by running with HA enabled, but this is not always possible.

Both problems can be addressed by observing that the database objects @@ -98,12 +98,12 @@ cpuid_feature_mask memory-ratio-hvm memory-ratio-pv $ cat xapi/pool/OpaqueRef\:39adc911-0c32-9e13-91a8-43a25939110b/other_config/cpuid_feature_mask -ffffff7f-ffffffff-ffffffff-ffffffff

Notice how:

  • every object is a directory
  • every key/value pair is represented as a file
\ No newline at end of file diff --git a/new-docs/design/emergency-network-reset/index.html b/new-docs/design/emergency-network-reset/index.html index 7aeaebb065..8724ed090b 100644 --- a/new-docs/design/emergency-network-reset/index.html +++ b/new-docs/design/emergency-network-reset/index.html @@ -7,7 +7,7 @@ A new XenAPI call Host.reset_networking, which removes all the PIFs, Bonds, VLANs and tunnels associated with the given host, and a call PIF.scan_bios to bring back the PIFs with device names as defined in the BIOS. A xe-reset-networking script that can be executed on a XenServer host, which prepares the reset and causes the host to reboot.">Emergency Network Reset Design :: XAPI Toolstack Developer Documentation -
Design document
Revisionv1
Status
Design document
Revisionv1
Statusreleased (6.0.2)

Emergency Network Reset Design

This document describes design details for the PR-1032 requirements.

The design consists of four parts:

  1. A new XenAPI call Host.reset_networking, which removes all the PIFs, Bonds, VLANs and tunnels associated with the given host, and a call PIF.scan_bios to bring back the PIFs with device names as @@ -47,12 +47,12 @@ Furthermore, in case the IP address of the master has changed, as a result of a network reset or Host.management_reconfigure, pool slaves may also use the network reset functionality to reconnect to -the master on its new IP.
\ No newline at end of file diff --git a/new-docs/design/emulated-pci-spec/index.html b/new-docs/design/emulated-pci-spec/index.html index b1bd078e77..766165811e 100644 --- a/new-docs/design/emulated-pci-spec/index.html +++ b/new-docs/design/emulated-pci-spec/index.html @@ -7,13 +7,13 @@ Hardcoded defaults are used for the vendor ID and all other parameters except device_id and revision. Only one emulated PCI device can be specified.">Specifying Emulated PCI Devices :: XAPI Toolstack Developer Documentation -
Design document
Revisionv1
Statusproposed

Specifying Emulated PCI Devices

Background and goals

At present (early March 2015) the datamodel defines a VM as having a “platform” string-string map, in which two keys are interpreted as specifying a PCI device which should be emulated for the VM. Those keys are “device_id” and “revision” (with int values represented as decimal strings).

Limitations:

  • Hardcoded defaults are used for the vendor ID and all other parameters except device_id and revision.
  • Only one emulated PCI device can be specified.

When instructing qemu to emulate PCI devices, qemu accepts twelve parameters for each device.

Future guest-agent features rely on additional emulated PCI devices. We cannot know in advance the full details of all the devices that will be needed, but we can predict some.

We need a way to configure VMs such that they will be given additional emulated PCI devices.

Design

In the datamodel, there will be a new type of object for emulated PCI devices.

Tentative name: “emulated_pci_device”

Fields to be passed through to qemu are the following, all static read-only, and all ints except devicename:

  • devicename (string)
  • vendorid
  • deviceid
  • command
  • status
  • revision
  • classcode
  • headertype
  • subvendorid
  • subsystemid
  • interruptline
  • interruptpin

We also need a “built_in” flag: see below.

Allow creation of these objects through the API (and CLI).

(It would be nice, but by no means essential, to be able to create one by specifying an existing one as a basis, along with one or more altered fields, e.g. “Make a new one just like that existing one except with interruptpin=9.”)

Create some of these devices to be defined as standard in XenServer, along the same lines as the VM templates. Those ones should have built_in=true.

Allow destruction of these objects through the API (and CLI), but not if they are in use or if they have built_in=true.

A VM will have a list of zero or more of these emulated-pci-device objects. (OPEN QUESTION: Should we forbid having more than one of a given device?)

Provide API (and CLI) commands to add and remove one of these devices from a VM (identifying the VM and device by uuid or other identifier such as name).

The CLI should allow performing this on multiple VMs in one go, based on a selector or filter for the VMs. We have this concept already in the CLI in commands such as vm-start.

In the function that adds an emulated PCI device to a VM, we must check if this is the first device to be added, and must refuse if the VM’s Virtual Hardware Platform Version is too low. (Or should we just raise the version automatically if needed?)

When starting a VM, check its list of emulated pci devices and pass the details through to qemu (via xenopsd).

\ No newline at end of file diff --git a/new-docs/design/fcoe-nics/index.html b/new-docs/design/fcoe-nics/index.html index 01733522d3..2a00c8309d 100644 --- a/new-docs/design/fcoe-nics/index.html +++ b/new-docs/design/fcoe-nics/index.html @@ -3,14 +3,14 @@ Introduction FCoE supported on a NIC is a hardware property. With the help of dcbtool, we can identify which NIC support FCoE. The new field capabilities will be Set(String) in PIF object. For FCoE capable NIC will have string “fcoe” in PIF capabilities field. capabilities field will be ReadOnly, This field cannot be modified by user.">FCoE capable NICs :: XAPI Toolstack Developer Documentation -
Design document
Revisionv3
Status
Design document
Revisionv3
Statusproposed
Review#120

FCoE capable NICs

It has been possible to identify the NICs of a Host which can support FCoE. -This property can be listed in PIF object under capabilities field.

Introduction

  • FCoE supported on a NIC is a hardware property. With the help of dcbtool, we can identify which NIC support FCoE.
  • The new field capabilities will be Set(String) in PIF object. For FCoE capable NIC will have string “fcoe” in PIF capabilities field.
  • capabilities field will be ReadOnly, This field cannot be modified by user.

PIF Object

New field:

  • Field PIF.capabilities will be type Set(string).
  • Default value in PIF capabilities will have an empty set.

Xapi Changes

  • Set the field capabilities “fcoe” depending on output of xcp-networkd call get_capabilities.
  • Field capabilities “fcoe” can be set during introduce_internal on when creating a PIF.
  • Field capabilities “fcoe” can be updated during refresh_all on xapi startup.
  • The above field will be set everytime when xapi-restart.

XCP-Networkd Changes

New function:

  • String list string list get_capabilties (string)
  • Argument: device_name for the PIF.
  • This function calls method capable exposed by fcoe_driver.py as part of dom0.
  • It returns string list [“fcoe”] or [] depending on capable method output.

Defaults, Installation and Upgrade

  • Any newly introduced PIF will have its capabilities field as empty set until fcoe_driver method capable states FCoE is supported on the NIC.
  • It includes PIFs obtained after a fresh install of Xenserver, as well as PIFs created using PIF.introduce then PIF.scan.
  • During an upgrade Xapi Restart will call refresh_all which then populate the capabilities field as empty set.

Command Line Interface

  • The PIF.capabilities field is exposed through xe pif-list and xe pif-param-list as usual.
\ No newline at end of file diff --git a/new-docs/design/gpu-passthrough/index.html b/new-docs/design/gpu-passthrough/index.html index 9bc83074ea..8f55a37c05 100644 --- a/new-docs/design/gpu-passthrough/index.html +++ b/new-docs/design/gpu-passthrough/index.html @@ -3,7 +3,7 @@ Overview Rather than modelling GPU pass-through from a PCI perspective, and having the user manipulate PCI devices directly, we are taking a higher-level view by introducing a dedicated graphics model. The graphics model is similar to the networking and storage model, in which virtual and physical devices are linked through an intermediate abstraction layer (e.">GPU pass-through support :: XAPI Toolstack Developer Documentation -
Design document
Revisionv1
Status
Design document
Revisionv1
Statusreleased (6.0)

GPU pass-through support

This document contains the software design for GPU pass-through. This code was originally included in the version of Xapi used in XenServer 6.0.

Overview

Rather than modelling GPU pass-through from a PCI perspective, and having the user manipulate PCI devices directly, we are taking a @@ -87,12 +87,12 @@ stable list of device types (as in /usr/share/hwdata/pci.ids).

  • The device and vendor ID+name (currently, for PIFs, xapi looks up the name in /usr/share/hwdata/pci.ids).
  • Which other devices/functions are required to be passed through to the same VM (co-located), e.g. other functions of a compound PCI -device.

    • Command-Line Interface (xe)

    • xe pgpu-list
    • xe pgpu-param-list/get/set/add/remove/clear
    • xe gpu-group-list
    • xe gpu-group-param-list/get/set/add/remove/clear
    • xe vgpu-list
    • xe vgpu-create
    • xe vgpu-destroy
    • xe vgpu-param-list/get/set/add/remove/clear
    • xe host-param-get param-name=chipset-info param-key=iommu
    \ No newline at end of file diff --git a/new-docs/design/gpu-support-evolution/index.html b/new-docs/design/gpu-support-evolution/index.html index fbb44ab4bc..2706f7d9d9 100644 --- a/new-docs/design/gpu-support-evolution/index.html +++ b/new-docs/design/gpu-support-evolution/index.html @@ -7,7 +7,7 @@ qemu is launched either with no special parameter, if the basic Cirrus graphics processor is required, otherwise qemu is launched with the -std-vga flag.">GPU support evolution :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv3
    Status
    Design document
    Revisionv3
    Statusreleased (7.0)
    Revision history
    v1Documented interface changes between xapi and xenopsd for vGPU
    v2Added design for storing vGPU-to-pGPU allocation in xapi database
    v3Marked new xapi DB fields as internal-only

    GPU support evolution

    Introduction

    As of XenServer 6.5, VMs can be provisioned with access to graphics processors (either emulated or passed through) in four different ways. Virtualisation of Intel graphics processors will exist as a fifth kind of graphics processing @@ -114,12 +114,12 @@ rather than copying it to VM.platform.

    To support other vGPU implementations, we will add another internal-only database field:

    • VGPU_type.implementation enum(Passthrough|Nvidia|GVT_g)

    For the GVT_g implementation, no config file is needed. Instead, VGPU_type.internal_config will contain three key-value pairs, with the keys

    • vgt_low_gm_sz
    • vgt_high_gm_sz
    • vgt_fence_sz

    The values of these pairs will be used to construct a value of type -Xenops_interface.Vgpu.gvt_g, which will be passed down to xenopsd.

    \ No newline at end of file diff --git a/new-docs/design/heterogeneous-pools/index.html b/new-docs/design/heterogeneous-pools/index.html index b5db40b773..7e4b52af2b 100644 --- a/new-docs/design/heterogeneous-pools/index.html +++ b/new-docs/design/heterogeneous-pools/index.html @@ -1,5 +1,5 @@ Heterogeneous pools :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv1
    Status
    Design document
    Revisionv1
    Statusreleased (5.6)

    Heterogeneous pools

    Notes

    • The cpuid instruction is used to obtain a CPU’s manufacturer, family, model, stepping and features information.
    • The feature bitvector is 128 bits wide: 2 times 32 bits of base features plus 2 times 32 bits of extended features, which are @@ -97,12 +97,12 @@ feature masks, after which the machine will again boot normally (note: in our set-up, there is first a PXE boot prompt; the second prompt is the one we mean here).

      The API/CLI documentation should stress the potential danger of using -this functionality, and explain how to get out of trouble again.

    \ No newline at end of file diff --git a/new-docs/design/index.html b/new-docs/design/index.html index 05641c653d..0616bde23e 100644 --- a/new-docs/design/index.html +++ b/new-docs/design/index.html @@ -1,5 +1,5 @@ Design Documents :: XAPI Toolstack Developer Documentation -

    Design Documents

    Key: Revision +

    Design Documents

    Key: Revision Proposed Confirmed Released (vA.B) @@ -126,12 +126,12 @@ label-success">released (8.2)
    RDP control v2 released (xenserver 6.5 sp1)
    \ No newline at end of file + 
    \ No newline at end of file diff --git a/new-docs/design/index.print.html b/new-docs/design/index.print.html index 519f0f61ae..5c1c4cc0d2 100644 --- a/new-docs/design/index.print.html +++ b/new-docs/design/index.print.html @@ -1,5 +1,5 @@ Design Documents :: XAPI Toolstack Developer Documentation -

    Design Documents

    Key: Revision +

    Design Documents

    Key: Revision Proposed Confirmed Released (vA.B) @@ -2436,4 +2436,4 @@ create a new VGPU_type object.

    Design document
    Revisionv1
    Statusreleased (7.0)

    Virtual Hardware Platform Version

    Background and goal

    Some VMs can only be run on hosts of sufficiently recent versions.

    We want a clean way to ensure that xapi only tries to run a guest VM on a host that supports the “virtual hardware platform” required by the VM.

    Suggested design

    • In the datamodel, VM has a new integer field “hardware_platform_version” which defaults to zero.
    • In the datamodel, Host has a corresponding new integer-list field “virtual_hardware_platform_versions” which defaults to list containing a single zero element (i.e. [0] or [0L] in OCaml notation). The zero represents the implicit version supported by older hosts that lack the code to handle the Virtual Hardware Platform Version concept.
    • When a host boots it populates its own entry from a hardcoded value, currently [0; 1] i.e. a list containing the two integer elements 0 and 1. (Alternatively this could come from a config file.)
      • If this new version-handling functionality is introduced in a hotfix, at some point the pool master will have the new functionality while at least one slave does not. An old slave-host that does not yet have software to handle this feature will not set its DB entry, which will therefore remain as [0] (maintained in the DB by the master).
    • The existing test for whether a VM can run on (or migrate to) a host must include a check that the VM’s virtual hardware platform version is in the host’s list of supported versions.
    • When a VM is made to start using a feature that is available only in a certain virtual hardware platform version, xapi must set the VM’s hardware_platform_version to the maximum of that version-number and its current value (i.e. raise if needed).

    For the version we could consider some type other than integer, but a strict ordering is needed.

    First use-case

    Version 1 denotes support for a certain feature:

    When a VM starts, if a certain flag is set in VM.platform then XenServer will provide an emulated PCI device which will trigger the guest Windows OS to seek drivers for the device, or updates for those drivers. Thus updated drivers can be obtained through the standard Windows Update mechanism.

    If the PCI device is removed, the guest OS will fail to boot. A VM using this feature must not be migrated to or started on a XenServer that lacks support for the feature.

    Therefore at VM start, we can look at whether this feature is being used; if it is, then if the VM’s Virtual Hardware Platform Version is less than 1 we should raise it to 1.

    Limitation

    Consider a VM that requires version 1 or higher. Suppose it is exported, then imported into an old host that does not support this feature. Then the host will not check the versions but will attempt to run the VM, which will then have difficulties.

    The only way to prevent this would be to make a backwards-incompatible change to the VM metadata (e.g. a new item in an enum) so that the old hosts cannot read it, but that seems like a bad idea.

    Design document
    Revisionv2
    Statusproposed

    XenPrep

    Background

    Windows guests should have XenServer-specific drivers installed. As of mid-2015 these have been always been installed and upgraded by an essentially manual process involving an ISO carrying the drivers. We have a plan to enable automation through the standard Windows Update mechanism. This will involve a new additional virtual PCI device being provided to the VM, to trigger Windows Update to fetch drivers for the device.

    There are many existing Windows guests that have drivers installed already. These drivers must be uninstalled before the new drivers are installed (and ideally before the new PCI device is added). To make this easier, we are planning a XenAPI call that will cause the removal of the old drivers and the addition of the new PCI device.

    Since this is only to help with updating old guests, the call may well be removed at some point in the future.

    Brief high-level design

    The XenAPI call will be called VM.xenprep_start. It will update the VM record to note that the process has started, and will insert a special ISO into the VM’s virtual CD drive.

    That ISO will contain a tool which will be set up to auto-run (if auto-run is enabled in the guest). The tool will:

    1. Lock the CD drive so other Windows programs cannot eject the disc.
    2. Uninstall the old drivers.
    3. Eject the CD to signal success.
    4. Shut down the VM.

    XenServer will interpret the ejection of the CD as a success signal, and when the VM shuts down without the special ISO in the drive, XenServer will:

    1. Update the VM record:
    • Remove the mark that shows that the xenprep process is in progress
    • Give it the new PCI device: set VM.auto_update_drivers to true.
    • If VM.virtual_hardware_platform_version is less than 2, then set it to 2.
    1. Start the VM.

    More details of the xapi-project parts

    (The tool that runs in the guest is out of scope for this document.)

    Start

    The XenAPI call VM.xenprep_start will throw a power-state error if the VM is not running. -For RBAC roles, it will be available to “VM Operator” and above.

    It will:

    1. Insert the xenprep ISO into the VM’s virtual CD drive.
    2. Write VM.other_config key xenprep_progress=ISO_inserted to record the fact that the xenprep process has been initiated.

    If xenprep_start is called on a VM already undergoing xenprep, the call will return successfully but will not do anything.

    If the VM does not have an empty virtual CD drive, the call will fail with a suitable error.

    Cancellation

    While xenprep is in progress, any request to eject the xenprep ISO (except from inside the guest) will be rejected with a new error “VBD_XENPREP_CD_IN_USE”.

    There will be a new XenAPI call VM.xenprep_abort which will:

    1. Remove the xenprep_progress entry from VM.other_config.
    2. Make a best-effort attempt to eject the CD. (The guest might prevent ejection.)

    This is not intended for cancellation while the xenprep tool is running, but rather for use before it starts, for example if auto-run is disabled or if the VM has a non-Windows OS.

    Completion

    Aim: when the guest shuts down after ejecting the CD, XenServer will start the guest again with the new PCI device.

    Xapi works through the queue of events it receives from xenopsd. It is possible that by the time xapi processes the cd-eject event, the guest might have shut down already.

    When the shutdown (not reboot) event is handled, we shall check whether we need to do anything xenprep-related. If

    • The VM other_config map has xenprep_progress as either of ISO_inserted or shutdown, and
    • The xenprep ISO is no longer in the drive

    then we must (in the specified order)

    1. Update the VM record:
    2. In VM.other_config set xenprep_progress=shutdown
    3. If VM.virtual_hardware_platform_version is less than 2, then set it to 2.
    4. Give it the new PCI device: set VM.auto_update_drivers to true.
    5. Initiate VM start.
    6. Remove xenprep_progress from VM.other_config

    The most relevant code is probably the update_vm function in ocaml/xapi/xapi_xenops.ml in the xen-api repo (or in some function called from there).

    \ No newline at end of file +For RBAC roles, it will be available to “VM Operator” and above.

    It will:

    1. Insert the xenprep ISO into the VM’s virtual CD drive.
    2. Write VM.other_config key xenprep_progress=ISO_inserted to record the fact that the xenprep process has been initiated.

    If xenprep_start is called on a VM already undergoing xenprep, the call will return successfully but will not do anything.

    If the VM does not have an empty virtual CD drive, the call will fail with a suitable error.

    Cancellation

    While xenprep is in progress, any request to eject the xenprep ISO (except from inside the guest) will be rejected with a new error “VBD_XENPREP_CD_IN_USE”.

    There will be a new XenAPI call VM.xenprep_abort which will:

    1. Remove the xenprep_progress entry from VM.other_config.
    2. Make a best-effort attempt to eject the CD. (The guest might prevent ejection.)

    This is not intended for cancellation while the xenprep tool is running, but rather for use before it starts, for example if auto-run is disabled or if the VM has a non-Windows OS.

    Completion

    Aim: when the guest shuts down after ejecting the CD, XenServer will start the guest again with the new PCI device.

    Xapi works through the queue of events it receives from xenopsd. It is possible that by the time xapi processes the cd-eject event, the guest might have shut down already.

    When the shutdown (not reboot) event is handled, we shall check whether we need to do anything xenprep-related. If

    • The VM other_config map has xenprep_progress as either of ISO_inserted or shutdown, and
    • The xenprep ISO is no longer in the drive

    then we must (in the specified order)

    1. Update the VM record:
    2. In VM.other_config set xenprep_progress=shutdown
    3. If VM.virtual_hardware_platform_version is less than 2, then set it to 2.
    4. Give it the new PCI device: set VM.auto_update_drivers to true.
    5. Initiate VM start.
    6. Remove xenprep_progress from VM.other_config

    The most relevant code is probably the update_vm function in ocaml/xapi/xapi_xenops.ml in the xen-api repo (or in some function called from there).

    \ No newline at end of file diff --git a/new-docs/design/integrated-gpu-passthrough/index.html b/new-docs/design/integrated-gpu-passthrough/index.html index 08b577d8bf..7c48ecbf0f 100644 --- a/new-docs/design/integrated-gpu-passthrough/index.html +++ b/new-docs/design/integrated-gpu-passthrough/index.html @@ -7,7 +7,7 @@ PGPU.dom0_access enum(enabled|disable_on_reboot|disabled|enable_on_reboot) host.">Integrated GPU passthrough support :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv3
    Status
    Design document
    Revisionv3
    Statusreleased (6.5 sp1)
    Review#33

    Integrated GPU passthrough support

    Introduction

    Passthrough of discrete GPUs has been available since XenServer 6.0. With some extensions, we will also be able to support passthrough of integrated @@ -31,12 +31,12 @@ determined from VM.platform:vga. xapi will consider a GPU to be integrated if both:

    1. It resides on bus 0.
    2. The vendor ID of the device is contained in a whitelist provided by xapi’s config file.

    When xenopsd starts qemu for a VM with a video_card of type IGD_passthrough, -it will pass the flags “-std-vga” AND “-gfx_passthru”.

    \ No newline at end of file diff --git a/new-docs/design/local-database/index.html b/new-docs/design/local-database/index.html index 5200db5c0c..eb83ed1c60 100644 --- a/new-docs/design/local-database/index.html +++ b/new-docs/design/local-database/index.html @@ -3,7 +3,7 @@ The reliability problem can be avoided by running with HA or the redo log enabled, but this is not always possible.">Local database :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv1
    Status
    Design document
    Revisionv1
    Statusproposed

    Local database

    All hosts in a pool use the shared database by sending queries to the pool master. This creates a performance bottleneck as the pool size increases. All hosts in a pool receive a database backup from @@ -34,12 +34,12 @@ is only a problem when we are using the database for locking and are attempting to hand over a lock to another host. We are already using XenAPI calls forwarded to the master for some of this, but may need to do a bit more of this; in -particular the storage backends may need some updating.

    \ No newline at end of file diff --git a/new-docs/design/management-interface-on-vlan/index.html b/new-docs/design/management-interface-on-vlan/index.html index 499d63b42d..0d78386bf5 100644 --- a/new-docs/design/management-interface-on-vlan/index.html +++ b/new-docs/design/management-interface-on-vlan/index.html @@ -7,7 +7,7 @@ Check the PIFs created on a Host for physical devices eth0, eth1. xe pif-list params=uuid physical=true host-uuid=UUID this will list pif-UUID Create a new network for the VLAN interface.">Management Interface on VLAN :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv3
    Status
    Design document
    Revisionv3
    Statusproposed
    Revision history
    v1Initial version
    v2Addition of `networkd_db` update for Upgrade
    v3More info on `networkd_db` and API Errors

    Management Interface on VLAN

    This document describes design details for the REQ-42: Support Use of VLAN on XAPI Management Interface.

    XAPI and XCP-Networkd

    Creating a VLAN

    Creating a VLAN is already there, Lisiting the steps to create a VLAN which is used later in the document. Steps:

    1. Check the PIFs created on a Host for physical devices eth0, eth1. @@ -52,12 +52,12 @@ We would assume that VLAN is available on the switch/network.

      Manual steps to be performed before calling new API:

      1. Create a vlan network on pool via network.create, In case of pyhsical NICs network must be present.
      2. Create a vlan pif on each host via VLAN.create using above network ref, physical PIF ref and vlanID, Not needed in case of pyhsical network. Or An Alternate call pool.create_VLAN providing device and above network will create vlan PIFs for all hosts in a pool.
      3. Perform PIF.reconfigure_ip for each new Network PIF on each Host.

      If User wishes to change the management interface manually on each Host in a Pool, We should allow it, There will be a guideline for that:

      User can individually change management interface on each host calling Host.management_reconfigure using pifs on physical devices or vlan pifs. This must be perfomed on slaves first and lastly on Master, As changing management_interface on master will disconnect slaves from master then further calls Host.management_reconfigure cannot be performed till master recover slaves via call pool.recover_slaves.

      API Details

      • Pool.management_reconfigure
        • Parameter: network reference network.
        • Calling this function configures management_interface on each host of a pool.
        • For the network provided it will check pifs are present on each Host, -In case of VLAN network it will check vlan pifs on provided network are present on each Host of Pool.
        • Check IP is configured on above pifs on each Host.
        • If PIFs are not present or IP is not configured on PIFs this call must fail gracefully, Asking user to configure them.
        • Call Host.management_reconfigure on each slave then lastly on master.
        • Call pool.recover_slaves on master inorder to recover slaves which might have lost the connection to master.

      API errors

      Possible API errors that may be raised by pool.management_reconfigure:

      • INTERFACE_HAS_NO_IP : the specified PIF (pif parameter) has no IP configuration. The new API checks for all PIFs on the new Network has IP configured. There might be a case when user has forgotten to configure IP on PIF on one or many of the Hosts in a Pool.

      New API ERROR:

      • REQUIRED_PIF_NOT_PRESENT : the specified Network (network parameter) has no PIF present on the host in pool. There might be a case when user has forgotten to create vlan pif on one or many of the Hosts in a Pool.

      CP-Tickets

      1. CP-14027
      2. CP-14028
      3. CP-14029
      4. CP-14030
      5. CP-14031
      6. CP-14032
      7. CP-14033
    \ No newline at end of file diff --git a/new-docs/design/multiple-cluster-managers/index.html b/new-docs/design/multiple-cluster-managers/index.html index 140d57758d..9718bd0cab 100644 --- a/new-docs/design/multiple-cluster-managers/index.html +++ b/new-docs/design/multiple-cluster-managers/index.html @@ -3,14 +3,14 @@ modify the other software to take membership information from xapi; or modify xapi to take membership information from this other software.">Multiple Cluster Managers :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv2
    Status
    Design document
    Revisionv2
    Statusconfirmed
    Revision history
    v1Initial revision
    v2Short-term simplications and scope reduction

    Multiple Cluster Managers

    Introduction

    Xapi currently uses a cluster manager called xhad. Sometimes other software comes with its own built-in way of managing clusters, which would clash with xhad (example: xhad could choose to fence node ‘a’ while the other system could fence node ‘b’ resulting in a total failure). To integrate xapi with this other software we have 2 choices:

    1. modify the other software to take membership information from xapi; or
    2. modify xapi to take membership information from this other software.

    This document proposes a way to do the latter.

    XenAPI changes

    New field

    We will add the following new field:

    • pool.ha_cluster_stack of type string (read-only)
      • If HA is enabled, this field reflects which cluster stack is in use.
      • Set to "xhad" on upgrade, which implies that so far we have used XenServer’s own cluster stack, called xhad.

    Cluster-stack choice

    We assume for now that a particular cluster manager will be mandated (only) by certain types of clustered storage, recognisable by SR type (e.g. OCFS2 or Melio). The SR backend will be able to inform xapi if the SR needs a particular cluster stack, and if so, what is the name of the stack.

    When pool.enable_ha is called, xapi will determine which cluster stack to use based on the presence or absence of such SRs:

    • If an SR that needs its own cluster stack is attached to the pool, then xapi will use that cluster stack.
    • If no SR that needs a particular cluster stack is attached to the pool, then xapi will use xhad.

    If multiple SRs that need a particular cluster stack exist, then the storage parts of xapi must ensure that no two such SRs are ever attached to a pool at the same time.

    New errors

    We will add the following API error that may be raised by pool.enable_ha:

    • INCOMPATIBLE_STATEFILE_SR: the specified SRs (heartbeat_srs parameter) are not of the right type to hold the HA statefile for the cluster_stack that will be used. For example, there is a Melio SR attached to the pool, and therefore the required cluster stack is the Melio one, but the given heartbeat SR is not a Melio SR. The single parameter will be the name of the required SR type.

    The following new API error may be raised by PBD.plug:

    • INCOMPATIBLE_CLUSTER_STACK_ACTIVE: the operation cannot be performed because an incompatible cluster stack is active. The single parameter will be the name of the required cluster stack. This could happen (or example) if you tried to create an OCFS2 SR with XenServer HA already enabled.

    Future extensions

    In future, we may add a parameter to explicitly choose the cluster stack:

    • New parameter to pool.enable_ha called cluster_stack of type string which will have the default value of empty string (meaning: let the implementation choose).
    • With the additional parameter, pool.enable_ha may raise two new errors:
      • UNKNOWN_CLUSTER_STACK: -The operation cannot be performed because the requested cluster stack does not exist. The user should check the name was entered correctly and, failing that, check to see if the software is installed. The exception will have a single parameter: the name of the cluster stack which was not found.
      • CLUSTER_STACK_CONSTRAINT: HA cannot be enabled with the provided cluster stack because some third-party software is already active which requires a different cluster stack setting. The two parameters are: a reference to an object (such as an SR) which has created the restriction, and the name of the cluster stack that this object requires.

    Implementation

    The xapi.conf file will have a new field: cluster-stack-root which will have the default value /usr/libexec/xapi/cluster-stack. The existing xhad scripts and tools will be moved to /usr/libexec/xapi/cluster-stack/xhad/. A hypothetical cluster stack called foo would be placed in /usr/libexec/xapi/cluster-stack/foo/.

    In Pool.enable_ha with cluster_stack="foo" we will verify that the subdirectory <cluster-stack-root>/foo exists. If it does not exist, then the call will fail with UNKNOWN_CLUSTER_STACK.

    Alternative cluster stacks will need to conform to the exact same interface as xhad.

    \ No newline at end of file diff --git a/new-docs/design/multiple-device-emulators/index.html b/new-docs/design/multiple-device-emulators/index.html index 972aa50f44..98ff6bb01b 100644 --- a/new-docs/design/multiple-device-emulators/index.html +++ b/new-docs/design/multiple-device-emulators/index.html @@ -3,7 +3,7 @@ ioreq-server is currently used in XenServer to support vGPU, where it is configured via the legacy toolstack interface.">Multiple device emulators :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv1
    Status
    Design document
    Revisionv1
    Statusproposed

    Multiple device emulators

    Xen’s ioreq-server feature allows for several device emulator processes to be attached to the same domain, each emulating different sets of virtual hardware. This makes it possible, for example, to @@ -32,12 +32,12 @@ required to support vGPU through the generic device model mechanism. Changes would be required if we decided to expose the additional device models through the API, but in the near future it is more likely that -any additional device models will be dealt with entirely by xenopsd.

    \ No newline at end of file diff --git a/new-docs/design/numa/index.html b/new-docs/design/numa/index.html index 0177af1d60..302bfd438f 100644 --- a/new-docs/design/numa/index.html +++ b/new-docs/design/numa/index.html @@ -1,5 +1,5 @@ NUMA :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv1
    Status
    Design document
    Revisionv1
    Statusproposed

    NUMA

    NUMA

    NUMA stands for Non-Uniform Memory Access and describes that RAM access for CPUs in a large system is not equally fast for all of them. CPUs are grouped into so-called nodes and each node has fast access to RAM @@ -51,12 +51,12 @@ (VM.numa_nodes = 1) or (VM.numa_nodes < physinfo.Xenctrl.nr_nodes)

    Details

    The three new fields that become part of type VM.state are updated as -part of get_state() using the primitives above.

    \ No newline at end of file diff --git a/new-docs/design/ocfs2/index.html b/new-docs/design/ocfs2/index.html index d95a232984..c796c643bb 100644 --- a/new-docs/design/ocfs2/index.html +++ b/new-docs/design/ocfs2/index.html @@ -11,7 +11,7 @@ The following diagram shows the proposed architecture with xapi: Please note the following: OCFS2 is configured to use global heartbeats rather than per-mount heartbeats because we quite often have many SRs and therefore many mountpoints The OCFS2 global heartbeat should be collocated on the same SR as the XenServer HA SR so that we depend on fewer SRs (the storage is a single point of failure for OCFS2) The OCFS2 global heartbeat should itself be a raw VDI within an LVHDSR.">OCFS2 storage :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv1
    Status
    Design document
    Revisionv1
    Statusproposed

    OCFS2 storage

    OCFS2 is a (host-)clustered filesystem which runs on top of a shared raw block device. Hosts using OCFS2 form a cluster using a combination of network and storage heartbeats and host fencing to avoid split-brain.

    The following diagram shows the proposed architecture with xapi:

    Proposed architecture

    Please note the following:

    • OCFS2 is configured to use global heartbeats rather than per-mount heartbeats @@ -182,12 +182,12 @@ This means that VMs with more than 2 snapshots will have their snapshots deleted; it means you need to provision another storage target, temporarily doubling your storage needs; and it will take a long time.
    • There will now be 2 different reasons why a host has fenced which the -admin needs to understand.
    \ No newline at end of file diff --git a/new-docs/design/patches-in-vdis/index.html b/new-docs/design/patches-in-vdis/index.html index 0a27ac8186..913ae1dc10 100644 --- a/new-docs/design/patches-in-vdis/index.html +++ b/new-docs/design/patches-in-vdis/index.html @@ -3,7 +3,7 @@ Instead of writing patches to the dom0 filesystem, we shall write them to disk images (VDIs) instead.">patches in VDIs :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv1
    Status
    Design document
    Revisionv1
    Statusproposed

    patches in VDIs

    “Patches” are signed binary blobs which can be queried and applied. They are stored in the dom0 filesystem under /var/patch. Unfortunately the patches can be quite large – imagine a repo full of RPMs – and @@ -34,12 +34,12 @@ If the patch is still in the master’s dom0 filesystem then it will fall back to the HTTP handler.

    Summary of the impact on the admin

    • There will no longer be a size limit on hotfixes imposed by the mechanism itself.
    • There must be enough free space in an SR connected to the host to be able -to apply a patch on that host.
    \ No newline at end of file diff --git a/new-docs/design/pci-passthrough/index.html b/new-docs/design/pci-passthrough/index.html index 18556eef3d..d7871e60e9 100644 --- a/new-docs/design/pci-passthrough/index.html +++ b/new-docs/design/pci-passthrough/index.html @@ -19,7 +19,7 @@ PCI.disable_dom0_access PCI.get_dom0_access_status: compares the outputs of /opt/xensource/libexec/xen-cmdline and /proc/cmdline to produce one of the four values that can be currently contained in the PGPU.dom0_access field: disabled disabled_on_reboot enabled enabled_on_reboot How do determine the expected dom0 access state: If the device id is present in both pciback.">PCI passthrough support :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv1
    Status
    Design document
    Revisionv1
    Statusproposed

    PCI passthrough support

    Introduction

    GPU passthrough is already available in XAPI, this document proposes to also offer passthrough for all PCI devices through XAPI.

    Design proposal

    New methods for PCI object:

    • PCI.enable_dom0_access

    • PCI.disable_dom0_access

    • PCI.get_dom0_access_status: compares the outputs of /opt/xensource/libexec/xen-cmdline and /proc/cmdline to produce one of the four values that can be currently contained @@ -28,12 +28,12 @@ If the device id is present not in both pciback.hide of /proc/cmdline and xen-cmdline: disabled If the device id is present in the pciback.hide of /proc/cmdline but not in the one of xen-cmdline: disabled_on_reboot If the device id is not present in the pciback.hide of /proc/cmdline but is in the one of xen-cmdline: enabled_on_reboot

      A function rather than a field makes the data always accurate and even accounts for -changes made by users outside XAPI, directly through /opt/xensource/libexec/xen-cmdline

    With these generic methods available, the following field and methods will be deprecated:

    • PGPU.enable_dom0_access
    • PGPU.disable_dom0_access
    • PGPU.dom0_access (DB field)

    They would still be usable and up to date with the same info as for the PCI methods.

    Test cases

    • hide a PCI:

      • call PCI.disable_dom0_access on an enabled PCI
      • check the PCI goes in state disabled_on_reboot
      • reboot the host
      • check the PCI goes in state disabled

    • unhide a PCI:

      • call PCI.enable_dom0_access on an disabled PCI
      • check the PCI goes in state enabled_on_reboot
      • reboot the host
      • check the PCI goes in state enabled

    • get a PCI dom0 access state:

      • on a enabled PCI, make sure the get_dom0_access_status returns enabled
      • hide the PCI
      • make sure the get_dom0_access_status returns disabled_on_reboot
      • reboot
      • make sure the get_dom0_access_status returns disabled
      • unhide the PCI
      • make sure the get_dom0_access_status returns enabled_on_reboot
      • reboot
      • make sure the get_dom0_access_status returns enabled

    • Check PCI/PGPU dom0 access coherence:

      • hide a PCI belonging to a PGPU and make sure both states remains coherent at every step
      • unhide a PCI belonging to a PGPU and make sure both states remains coherent at every step
      • hide a PGPU and make sure its and its PCI’s states remains coherent at every step
      • unhide a PGPU and make sure its and its PCI’s states remains coherent at every step
    \ No newline at end of file diff --git a/new-docs/design/pif-properties/index.html b/new-docs/design/pif-properties/index.html index e66666a765..ababb71e7e 100644 --- a/new-docs/design/pif-properties/index.html +++ b/new-docs/design/pif-properties/index.html @@ -7,7 +7,7 @@ xe pif-param-set uuid= other-config:ethtool-gro=on xe pif-plug uuid= The other-config field is a grab-bag of options that are not clearly defined. The options exposed through other-config are mostly experimental features, and the interface is not considered stable.">GRO and other properties of PIFs :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv1
    Status
    Design document
    Revisionv1
    Statusreleased (6.5)

    GRO and other properties of PIFs

    It has been possible to enable and disable GRO and other “ethtool” features on PIFs for a long time, but there was never an official API for it. Now there is.

    Introduction

    The former way to enable GRO via the CLI is as follows:

    xe pif-param-set uuid=<pif-uuid> other-config:ethtool-gro=on
 xe pif-plug uuid=<pif-uuid>
@@ -16,12 +16,12 @@
 the interface is not considered stable. Furthermore, the field is read/write
 and does not have any input validation, and cannot not trigger any actions
 immediately. The latter is why it is needed to call pif-plug after setting
-the ethtool-gro key, in order to actually make things happen.
    New API
    New field:
    • Field PIF.properties of type (string -> string) map.
    • Physical and bond PIFs have a gro key in their properties, with possible values on and off. There are currently no other properties defined.
    • VLAN and Tunnel PIFs do not have any properties. They implicitly inherit the properties from the PIF they are based upon (either a physical PIF or a bond).
    • For backwards compatibility, if there is a other-config:ethtool-gro key present on the PIF, it will be treated as an override of the gro key in PIF.properties.
    New function:
    • Message void PIF.set_property (PIF ref, string, string).
    • First argument: the reference of the PIF to act on.
    • Second argument: the key to change in the properties field.
    • Third argument: the value to write.
    • The function can only be used on physical PIFs that are not bonded, and on bond PIFs. Attempts to call the function on bond slaves, VLAN PIFs, or Tunnel PIFs, fail with CANNOT_CHANGE_PIF_PROPERTIES.
    • Calls with invalid keys or values fail with INVALID_VALUE.
    • When called on a bond PIF, the key in the properties of the associated bond slaves will also be set to same value.
    • The function automatically causes the settings to be applied to the network devices (no additional plug is needed). This includes any VLANs that are on top of the PIF to-be-changed, as well as any bond slaves.
    Defaults, Installation and Upgrade
    • Any newly introduced PIF will have its properties field set to "gro" -> "on". This includes PIFs obtained after a fresh installation of XenServer, as well as PIFs created using PIF.introduce or PIF.scan. In other words, GRO will be “on” by default.
    • An upgrade from a version of XenServer that does not have the PIF.properties field, will give every physical and bond PIF a properties field set to "gro" -> "on". In other words, GRO will be “on” by default after an upgrade.
    Bonding
    • When creating a bond, the bond-slaves-to-be must all have equal PIF.properties. If not, the bond.create call will fail with INCOMPATIBLE_BOND_PROPERTIES.
    • When a bond is created successfully, the properties of the bond PIF will be equal to the properties of the bond slaves.
    Command Line Interface
    • The PIF.properties field is exposed through xe pif-list and xe pif-param-list as usual.
    • The PIF.set_property call is exposed through xe pif-param-set. For example: xe pif-param-set uuid=<pif-uuid> properties:gro=off.
    \ No newline at end of file diff --git a/new-docs/design/plugin-protocol-v2/index.html b/new-docs/design/plugin-protocol-v2/index.html index d3f23512d2..715d76aef9 100644 --- a/new-docs/design/plugin-protocol-v2/index.html +++ b/new-docs/design/plugin-protocol-v2/index.html @@ -11,7 +11,7 @@ DATASOURCES 000001e4 dba4bf7a84b6d11d565d19ef91f7906e { "timestamp": 1339685573.245, "data_sources": { "cpu-temp-cpu0": { "description": "Temperature of CPU 0", "type": "absolute", "units": "degC", "value": "64.33" "value_type": "float", }, "cpu-temp-cpu1": { "description": "Temperature of CPU 1", "type": "absolute", "units": "degC", "value": "62.14" "value_type": "float", } } } This format contains four main components: A constant header string DATASOURCES This should always be present.'>RRDD plugin protocol v2 :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv1
    Status
    Design document
    Revisionv1
    Statusreleased (7.0)
    Revision history
    v1Initial version

    RRDD plugin protocol v2

    Motivation

    rrdd plugins currently report datasources via a shared-memory file, using the following format:

    DATASOURCES
 000001e4
@@ -117,12 +117,12 @@
         update(datasource, value)

    This means that for a normal update, RRDD will only have to read the header plus the first (16 + 16 + 4 + 8 + 8*n) bytes of data, where n is the number of datasources exported by the plugin. If the metadata changes RRDD will have to -read all the data (and parse the metadata).

    \ No newline at end of file diff --git a/new-docs/design/plugin-protocol-v3/index.html b/new-docs/design/plugin-protocol-v3/index.html index ab0aed71b4..e0a710f3a4 100644 --- a/new-docs/design/plugin-protocol-v3/index.html +++ b/new-docs/design/plugin-protocol-v3/index.html @@ -3,7 +3,7 @@ metrics are unique by their names, thus it is not possible cannot have several metrics that shares a same name (e.g vCPU usage per vm) only number metrics are supported, for example we can’t expose string metrics (e.g CPU Model) Therefore, it implies various limitations on plugins and limits OpenMetrics support for the metrics daemon.">RRDD plugin protocol v3 :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv1
    Status
    Design document
    Revisionv1
    Statusproposed
    Revision history
    v1Initial version

    RRDD plugin protocol v3

    Motivation

    rrdd plugins protocol v2 report datasources via shared-memory file, however it has various limitations :

    • metrics are unique by their names, thus it is not possible cannot have several metrics that shares a same name (e.g vCPU usage per vm)
    • only number metrics are supported, for example we can’t expose string @@ -29,12 +29,12 @@ else create_family(family) -track_removed_families(metrics)
    \ No newline at end of file diff --git a/new-docs/design/pool-certificates/index.html b/new-docs/design/pool-certificates/index.html index 80f81c634d..3d2cf05869 100644 --- a/new-docs/design/pool-certificates/index.html +++ b/new-docs/design/pool-certificates/index.html @@ -3,7 +3,7 @@ In order to enable certificate verification, certificate exchange as well as proper set up to trust them must be provided by xapi. This is currently done by allowing users to generate, sign and install the certificates themselves; and then enable the Common Criteria mode.">TLS vertification for intra-pool communications :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv2
    Status
    Design document
    Revisionv2
    Statusreleased (22.6.0)

    TLS vertification for intra-pool communications

    Overview

    Xenserver has used TLS-encrypted communications between xapi daemons in a pool since its first release. However it does not use TLS certificates to authenticate the servers it connects to. This allows possible attackers opportunities to impersonate servers when the pools’ management network is compromised.

    In order to enable certificate verification, certificate exchange as well as proper set up to trust them must be provided by xapi. @@ -103,12 +103,12 @@ usr5origin55.4.3.21020200922T15:03:13Z usr4UA620200922T15:03:13Z UA4.3.2.1420200922T14:57:11Z -10

    \ No newline at end of file diff --git a/new-docs/design/pool-wide-ssh/index.html b/new-docs/design/pool-wide-ssh/index.html index e9a99b397d..2e164ae895 100644 --- a/new-docs/design/pool-wide-ssh/index.html +++ b/new-docs/design/pool-wide-ssh/index.html @@ -3,7 +3,7 @@ However, many of the currently existing SMAPIv1 backends require inter-host operations to be performed. This is achieved via the use of the Xen-API call ‘host.">Pool-wide SSH :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv1
    Status
    Design document
    Revisionv1
    Statusproposed

    Pool-wide SSH

    Background

    The SMAPIv3 plugin architecture requires that storage plugins are able to work in the absence of xapi. Amongst other benefits, this allows them to be tested in isolation, are able to be shared more widely than just within the XenServer @@ -35,12 +35,12 @@ ssh_ready value for the local host. When this is set to false, the host then adds the keys from xapi’s database to the appropriate places in the ssh configuration files and restarts sshd. Once this is done, the host sets the -ssh_ready field to ’true’

    Pool Eject

    On pool eject, the host’s ssh_public_host_key is lost, but the authetication key is added to a list of revoked keys on the pool object. This allows all other hosts to remove the key from the authorized_keys list when they next sync, which in the usual case is immediately the database is modified due to the event watch thread. If the host is offline though, the authorized_keys file will be updated the next time the host comes online.

    Questions

    • Do we want a new user? e.g. ‘xapi’ - how would we then use this user to execute privileged things? setuid binaries?
    • Is keeping the revoked_keys list useful? If we ‘control the world’ of the authorized_keys file, we could just remove anything that’s currently in there that xapi doesn’t know about
    \ No newline at end of file diff --git a/new-docs/design/schedule-snapshot/index.html b/new-docs/design/schedule-snapshot/index.html index ab06505338..5889cddfe5 100644 --- a/new-docs/design/schedule-snapshot/index.html +++ b/new-docs/design/schedule-snapshot/index.html @@ -3,13 +3,13 @@ Introduction Schedule snapshot will be a new object in xapi as VMSS. A pool can have multiple VMSS. Multiple VMs can be a part of VMSS but a VM cannot be a part of multiple VMSS. A VMSS takes VMs snapshot with type [snapshot, checkpoint, snapshot_with_quiesce]. VMSS takes snapshot of VMs on configured intervals: hourly -> On every day, Each hour, Mins [0;15;30;45] daily -> On every day, Hour [0 to 23], Mins [0;15;30;45] weekly -> Days [Monday,Tuesday,Wednesday,Thursday,Friday,Saturday,Sunday], Hour[0 to 23], Mins [0;15;30;45] VMSS will have a limit on retaining number of VM snapshots in range [1 to 10].">Schedule Snapshot Design :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv2
    Statusproposed
    Review#186
    Revision history
    v1Initial version
    v2Renaming VMSS fields and APIs. API message_create superseeds vmss_create_alerts.
    v3Remove VMSS alarm_config details and use existing pool wide alarm config
    v4Renaming field from retention-value to retained-snapshots and schedule-snapshot to scheduled-snapshot
    v5Add new API task_set_status

    Schedule Snapshot Design

    The scheduled snapshot feature will utilize the existing architecture of VMPR. In terms of functionality, scheduled snapshot is basically VMPR without its archiving capability.

    Introduction

    • Schedule snapshot will be a new object in xapi as VMSS.
    • A pool can have multiple VMSS.
    • Multiple VMs can be a part of VMSS but a VM cannot be a part of multiple VMSS.
    • A VMSS takes VMs snapshot with type [snapshot, checkpoint, snapshot_with_quiesce].
    • VMSS takes snapshot of VMs on configured intervals:
      • hourly -> On every day, Each hour, Mins [0;15;30;45]
      • daily -> On every day, Hour [0 to 23], Mins [0;15;30;45]
      • weekly -> Days [Monday,Tuesday,Wednesday,Thursday,Friday,Saturday,Sunday], Hour[0 to 23], Mins [0;15;30;45]
    • VMSS will have a limit on retaining number of VM snapshots in range [1 to 10].

    Datapath Design

    • There will be a cron job for VMSS.
    • VMSS plugin will go through all the scheduled snapshot policies in the pool and check if any of them are due.
    • If a snapshot is due then : Go through all the VM objects in XAPI associated with this scheduled snapshot policy and create a new snapshot.
    • If the snapshot operation fails, create a notification alert for the event and move to the next VM.
    • Check if an older snapshot now needs to be deleted to comply with the retained snapshots defined in the scheduled policy.
    • If we need to delete any existing snapshots, delete the oldest snapshot created via scheduled policy.
    • Set the last-run timestamp in the scheduled policy.

    Xapi Changes

    There is a new record for VM Scheduled Snapshot with new fields.

    New fields:

    • name-label type String : Name label for VMSS.
    • name-description type String : Name description for VMSS.
    • enabled type Bool : Enable/Disable VMSS to take snapshot.
    • type type Enum [snapshot; checkpoint; snapshot_with_quiesce] : Type of snapshot VMSS takes.
    • retained-snapshots type Int64 : Number of snapshots limit for a VM, max limit is 10 and default is 7.
    • frequency type Enum [hourly; daily; weekly] : Frequency of taking snapshot of VMs.
    • schedule type Map(String,String) with (key, value) pair:
      • hour : 0 to 23
      • min : [0;15;30;45]
      • days : [Monday,Tuesday,Wednesday,Thursday,Friday,Saturday,Sunday]
    • last-run-time type Date : DateTime of last execution of VMSS.
    • VMs type VM refs : List of VMs part of VMSS.

    New fields to VM record:

    • scheduled-snapshot type VMSS ref : VM part of VMSS.
    • is-vmss-snapshot type Bool : If snapshot created from VMSS.

    New APIs

    • vmss_snapshot_now (Ref vmss, Pool_Operater) -> String : This call executes the scheduled snapshot immediately.
    • vmss_set_retained_snapshots (Ref vmss, Int value, Pool_Operater) -> unit : Set the value of vmss retained snapshots, max is 10.
    • vmss_set_frequency (Ref vmss, String “value”, Pool_Operater) -> unit : Set the value of the vmss frequency field.
    • vmss_set_type (Ref vmss, String “value”, Pool_Operater) -> unit : Set the snapshot type of the vmss type field.
    • vmss_set_scheduled (Ref vmss, Map(String,String) “value”, Pool_Operater) -> unit : Set the vmss scheduled to take snapshot.
    • vmss_add_to_schedule (Ref vmss, String “key”, String “value”, Pool_Operater) -> unit : Add key value pair to VMSS schedule.
    • vmss_remove_from_schedule (Ref vmss, String “key”, Pool_Operater) -> unit : Remove key from VMSS schedule.
    • vmss_set_last_run_time (Ref vmss, DateTime “value”, Local_Root) -> unit : Set the last run time for VMSS.
    • task_set_status (Ref task, status_type “value”, READ_ONLY) -> unit : Set the status of task owned by same user, Pool_Operator can set status for any tasks.

    New CLIs

    • vmss-create (required : “name-label”;“type”;“frequency”, optional : “name-description”;“enabled”;“schedule:”;“retained-snapshots”) -> unit : Creates VM scheduled snapshot.
    • vmss-destroy (required : uuid) -> unit : Destroys a VM scheduled snapshot.
    \ No newline at end of file diff --git a/new-docs/design/sm-supported-image-formats/index.html b/new-docs/design/sm-supported-image-formats/index.html index 6db7d2141b..fb49a5e373 100644 --- a/new-docs/design/sm-supported-image-formats/index.html +++ b/new-docs/design/sm-supported-image-formats/index.html @@ -3,7 +3,7 @@ We need a mechanism to expose to end users which image formats are supported by a given SR.">Add supported image formats in sm-list :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv3
    Status
    Design document
    Revisionv3
    Statusproposed

    Add supported image formats in sm-list

    Introduction

    At XCP-ng, we are enhancing support for QCOW2 images in SMAPI. The primary motivation for this change is to overcome the 2TB size limitation imposed by the VHD format. By adding support for QCOW2, a Storage Repository (SR) will @@ -83,12 +83,12 @@ whether all VDIs can be migrated.

    To be able to check the format, we will need to modify VM.assert_can_migrate and add the mapping from VDI references to their image formats, as is done in VM.migrate_send.

    Impact

    It should have no impact on existing storage repositories that do not provide any information about the supported image format.

    This change impacts the SM data model, and as such, the XAPI database version will -be incremented. It also impacts the API.

    • Data Model:
      • A new field (supported_image_formats) is added to the SM records.
      • A new parameter is added to VM.migrate_send: (VDI ref -> string) map
      • A new parameter is added to VM.assert_can_migrate: (VDI ref -> string) map
      • A new parameter is added to VDI.pool_migrate: string
    • Client Awareness: Clients like the xe CLI will now be able to query and display the supported image formats for a given SR.
    • Database Versioning: The XAPI database version will be updated to reflect this change.
    \ No newline at end of file diff --git a/new-docs/design/smapiv3/index.html b/new-docs/design/smapiv3/index.html index f9c960f7c9..86b110b497 100644 --- a/new-docs/design/smapiv3/index.html +++ b/new-docs/design/smapiv3/index.html @@ -7,7 +7,7 @@ the protocol expects plugin authors to have a deep knowledge of the Xen storage datapath (tapdisk, blkback etc) and the storage.">SMAPIv3 :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv1
    Status
    Design document
    Revisionv1
    Statusreleased (7.6)

    SMAPIv3

    Xapi accesses storage through “plugins” which currently use a protocol called “SMAPIv1”. This protocol has a number of problems:

    1. the protocol has many missing features, and this leads to people using the XenAPI from within a plugin, which is racy, difficult to @@ -25,12 +25,12 @@ plugin, and see whether

      • the queue is being served or not (perhaps the xapi-storage-script has crashed)
      • there are unanswered messages (perhaps one of the messages has caused a deadlock in the implementation?)

      It will be possible to

      • delete/clear queues/messages
      • download a message-sequence chart of the last N messages for inclusion in -bugtools.

      Anatomy of a plugin

      The following diagram shows what a plugin would look like:

      Anatomy of a plugin

      The SMAPIv3

      Please read the current SMAPIv3 documentation.

    \ No newline at end of file diff --git a/new-docs/design/snapshot-revert/index.html b/new-docs/design/snapshot-revert/index.html index 494295bae2..eaa3e4e018 100644 --- a/new-docs/design/snapshot-revert/index.html +++ b/new-docs/design/snapshot-revert/index.html @@ -7,7 +7,7 @@ Clients such as Apache CloudStack need to include complex logic to keep track of the disks they are actively managing Because the snapshot is cloned and the original vdi is deleted, VDI references to the VDI become invalid, like VDI.">Better VM revert :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv2
    Status
    Design document
    Revisionv2
    Statusconfirmed

    Better VM revert

    Overview

    XenAPI allows users to rollback the state of a VM to a previous state, which is stored in a snapshot, using the call VM.revert. Because there is no VDI.revert call, VM.revert uses VDI.clone on the snapshot to duplicate @@ -50,12 +50,12 @@ and recovered, using the existing method

  • expose a new xe vdi-revert CLI command

    • SM changes

    We will modify

    • SRCommand.py and VDI.py to add a new vdi_revert function which throws a ’not implemented’ exception
    • FileSR.py to implement VDI.revert using a variant of the existing snapshot/clone machinery
    • EXTSR.py and NFSSR.py to advertise the VDI_REVERT capability
    • LVHDSR.py to implement VDI.revert using a variant of the existing -snapshot/clone machinery
    • LVHDoISCSISR.py and LVHDoHBASR.py to advertise the VDI_REVERT capability

    Prototype code from the previous proposal

    Prototype code exists here:

    \ No newline at end of file diff --git a/new-docs/design/sr-level-rrds/index.html b/new-docs/design/sr-level-rrds/index.html index 09ff5ac6a2..70c817efc7 100644 --- a/new-docs/design/sr-level-rrds/index.html +++ b/new-docs/design/sr-level-rrds/index.html @@ -1,5 +1,5 @@ SR-Level RRDs :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv11
    Status
    Design document
    Revisionv11
    Statusconfirmed
    Review#139
    Revision history
    v1Initial version
    v2Added details about the VDI's binary format and size, and the SR capability name.
    v3Tar was not needed after all!
    v4Add details about discovering the VDI using a new vdi_type.
    v5Add details about the http handlers and interaction with xapi's database
    v6Add details about the framing of the data within the VDI
    v7Redesign semantics of the rrd_updates handler
    v8Redesign semantics of the rrd_updates handler (again)
    v9Magic number change in framing format of vdi
    v10Add details of new APIs added to xapi and xcp-rrdd
    v11Remove unneeded API calls

    SR-Level RRDs

    Introduction

    Xapi has RRDs to track VM- and host-level metrics. There is a desire to have SR-level RRDs as a new category, because SR stats are not specific to a certain VM or host. Examples are size and free space on the SR. While recording SR metrics is relatively straightforward within the current RRD system, the main question is where to archive them, which is what this design aims to address.

    Stats Collection

    All SR types, including the existing ones, should be able to have RRDs defined for them. Some RRDs, such as a “free space” one, may make sense for multiple (if not all) SR types. However, the way to measure something like free space will be SR specific. Furthermore, it should be possible for each type of SR to have its own specialised RRDs.

    It follows that each SR will need its own xcp-rrdd plugin, which runs on the SR master and defines and collects the stats. For the new thin-lvhd SR this could be xenvmd itself. The plugin registers itself with xcp-rrdd, so that the latter records the live stats from the plugin into RRDs.

    Archiving

    SR-level RRDs will be archived in the SR itself, in a VDI, rather than in the local filesystem of the SR master. This way, we don’t need to worry about master failover.

    The VDI will be 4MB in size. This is a little more space than we would need for the RRDs we have in mind at the moment, but will give us enough headroom for the foreseeable future. It will not have a filesystem on it for simplicity and performance. There will only be one RRD archive file for each SR (possibly containing data for multiple metrics), which is gzipped by xcp-rrdd, and can be copied onto the VDI.

    There will be a simple framing format for the data on the VDI. This will be as follows:

    OffsetTypeNameComment
    032 bit network-order intmagicMagic number = 0x7ada7ada
    432 bit network-order intversion1
    832 bit network-order intlengthlength of payload
    12gzipped datadata

    Xapi will be in charge of the lifecycle of this VDI, not the plugin or xcp-rrdd, which will make it a little easier to manage them. Only xapi will attach/detach and read from/write to this VDI. We will keep xcp-rrdd as simple as possible, and have it archive to its standard path in the local file system. Xapi will then copy the RRDs in and out of the VDI.

    A new value "rrd" in the vdi_type enum of the datamodel will be defined, and the VDI.type of the VDI will be set to that value. The storage backend will write the VDI type to the LVM metadata of the VDI, so that xapi can discover the VDI containing the SR-level RRDs when attaching an SR to a new pool. This means that SR-level RRDs are currently restricted to LVM SRs.

    Because we will not write plugins for all SRs at once, and therefore do not need xapi to set up the VDI for all SRs, we will add an SR “capability” for the backends to be able to tell xapi whether it has the ability to record stats and will need storage for them. The capability name will be: SR_STATS.

    Management of the SR-stats VDI

    The SR-stats VDI will be attached/detached on PBD.plug/unplug on the SR master.

    • On PBD.plug on the SR master, if the SR has the stats capability, xapi:

      • Creates a stats VDI if not already there (search for an existing one based on the VDI type).
      • Attaches the stats VDI if it did already exist, and copies the RRDs to the local file system (standard location in the filesystem; asks xcp-rrdd where to put them).
      • Informs xcp-rrdd about the RRDs so that it will load the RRDs and add newly recorded data to them (needs a function like push_rrd_local for VM-level RRDs).
      • Detaches stats VDI.

    • On PBD.unplug on the SR master, if the SR has the stats capability xapi:

      • Tells xcp-rrdd to archive the RRDs for the SR, which it will do to the local filesystem.
      • Attaches the stats VDI, copies the RRDs into it, detaches VDI.

    Periodic Archiving

    Xapi’s periodic scheduler regularly triggers xcp-rrdd to archive the host and VM RRDs. It will need to do this for the SR ones as well. Furthermore, xapi will need to attach the stats VDI and copy the RRD archives into it (as on PBD.unplug).

    Exporting

    There will be a new handler for downloading an SR RRD:

    http://<server>/sr_rrd?session_id=<SESSION HANDLE>&uuid=<SR UUID>

    RRD updates are handled via a single handler for the host, VM and SR UUIDs RRD updates for the host, VMs and SRs are handled by a a single handler at @@ -27,12 +27,12 @@ open to suggestions here), will query the attached SRs, then query RRDD for the latest data source for these, and update the database.

    The utilisation of VDIs will not be updated in this way until scalability worries for RRDs are addressed.

    Xapi will cache whether it is SR master for every attached SR and only -attempt to update if it is the SR master.

    New APIs.

    xcp-rrdd:

    • Get the filesystem location where sr rrds are archived: val sr_rrds_path : uid:string -> string

    • Archive the sr rrds to the filesystem: val archive_sr_rrd : sr_uuid:string -> unit

    • Load the sr rrds from the filesystem: val push_sr_rrd : sr_uuid:string -> unit

    \ No newline at end of file diff --git a/new-docs/design/thin-lvhd/index.html b/new-docs/design/thin-lvhd/index.html index 114977bdb6..752f6d8fe8 100644 --- a/new-docs/design/thin-lvhd/index.html +++ b/new-docs/design/thin-lvhd/index.html @@ -1,5 +1,5 @@ thin LVHD storage :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv3
    Status
    Design document
    Revisionv3
    Statusproposed

    thin LVHD storage

    LVHD is a block-based storage system built on top of Xapi and LVM. LVHD disks are represented as LVM LVs with vhd-format data inside. When a disk is snapshotted, the LVM LV is “deflated” to the minimum-possible @@ -296,12 +296,12 @@ from its free pool into the fake LV, which xenvmd will then delete as soon as it gets the update.

    Xenvmd keeps track of the last update it has sent to the local allocator, and will not resend the same request twice, unless it -is restarted.

    \ No newline at end of file diff --git a/new-docs/design/tunnelling/index.html b/new-docs/design/tunnelling/index.html index 819ec9258b..eaccbea38b 100644 --- a/new-docs/design/tunnelling/index.html +++ b/new-docs/design/tunnelling/index.html @@ -1,5 +1,5 @@ Tunnelling API design :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv1
    Status
    Design document
    Revisionv1
    Statusreleased (5.6 fp1)

    Tunnelling API design

    To isolate network traffic between VMs (e.g. for security reasons) one can use VLANs. The number of possible VLANs on a network, however, is limited, and setting up a VLAN requires configuring the physical switches in the network. @@ -72,12 +72,12 @@ (at the same time tunnel.status:active will be set to "true").

  • When the tunnel infrastructure is not up and running, the controller may remove the key tunnel.status:key (optional; the key should anyway be disregarded if -tunnel.status:active is "false").

    • CLI

    New xe commands (analogous to xe vlan-):

    • tunnel-create
    • tunnel-destroy
    • tunnel-list
    • tunnel-param-get
    • tunnel-param-list
    \ No newline at end of file diff --git a/new-docs/design/user-certificates/index.html b/new-docs/design/user-certificates/index.html index d03d00bb95..5bf606268e 100644 --- a/new-docs/design/user-certificates/index.html +++ b/new-docs/design/user-certificates/index.html @@ -3,7 +3,7 @@ Historically there has not been a supported mechanism to do this, and as a result users have had to rely on guides written by third parties that show how to manually replace the xapi-ssl.">User-installable host certificates :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv2
    Status
    Design document
    Revisionv2
    Statusreleased (8.2)

    User-installable host certificates

    Introduction

    It is often necessary to replace the TLS certificate used to secure communications to Xenservers hosts, for example to allow a XenAPI user such as Citrix Virtual Apps and Desktops (CVAD) to validate that the host is genuine @@ -125,12 +125,12 @@ Parameters, specifically 512 bits worth. We no longer support any ciphers which require DH parameters, so these are no longer needed, and it is acceptable for them to be lost as part of installing a new certificate/key pair.

    The generation should also be modified to avoid creating these for new -installations.

    \ No newline at end of file diff --git a/new-docs/design/vgpu-type-identifiers/index.html b/new-docs/design/vgpu-type-identifiers/index.html index 8c9ff0974b..845747e053 100644 --- a/new-docs/design/vgpu-type-identifiers/index.html +++ b/new-docs/design/vgpu-type-identifiers/index.html @@ -3,7 +3,7 @@ When deciding which VGPU_type objects need to be created, xapi needs to determine whether a suitable VGPU_type object already exists, as there should never be duplicates. At the moment the combination of vendor name and model name is used as a primary key, but this is not ideal as these values are subject to change.">VGPU type identifiers :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv1
    Status
    Design document
    Revisionv1
    Statusreleased (7.0)
    Review#156
    Revision history
    v1Initial version

    VGPU type identifiers

    Introduction

    When xapi starts, it may create a number of VGPU_type objects. These act as VGPU presets, and exactly which VGPU_type objects are created depends on the installed hardware and in certain cases the presence of certain files in dom0.

    When deciding which VGPU_type objects need to be created, xapi needs to @@ -55,12 +55,12 @@ identifier string and use it to look for existing VGPU_type objects in the database. If none are found, xapi will look for existing VGPU_type objects with the tuple of model name and vendor name. If still none are found, xapi will -create a new VGPU_type object.

    \ No newline at end of file diff --git a/new-docs/design/virt-hw-platform-vn/index.html b/new-docs/design/virt-hw-platform-vn/index.html index b5020341b5..0c9abe4cf3 100644 --- a/new-docs/design/virt-hw-platform-vn/index.html +++ b/new-docs/design/virt-hw-platform-vn/index.html @@ -7,13 +7,13 @@ Suggested design In the datamodel, VM has a new integer field “hardware_platform_version” which defaults to zero. In the datamodel, Host has a corresponding new integer-list field “virtual_hardware_platform_versions” which defaults to list containing a single zero element (i.">Virtual Hardware Platform Version :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv1
    Statusreleased (7.0)

    Virtual Hardware Platform Version

    Background and goal

    Some VMs can only be run on hosts of sufficiently recent versions.

    We want a clean way to ensure that xapi only tries to run a guest VM on a host that supports the “virtual hardware platform” required by the VM.

    Suggested design

    • In the datamodel, VM has a new integer field “hardware_platform_version” which defaults to zero.
    • In the datamodel, Host has a corresponding new integer-list field “virtual_hardware_platform_versions” which defaults to list containing a single zero element (i.e. [0] or [0L] in OCaml notation). The zero represents the implicit version supported by older hosts that lack the code to handle the Virtual Hardware Platform Version concept.
    • When a host boots it populates its own entry from a hardcoded value, currently [0; 1] i.e. a list containing the two integer elements 0 and 1. (Alternatively this could come from a config file.)
      • If this new version-handling functionality is introduced in a hotfix, at some point the pool master will have the new functionality while at least one slave does not. An old slave-host that does not yet have software to handle this feature will not set its DB entry, which will therefore remain as [0] (maintained in the DB by the master).
    • The existing test for whether a VM can run on (or migrate to) a host must include a check that the VM’s virtual hardware platform version is in the host’s list of supported versions.
    • When a VM is made to start using a feature that is available only in a certain virtual hardware platform version, xapi must set the VM’s hardware_platform_version to the maximum of that version-number and its current value (i.e. raise if needed).

    For the version we could consider some type other than integer, but a strict ordering is needed.

    First use-case

    Version 1 denotes support for a certain feature:

    When a VM starts, if a certain flag is set in VM.platform then XenServer will provide an emulated PCI device which will trigger the guest Windows OS to seek drivers for the device, or updates for those drivers. Thus updated drivers can be obtained through the standard Windows Update mechanism.

    If the PCI device is removed, the guest OS will fail to boot. A VM using this feature must not be migrated to or started on a XenServer that lacks support for the feature.

    Therefore at VM start, we can look at whether this feature is being used; if it is, then if the VM’s Virtual Hardware Platform Version is less than 1 we should raise it to 1.

    Limitation

    Consider a VM that requires version 1 or higher. Suppose it is exported, then imported into an old host that does not support this feature. Then the host will not check the versions but will attempt to run the VM, which will then have difficulties.

    The only way to prevent this would be to make a backwards-incompatible change to the VM metadata (e.g. a new item in an enum) so that the old hosts cannot read it, but that seems like a bad idea.

    \ No newline at end of file diff --git a/new-docs/design/xenopsd_events/index.html b/new-docs/design/xenopsd_events/index.html index 28383fe368..f4f71e0fa5 100644 --- a/new-docs/design/xenopsd_events/index.html +++ b/new-docs/design/xenopsd_events/index.html @@ -3,7 +3,7 @@ XAPI currently processes all events from xenopsd in a single thread, the unpause events get queued up behind a lot of other events generated by the already running VMs.">Process events from xenopsd in a timely manner :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv1
    Status
    Design document
    Revisionv1
    Statusproposed

    Process events from xenopsd in a timely manner

    Background

    There is a significant delay between the VM being unpaused and XAPI reporting it as started during a bootstorm. It can happen that the VM is able to send UDP packets already, but XAPI still reports it as not started for minutes.

    XAPI currently processes all events from xenopsd in a single thread, the unpause @@ -16,12 +16,12 @@ While the item is executed the tag temporarily doesn’t participate in round-robin scheduling. If during execution more items get queued with the same tag they get redirected to a private queue. Once the item finishes execution the tag will participate in RR scheduling again.

    This ensures that items with the same tag do not get executed in parallel, -and that a tag with a lot of items does not starve the execution of other tags.

    The XAPI side of the changes will look like this

    Known limitations: The active per-VM events should be a small number, this is already ensured in the push_with_coalesce / should_keep code on the xenopsd side. Events to XAPI from xenopsd should already arrive coalesced.

    \ No newline at end of file diff --git a/new-docs/design/xenprep/index.html b/new-docs/design/xenprep/index.html index e333df4a74..2c7da87111 100644 --- a/new-docs/design/xenprep/index.html +++ b/new-docs/design/xenprep/index.html @@ -3,14 +3,14 @@ There are many existing Windows guests that have drivers installed already.">XenPrep :: XAPI Toolstack Developer Documentation -
    Design document
    Revisionv2
    Status
    Design document
    Revisionv2
    Statusproposed

    XenPrep

    Background

    Windows guests should have XenServer-specific drivers installed. As of mid-2015 these have been always been installed and upgraded by an essentially manual process involving an ISO carrying the drivers. We have a plan to enable automation through the standard Windows Update mechanism. This will involve a new additional virtual PCI device being provided to the VM, to trigger Windows Update to fetch drivers for the device.

    There are many existing Windows guests that have drivers installed already. These drivers must be uninstalled before the new drivers are installed (and ideally before the new PCI device is added). To make this easier, we are planning a XenAPI call that will cause the removal of the old drivers and the addition of the new PCI device.

    Since this is only to help with updating old guests, the call may well be removed at some point in the future.

    Brief high-level design

    The XenAPI call will be called VM.xenprep_start. It will update the VM record to note that the process has started, and will insert a special ISO into the VM’s virtual CD drive.

    That ISO will contain a tool which will be set up to auto-run (if auto-run is enabled in the guest). The tool will:

    1. Lock the CD drive so other Windows programs cannot eject the disc.
    2. Uninstall the old drivers.
    3. Eject the CD to signal success.
    4. Shut down the VM.

    XenServer will interpret the ejection of the CD as a success signal, and when the VM shuts down without the special ISO in the drive, XenServer will:

    1. Update the VM record:
    • Remove the mark that shows that the xenprep process is in progress
    • Give it the new PCI device: set VM.auto_update_drivers to true.
    • If VM.virtual_hardware_platform_version is less than 2, then set it to 2.
    1. Start the VM.

    More details of the xapi-project parts

    (The tool that runs in the guest is out of scope for this document.)

    Start

    The XenAPI call VM.xenprep_start will throw a power-state error if the VM is not running. -For RBAC roles, it will be available to “VM Operator” and above.

    It will:

    1. Insert the xenprep ISO into the VM’s virtual CD drive.
    2. Write VM.other_config key xenprep_progress=ISO_inserted to record the fact that the xenprep process has been initiated.

    If xenprep_start is called on a VM already undergoing xenprep, the call will return successfully but will not do anything.

    If the VM does not have an empty virtual CD drive, the call will fail with a suitable error.

    Cancellation

    While xenprep is in progress, any request to eject the xenprep ISO (except from inside the guest) will be rejected with a new error “VBD_XENPREP_CD_IN_USE”.

    There will be a new XenAPI call VM.xenprep_abort which will:

    1. Remove the xenprep_progress entry from VM.other_config.
    2. Make a best-effort attempt to eject the CD. (The guest might prevent ejection.)

    This is not intended for cancellation while the xenprep tool is running, but rather for use before it starts, for example if auto-run is disabled or if the VM has a non-Windows OS.

    Completion

    Aim: when the guest shuts down after ejecting the CD, XenServer will start the guest again with the new PCI device.

    Xapi works through the queue of events it receives from xenopsd. It is possible that by the time xapi processes the cd-eject event, the guest might have shut down already.

    When the shutdown (not reboot) event is handled, we shall check whether we need to do anything xenprep-related. If

    • The VM other_config map has xenprep_progress as either of ISO_inserted or shutdown, and
    • The xenprep ISO is no longer in the drive

    then we must (in the specified order)

    1. Update the VM record:
    2. In VM.other_config set xenprep_progress=shutdown
    3. If VM.virtual_hardware_platform_version is less than 2, then set it to 2.
    4. Give it the new PCI device: set VM.auto_update_drivers to true.
    5. Initiate VM start.
    6. Remove xenprep_progress from VM.other_config

    The most relevant code is probably the update_vm function in ocaml/xapi/xapi_xenops.ml in the xen-api repo (or in some function called from there).

    \ No newline at end of file diff --git a/new-docs/index.html b/new-docs/index.html index 575df0c6eb..a228580f23 100644 --- a/new-docs/index.html +++ b/new-docs/index.html @@ -7,16 +7,16 @@ The source code is available on Github under the xapi-project. the main repository is xen-api.">XAPI Toolstack Developer Guide :: XAPI Toolstack Developer Documentation -

    XAPI Toolstack Developer Guide

    The XAPI Toolstack:

    • Forms the control plane of both XenServer as well as +

      XAPI Toolstack Developer Guide

      The XAPI Toolstack:

      • Forms the control plane of both XenServer as well as xcp-ng,
      • manages clusters of Xen hosts with shared storage and networking,
      • has a full-featured API, used by clients such as XenCenter and Xen Orchestra.

      The XAPI Toolstack is an open-source project developed by the xapi project, a sub-project of the Linux -Foundation Xen Project.

      The source code is available on Github under the xapi-project. the main repository is xen-api.

      This developer guide documents the internals of the Toolstack to help developers understand the code, fix bugs and add new features. It is a work-in-progress, with new documents added when ready and updated whenever needed.

    \ No newline at end of file + 
    \ No newline at end of file diff --git a/new-docs/index.print.html b/new-docs/index.print.html index 5e4274de51..536123a661 100644 --- a/new-docs/index.print.html +++ b/new-docs/index.print.html @@ -7,7 +7,7 @@ The source code is available on Github under the xapi-project. the main repository is xen-api.">XAPI Toolstack Developer Guide :: XAPI Toolstack Developer Documentation -

    XAPI Toolstack Developer Guide

    The XAPI Toolstack: