Xen

after you have written a config file /etc/xen/abcd.cfg you have to create the instance by

xm create abcd.cfg

(I didn't expect the suffix .cfg to be neccessary)

xm console <Domain>

to disconnect type ctrl-] (which means to press Str-AltGr-9 on a german keyboard)

xm destroy <Domain>

this will not(!) destroy your data or config, it will just “poweroff” the running virtual machine without a clean shutdown. (don't know what happens to snapshots)

Assuming, you have a bridge xenbr0 up and running on your Dom0 with 10.1.0.1 beeing the outbound gateway and a disk partitioning scheme defined in /etc/xen-tools/partitions.d/my-disk-set

• create /etc/xen-tools/skel/root/.ssh/authorized_keys
• run create_guest.sh <name>

create_guest.sh

#!/bin/bash
 
set -o errexit
 
name="${1:-xenguest}"
 
xen-create-image --hostname="${name}" \
 --randommac \
 --ip=10.1.0.2 \
 --gateway=10.1.0.1 \
 --netmask=255.255.255.0 \
 --mirror=http://http.debian.net/debian/ \
 --dist=bullseye \
 --lvm=vg_1 \
 --bridge=xenbr0 \
 --vifname=vif."${name}".0 \
 --partitions=my-disk-set \
 --vcpus=1 \
 --memory=512Mb \
 --arch=amd64 \
 --nokeep \
 --nohosts \
 --boot \
 --password "" \
 --pygrub
 
xl console "${name}"

/etc/xen-tools/skel/root/.ssh/authorized_keys

# replace 202204260000 by some day in the near future and add your own sshkey
expiry-time="202204260000" ssh-rsa AAAAB3N...

xenstore-ls -f -s /local/domain/0/backend/vbd | egrep '(domain|frontend|dev|params) = ' 

References:

• ukautz' answer to question "Xen find VBD id for physical disks" on Server Fault.
• https://xenbits.xen.org/docs/4.9-testing/man/xenstore-ls.1.html

This is for domU booted by pygrub

• on domU as guest of dom0
  • give all filesystem a label
  • replace device by label in /etc/fstab on domU
  • replace dev by label for root fs in /boot/grub/menu.lst of domU
  • install lvm2 and mdadm
    • run update-initramfs if not done automatically
• on dom0 run update-grub2
• boot into grub menu
  • edit config for booting into the domU system on bare metal
    • add console=tty0 console=ttyS0,57600 to linux kernel line
  • continue booting
• in domU system on bare metal, install grub-pc and
  • edit /etc/default/grub
    • GRUB_CMDLINE_LINUX=“console=tty0 console=ttyS0,57600”
    • GRUB_TERMINAL=serial
  • run grub-install –no-bootsector /dev/mapper/<lvm device>¹⁾
  • run update-grub2 ²⁾
  • label filesystems if not already done (see above)
    • don't forget to relabel swap space
    • edit /etc/fstab if not already done (see above)
  • update /boot/grub/custom.cfg on (previous) dom0, so it boots domU system on bare metal via config …

• • install xen packages (xen-system, xen-tools, grub-xen-host, …)
  • edit /etc/network/interfaces to setup ip etc. correctly
  • allow root login by ssh (if you want it)
  • run mdadm –detail –brief –scan » /etc/mdadm/mdadm.conf on domU system on bare metal
  • run update-initrams -k all -c on domU system on bare metal
• reboot and …

grub-install.sh

lvcreate -L 1G -n lv_GRUB0 /dev/vg_1
mke2fs -L "$HOSTNAME:GRUB0" -t ext4 /dev/vg_1/lv_GRUB0 
mkdir /GRUB0
mount LABEL="$HOSTNAME:GRUB0" /GRUB0/
grub-install --boot-directory /GRUB0 --recheck /dev/sdb
grub-install --boot-directory /GRUB0 --recheck /dev/sda

/GRUB0/grub/grub.cfg

serial --speed=57600 --unit=0 --word=8 --parity=no --stop=1
terminal_input serial
terminal_output serial
set timeout=600
set default="dom0: LVM vg_1/dom0-root boot/grub/grub.cfg"
 
menuentry 'Old dom0: /dev/md/0(/dev/sdb1,/dev/sda1) grub/grub.cfg' {
        insmod gzio
        insmod part_msdos
        insmod diskfilter
        insmod mdraid1x
        insmod lvm
        insmod ext2
        set root='mduuid/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
        echo 'Loading (/dev/md/0)/grub/grub.cfg'
        configfile /grub/grub.cfg
}
menuentry 'dom0: LVM vg_1/dom0-root boot/grub/grub.cfg' {
        insmod gzio
        insmod part_msdos
        insmod diskfilter
        insmod mdraid1x
        insmod lvm
        insmod ext2
        echo "Setting root='lvm/vg_1-dom0--root'"
        set root='lvm/vg_1-dom0--root'
        echo 'Loading (/dev/vg_1/dom0-root)/boot/grub/grub.cfg'
        configfile /boot/grub/grub.cfg
}
menuentry "hd0 (MBR)" {
        insmod chain
        set root=(hd0)
        chainloader +1
}
menuentry "hd1 (MBR)" {
        insmod chain
        set root=(hd1)
        chainloader +1
}
menuentry "Reboot" {
        reboot
}
menuentry "Halt" {
        halt
}

Assuming grub-xen-host is installed and /dev/md/0 is /boot of old dom0

/etc/xen/olddom0.cfg

kernel = '/usr/lib/grub-xen/grub-x86_64-xen.bin'
 
vcpus       = '1'
memory      = '2048'
 
root        = '/dev/xvda1 ro'
 
disk        = [
                  'phy:/dev/md/0,xvda1,w',
                  'phy:/dev/md/1,xvda2,w',
                  'phy:/dev/md/3,xvda3,w',
                  'phy:/dev/vg_1/lv_home,xvda4,w',
                  'phy:/dev/sda2,xvda5,w',
                  'phy:/dev/sdb2,xvda6,w',
              ]
 
name        = 'olddom0'
 
dhcp        = 'dhcp'
vif         = [ 'bridge=xenbr0,vifname=vif.oddom0.0,mac=00:16:3e:XX:XX:XX' ]

assuming lspci shows 0000:02:00.0 as your network adapter

Add

xen-pciback hide=(0000:02:00.0)

to /etc/initramfs-tools/modules, run update-initramfs -u Add

pci = [ "0000:02:00.0" ]

to /etc/xen/guest.cfg

Add (assuming your network driver is e1000e)

GRUB_CMDLINE_LINUX_XEN_REPLACE="$GRUB_CMDLINE_LINUX blacklist=e1000e"

to /etc/default/grub

Manual steps:

rmmod e1000e
rmmod xen-pciback
modprobe xen-pciback "hide=(0000:02:00.0)"
modprobe e1000e
# check result
ip link
xl pci-assignable-list

On dom0 (new one if you came from above):

xl network-attach //olddom0//
brctl addbr xenbr0
ifconfig xenbr 10.1.0.2 netmask 255.255.255.0
route add default gw 10.1.0.2

On domU (with internet access - former dom0 if you came from above)

assuming eth0 is the outer (physical) network interface and the xen virtual interface showed up as eth1

ifconfig eth0 10.1.0.1 netmask 255.255.255.0
INET_IP="`ifconfig eth0 | sed -n -e 's/^[[:space:]]*inet \([.0-9]\+\) .*$/\1/gp'`"
iptables -t nat -A POSTROUTING -o eth0 -s 10.1.0.0/24  ! -d  10.1.0.0/24 -j SNAT --to "$INET_IP"
sysctl -w net.ipv4.conf.all.forwarding=1

In domU connected to external interface

• Give parameeter net.ifnames=0 to the kernel at boot time
• Use mac/xx.xx.xx.xx.xx.xx.xx=eth0 in /etc/network/interfaces

/etc/network/interfaces.d/ifrename

rename mac/00:16:3e:XX:XX:XX=vif0
# rename mac/aa:bb:cc:??:??:??=eth0 # use this if you expect the network card to be replaced by another one of same type
rename mac/aa:bb:cc:dd:ee:ff=eth0

/etc/systemd/network/10-persistent-eth0.link

# https://wiki.debian.org/NetworkInterfaceNames
[Match]
MACAddress=aa:bb:cc:dd:ee:ff
 
[Link]
Name=eth0

/etc/systemd/network/11-persistent-vif0.link

# https://wiki.debian.org/NetworkInterfaceNames
[Match]
MACAddress=00:16:3e:XX:XX:XX
 
[Link]
Name=vif0

/etc/network/interfaces.d/vif0

auto vif0
iface vif0 inet static
 address 10.1.0.1
 netmask 255.255.255.0

/etc/ssh/sshd_config

:
PermitRootLogin yes
:

• man xl.cfg
• https://wiki.xenproject.org/wiki/Xen_PCI_Passthrough#How_can_I_tell_if_I_have_IOMMU_.2F_VT-D_support.3F
• https://documentation.suse.com/sles/15-SP1/html/SLES-all/cha-xen-vhost.html
• https://xenbits.xen.org/docs/4.13-testing/misc/xen-command-line.html
• https://xenbits.xen.org/docs/4.6-testing/misc/xl-network-configuration.html
• https://xenproject.org/2015/01/07/using-grub-2-as-a-bootloader-for-xen-pv-guests/
• https://wiki.debian.org/Xen
• Bugs I encountered
  • [PATCH for 4.14] libxl: allow passthrough to PV guests regardless of whether IOMMU is enabled
    • Workaround: Fixed that by downgrading to Xen 4.11 from Debian Buster
  • xen-create-image --nodhcp option results in dhcp being configured #60
    • Workaround: Do not use –nodhcp, it is not needed if ip and netmask is given.
  • libxl: fix pci device re-assigning after domain reboot
    • Workaround: Don't reboot domU with PCI passthrough - shutdown and boot instead.