User Tools

Site Tools


docs:tips_n_tricks:xen.html

Xen

Start Instance

after you have written a config file /etc/xen/abcd.cfg you have to create the instance by

xm create abcd.cfg

(I didn't expect the suffix .cfg to be neccessary)

Get a console

xm console <Domain>

to disconnect type ctrl-] (which means to press Str-AltGr-9 on a german keyboard)

Kill a running instance

xm destroy <Domain>

this will not(!) destroy your data or config, it will just “poweroff” the running virtual machine without a clean shutdown. (don't know what happens to snapshots)

Create new DomU

Assuming, you have a bridge xenbr0 up and running on your Dom0 with 10.1.0.1 beeing the outbound gateway and a disk partitioning scheme defined in /etc/xen-tools/partitions.d/my-disk-set

  • create /etc/xen-tools/skel/root/.ssh/authorized_keys
  • run create_guest.sh <name>
create_guest.sh
#!/bin/bash
 
set -o errexit
 
name="${1:-xenguest}"
 
xen-create-image --hostname="${name}" \
 --randommac \
 --ip=10.1.0.2 \
 --gateway=10.1.0.1 \
 --netmask=255.255.255.0 \
 --mirror=http://http.debian.net/debian/ \
 --dist=bullseye \
 --lvm=vg_1 \
 --bridge=xenbr0 \
 --vifname=vif."${name}".0 \
 --partitions=my-disk-set \
 --vcpus=1 \
 --memory=512Mb \
 --arch=amd64 \
 --nokeep \
 --nohosts \
 --boot \
 --password "" \
 --pygrub
 
xl console "${name}"
/etc/xen-tools/skel/root/.ssh/authorized_keys
# replace 202204260000 by some day in the near future and add your own sshkey
expiry-time="202204260000" ssh-rsa AAAAB3N...

Turn a domU into a new dom0 running with serial console

This is for domU booted by pygrub

  • on domU as guest of dom0
    • give all filesystem a label
    • replace device by label in /etc/fstab on domU
    • replace dev by label for root fs in /boot/grub/menu.lst of domU
    • install lvm2 and mdadm
      • run update-initramfs if not done automatically
  • on dom0 run update-grub2
  • boot into grub menu
    • edit config for booting into the domU system on bare metal
      • add console=tty0 console=ttyS0,57600 to linux kernel line
    • continue booting
  • in domU system on bare metal, install grub-pc and
    • edit /etc/default/grub
      • GRUB_CMDLINE_LINUX=“console=tty0 console=ttyS0,57600”
      • GRUB_TERMINAL=serial
    • run grub-install –no-bootsector /dev/mapper/<lvm device>1)
    • run update-grub2 2)
    • label filesystems if not already done (see above)
      • don't forget to relabel swap space
      • edit /etc/fstab if not already done (see above)
    • update /boot/grub/custom.cfg on (previous) dom0, so it boots domU system on bare metal via config …
One important thing is to not load raid3) and lvm modules in the same insmod line in grub/custom.cfg!
    • install xen packages (xen-system, xen-tools, grub-xen-host, …)
    • edit /etc/network/interfaces to setup ip etc. correctly
    • allow root login by ssh (if you want it)
    • run mdadm –detail –brief –scan » /etc/mdadm/mdadm.conf on domU system on bare metal
    • run update-initrams -k all -c on domU system on bare metal
  • reboot and …

Create a generic grub installation

grub-install.sh
lvcreate -L 1G -n lv_GRUB0 /dev/vg_1
mke2fs -L "$HOSTNAME:GRUB0" -t ext4 /dev/vg_1/lv_GRUB0 
mkdir /GRUB0
mount LABEL="$HOSTNAME:GRUB0" /GRUB0/
grub-install --boot-directory /GRUB0 --recheck /dev/sdb
grub-install --boot-directory /GRUB0 --recheck /dev/sda
/GRUB0/grub/grub.cfg
serial --speed=57600 --unit=0 --word=8 --parity=no --stop=1
terminal_input serial
terminal_output serial
set timeout=600
set default="dom0: LVM vg_1/dom0-root boot/grub/grub.cfg"
 
menuentry 'Old dom0: /dev/md/0(/dev/sdb1,/dev/sda1) grub/grub.cfg' {
        insmod gzio
        insmod part_msdos
        insmod diskfilter
        insmod mdraid1x
        insmod lvm
        insmod ext2
        set root='mduuid/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
        echo 'Loading (/dev/md/0)/grub/grub.cfg'
        configfile /grub/grub.cfg
}
menuentry 'dom0: LVM vg_1/dom0-root boot/grub/grub.cfg' {
        insmod gzio
        insmod part_msdos
        insmod diskfilter
        insmod mdraid1x
        insmod lvm
        insmod ext2
        echo "Setting root='lvm/vg_1-dom0--root'"
        set root='lvm/vg_1-dom0--root'
        echo 'Loading (/dev/vg_1/dom0-root)/boot/grub/grub.cfg'
        configfile /boot/grub/grub.cfg
}
menuentry "hd0 (MBR)" {
        insmod chain
        set root=(hd0)
        chainloader +1
}
menuentry "hd1 (MBR)" {
        insmod chain
        set root=(hd1)
        chainloader +1
}
menuentry "Reboot" {
        reboot
}
menuentry "Halt" {
        halt
}

Use old dom0 as domU on new dom0

Assuming grub-xen-host is installed and /dev/md/0 is /boot of old dom0

/etc/xen/olddom0.cfg
kernel = '/usr/lib/grub-xen/grub-x86_64-xen.bin'
 
vcpus       = '1'
memory      = '2048'
 
root        = '/dev/xvda1 ro'
 
disk        = [
                  'phy:/dev/md/0,xvda1,w',
                  'phy:/dev/md/1,xvda2,w',
                  'phy:/dev/md/3,xvda3,w',
                  'phy:/dev/vg_1/lv_home,xvda4,w',
                  'phy:/dev/sda2,xvda5,w',
                  'phy:/dev/sdb2,xvda6,w',
              ]
 
name        = 'olddom0'
 
dhcp        = 'dhcp'
vif         = [ 'bridge=xenbr0,vifname=vif.oddom0.0,mac=00:16:3e:XX:XX:XX' ]

Attach network to domU

assuming lspci shows 0000:02:00.0 as your network adapter

This does not work with Xen-4.14 on debian bullseye, see https://patchew.org/Xen/20200619073315.8414-1-paul@xen.org/.

Using Xen-4.11 from debian buster on debian bullseye works.

Add

xen-pciback hide=(0000:02:00.0)

to /etc/initramfs-tools/modules, run update-initramfs -u Add

pci = [ "0000:02:00.0" ]

to /etc/xen/guest.cfg

Add (assuming your network driver is e1000e)

GRUB_CMDLINE_LINUX_XEN_REPLACE="$GRUB_CMDLINE_LINUX blacklist=e1000e"

to /etc/default/grub

If you install GRUB from the new Dom0, make sure no outdated device.map is lying around. Best use the –recheck option to grub-install

Manual steps:

rmmod e1000e
rmmod xen-pciback
modprobe xen-pciback "hide=(0000:02:00.0)"
modprobe e1000e
# check result
ip link
xl pci-assignable-list

Internal network

On dom0 (new one if you came from above):

xl network-attach //olddom0//
brctl addbr xenbr0
ifconfig xenbr 10.1.0.2 netmask 255.255.255.0
route add default gw 10.1.0.2

On domU (with internet access - former dom0 if you came from above)

assuming eth0 is the outer (physical) network interface and the xen virtual interface showed up as eth1

ifconfig eth0 10.1.0.1 netmask 255.255.255.0
INET_IP="`ifconfig eth0 | sed -n -e 's/^[[:space:]]*inet \([.0-9]\+\) .*$/\1/gp'`"
iptables -t nat -A POSTROUTING -o eth0 -s 10.1.0.0/24  ! -d  10.1.0.0/24 -j SNAT --to "$INET_IP"
sysctl -w net.ipv4.conf.all.forwarding=1

Make network card names persistent with your own (traditional) naming scheme

In domU connected to external interface

The ifupdown way
  • Give parameeter net.ifnames=0 to the kernel at boot time
  • Use mac/xx.xx.xx.xx.xx.xx.xx=eth0 in /etc/network/interfaces
/etc/network/interfaces.d/ifrename
rename mac/00:16:3e:XX:XX:XX=vif0
# rename mac/aa:bb:cc:??:??:??=eth0 # use this if you expect the network card to be replaced by another one of same type
rename mac/aa:bb:cc:dd:ee:ff=eth0
The systemd way
/etc/systemd/network/10-persistent-eth0.link
# https://wiki.debian.org/NetworkInterfaceNames
[Match]
MACAddress=aa:bb:cc:dd:ee:ff
 
[Link]
Name=eth0
/etc/systemd/network/11-persistent-vif0.link
# https://wiki.debian.org/NetworkInterfaceNames
[Match]
MACAddress=00:16:3e:XX:XX:XX
 
[Link]
Name=vif0

Use persistent name in /etc/network/interfaces

/etc/network/interfaces.d/vif0
auto vif0
iface vif0 inet static
 address 10.1.0.1
 netmask 255.255.255.0

Allow ssh root login

/etc/ssh/sshd_config
:
PermitRootLogin yes
:

Further reading

1) , 2)
Why doesn't it run on deb install time???
3)
that is modules diskfilter and mdraid1x
docs/tips_n_tricks/xen.html.txt · Last modified: 27.04.2022 00:28 CEST by peter