Table of Contents
Xen
Start Instance
after you have written a config file /etc/xen/abcd.cfg
you have to create the instance by
xm create abcd.cfg
(I didn't expect the suffix .cfg
to be neccessary)
Get a console
xm console <Domain>
to disconnect type ctrl-]
(which means to press Str-AltGr-9 on a german keyboard)
Kill a running instance
xm destroy <Domain>
this will not(!) destroy your data or config, it will just “poweroff” the running virtual machine without a clean shutdown. (don't know what happens to snapshots)
Create new DomU
Assuming, you have a bridge xenbr0 up and running on your Dom0 with 10.1.0.1 beeing the outbound gateway and a disk partitioning scheme defined in /etc/xen-tools/partitions.d/my-disk-set
- create
/etc/xen-tools/skel/root/.ssh/authorized_keys
- run
create_guest.sh <name>
- create_guest.sh
#!/bin/bash set -o errexit name="${1:-xenguest}" xen-create-image --hostname="${name}" \ --randommac \ --ip=10.1.0.2 \ --gateway=10.1.0.1 \ --netmask=255.255.255.0 \ --mirror=http://http.debian.net/debian/ \ --dist=bullseye \ --lvm=vg_1 \ --bridge=xenbr0 \ --vifname=vif."${name}".0 \ --partitions=my-disk-set \ --vcpus=1 \ --memory=512Mb \ --arch=amd64 \ --nokeep \ --nohosts \ --boot \ --password "" \ --pygrub xl console "${name}"
- /etc/xen-tools/skel/root/.ssh/authorized_keys
# replace 202204260000 by some day in the near future and add your own sshkey expiry-time="202204260000" ssh-rsa AAAAB3N...
Matching physical and virtual Xen block (and other) devices and eventually their ids
xenstore-ls -f -s /local/domain/0/backend/vbd | egrep '(domain|frontend|dev|params) = '
References:
Turn a domU into a new dom0 running with serial console
This is for domU booted by pygrub
- on domU as guest of dom0
- give all filesystem a label
- replace device by label in /etc/fstab on domU
- replace dev by label for root fs in /boot/grub/menu.lst of domU
- install lvm2 and mdadm
- run update-initramfs if not done automatically
- on dom0 run
update-grub2
- boot into grub menu
- edit config for booting into the domU system on bare metal
- add
console=tty0 console=ttyS0,57600
to linux kernel line
- continue booting
- in domU system on bare metal, install grub-pc and
- edit
/etc/default/grub
GRUB_CMDLINE_LINUX=“console=tty0 console=ttyS0,57600”
GRUB_TERMINAL=serial
- run
grub-install –no-bootsector /dev/mapper/<lvm device>
1) - run
update-grub2
2) - label filesystems if not already done (see above)
- don't forget to relabel swap space
- edit
/etc/fstab
if not already done (see above)
- update /boot/grub/custom.cfg on (previous) dom0, so it boots domU system on bare metal via config …
insmod
line in grub/custom.cfg
!
- install xen packages (xen-system, xen-tools, grub-xen-host, …)
- edit
/etc/network/interfaces
to setup ip etc. correctly - allow root login by ssh (if you want it)
- run
mdadm –detail –brief –scan » /etc/mdadm/mdadm.conf
on domU system on bare metal - run u
pdate-initrams -k all -c
on domU system on bare metal
- reboot and …
Create a generic grub installation
- grub-install.sh
lvcreate -L 1G -n lv_GRUB0 /dev/vg_1 mke2fs -L "$HOSTNAME:GRUB0" -t ext4 /dev/vg_1/lv_GRUB0 mkdir /GRUB0 mount LABEL="$HOSTNAME:GRUB0" /GRUB0/ grub-install --boot-directory /GRUB0 --recheck /dev/sdb grub-install --boot-directory /GRUB0 --recheck /dev/sda
- /GRUB0/grub/grub.cfg
serial --speed=57600 --unit=0 --word=8 --parity=no --stop=1 terminal_input serial terminal_output serial set timeout=600 set default="dom0: LVM vg_1/dom0-root boot/grub/grub.cfg" menuentry 'Old dom0: /dev/md/0(/dev/sdb1,/dev/sda1) grub/grub.cfg' { insmod gzio insmod part_msdos insmod diskfilter insmod mdraid1x insmod lvm insmod ext2 set root='mduuid/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' echo 'Loading (/dev/md/0)/grub/grub.cfg' configfile /grub/grub.cfg } menuentry 'dom0: LVM vg_1/dom0-root boot/grub/grub.cfg' { insmod gzio insmod part_msdos insmod diskfilter insmod mdraid1x insmod lvm insmod ext2 echo "Setting root='lvm/vg_1-dom0--root'" set root='lvm/vg_1-dom0--root' echo 'Loading (/dev/vg_1/dom0-root)/boot/grub/grub.cfg' configfile /boot/grub/grub.cfg } menuentry "hd0 (MBR)" { insmod chain set root=(hd0) chainloader +1 } menuentry "hd1 (MBR)" { insmod chain set root=(hd1) chainloader +1 } menuentry "Reboot" { reboot } menuentry "Halt" { halt }
Use old dom0 as domU on new dom0
Assuming grub-xen-host is installed and /dev/md/0
is /boot
of old dom0
- /etc/xen/olddom0.cfg
kernel = '/usr/lib/grub-xen/grub-x86_64-xen.bin' vcpus = '1' memory = '2048' root = '/dev/xvda1 ro' disk = [ 'phy:/dev/md/0,xvda1,w', 'phy:/dev/md/1,xvda2,w', 'phy:/dev/md/3,xvda3,w', 'phy:/dev/vg_1/lv_home,xvda4,w', 'phy:/dev/sda2,xvda5,w', 'phy:/dev/sdb2,xvda6,w', ] name = 'olddom0' dhcp = 'dhcp' vif = [ 'bridge=xenbr0,vifname=vif.oddom0.0,mac=00:16:3e:XX:XX:XX' ]
Attach network to domU
assuming lspci shows 0000:02:00.0 as your network adapter
Using Xen-4.11 from debian buster on debian bullseye works.
Add
xen-pciback hide=(0000:02:00.0)
to /etc/initramfs-tools/modules
, run update-initramfs -u
Add
pci = [ "0000:02:00.0" ]
to /etc/xen/guest.cfg
Add (assuming your network driver is e1000e)
GRUB_CMDLINE_LINUX_XEN_REPLACE="$GRUB_CMDLINE_LINUX blacklist=e1000e"
to /etc/default/grub
device.map
is lying around. Best use the –recheck
option to grub-install
Manual steps:
rmmod e1000e rmmod xen-pciback modprobe xen-pciback "hide=(0000:02:00.0)" modprobe e1000e # check result ip link xl pci-assignable-list
Internal network
On dom0 (new one if you came from above):
xl network-attach //olddom0// brctl addbr xenbr0 ifconfig xenbr 10.1.0.2 netmask 255.255.255.0 route add default gw 10.1.0.2
On domU (with internet access - former dom0 if you came from above)
assuming eth0 is the outer (physical) network interface and the xen virtual interface showed up as eth1
ifconfig eth0 10.1.0.1 netmask 255.255.255.0 INET_IP="`ifconfig eth0 | sed -n -e 's/^[[:space:]]*inet \([.0-9]\+\) .*$/\1/gp'`" iptables -t nat -A POSTROUTING -o eth0 -s 10.1.0.0/24 ! -d 10.1.0.0/24 -j SNAT --to "$INET_IP" sysctl -w net.ipv4.conf.all.forwarding=1
Make network card names persistent with your own (traditional) naming scheme
In domU connected to external interface
The ifupdown way
- Give parameeter
net.ifnames=0
to the kernel at boot time - Use
mac/xx.xx.xx.xx.xx.xx.xx=eth0
in/etc/network/interfaces
- /etc/network/interfaces.d/ifrename
rename mac/00:16:3e:XX:XX:XX=vif0 # rename mac/aa:bb:cc:??:??:??=eth0 # use this if you expect the network card to be replaced by another one of same type rename mac/aa:bb:cc:dd:ee:ff=eth0
The systemd way
- /etc/systemd/network/10-persistent-eth0.link
# https://wiki.debian.org/NetworkInterfaceNames [Match] MACAddress=aa:bb:cc:dd:ee:ff [Link] Name=eth0
- /etc/systemd/network/11-persistent-vif0.link
# https://wiki.debian.org/NetworkInterfaceNames [Match] MACAddress=00:16:3e:XX:XX:XX [Link] Name=vif0
Use persistent name in /etc/network/interfaces
- /etc/network/interfaces.d/vif0
auto vif0 iface vif0 inet static address 10.1.0.1 netmask 255.255.255.0
Allow ssh root login
- /etc/ssh/sshd_config
: PermitRootLogin yes :
Further reading
- man xl.cfg
- Bugs I encountered
-
- Workaround: Fixed that by downgrading to Xen 4.11 from Debian Buster
-
- Workaround: Do not use
–nodhcp
, it is not needed if ip and netmask is given.
-
- Workaround: Don't reboot domU with PCI passthrough - shutdown and boot instead.
-