I searched on Google, and found nothing on this subject, so I figured I’d write it here. Perhaps it’ll get indexed. I’ll probably post it on a forum as well.
Basically, the short answer is Yes, it can be done. The long answer is, are you sure you want to? It’s not particularly difficult, I just had some trouble with the format of a few files.
You CAN install both to the same HD, and use GRUB to boot between them. Easier would be to install each to a different HD and use GRUB.
Here’s what I did to make it work. I’m going to (after backing up the config files I created) try it another way soon. But this works:

You’ll need:
ESXi installer disc.
Citrix XenServer 5 Disc
Some Live CD (Like Ubuntu or Gentoo)

Install ESXi. Once that’s working, reboot to live CD.

The Partition Layout is as follows for ESXi:

1 - Extended
2 - “Ghost of VMware” as they call it
3 - VMFS
4 - Hypervisor0 - 4MB boot partition that determines if to boot from primary or backup partition
5- Hypervisor1 - Primary partition
6 - Hypervisor2 - Backup partition
7 - Type FC - Diagnostic partition (blank)
8 - Hypervisor3 - Locker


Here’s what you need to do (assuming disk is called sda):

Open a terminal
sudo -s
dd if=/dev/sda2 |gzip > /sda2.gz (~5000 MB partition, shrinks down)

cfdisk /dev/sda

Delete sda2 - Type 06 FAT16
Delete sda3 - Type FB - VMFS

Create a new partition, logical, 18 GB, beginning of space (this is the primary partition for the Xen install), type Linux (default)
Create a new partition, logical, 18 GB, beginning of space (backup partition for Xen, leave blank), type Linux (default)
Create a new partition, logical, size (Space - 5G) /2, beginning of space (Storage for Xen), type 8E Linux LVM
Create a new partition, primary, size 5000 MB, beginning, type 06 FAT 16
Create a new partition, primary, size Remaining, beginning, type FB

Take the “Boot” flag off partition 1. Add “Boot” flag to partition 9 (18GB primary Xen partition), as you will be using GRUB.

Save and Exit.

When done, this should be the layout:

Disk /dev/sda: 499.9 GB, 499989348352 bytes
64 heads, 32 sectors/track, 476827 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Device Boot Start End Blocks Id System
/dev/sda1 5 255381 261506048 5 Extended
/dev/sda2 255382 259476 4193280 6 FAT16
/dev/sda3 259477 476827 222567424 fb Unknown
/dev/sda4 1 4 4080 4 FAT16 <32M
/dev/sda5 5 52 49136 6 FAT16
/dev/sda6 53 100 49136 6 FAT16
/dev/sda7 101 210 112624 fc Unknown
/dev/sda8 211 750 552944 6 FAT16
/dev/sda9 * 751 17916 17577968 83 Linux
/dev/sda10 17917 35082 17577968 83 Linux
/dev/sda11 35083 255381 225586160 8e Linux LVM

Partition table entries are not in disk order

Copy the files from the Xen CD in packages.main/.bz2 to somewhere on your LiveCD environment (ie, copy them over ssh from another machine with the CD mounted).

mkfs.ext3 -L root-primary0 /dev/sda9

mkdir /tmp/root
mount /dev/sda9 /tmp/root
cd /tmp/root
tar jxvf /PATH/to/packages.main/.bz2 # Not sure this works, I did it one at a time. Might need to use xargs. Here’s the order the installer does it in:

license
el4.5-binaries
dom0fs
xapi
tools-iso
el4.6-binaries
xha
el5.2-binaries
el5.1-binaries
el5.0-binaries
el4.7-binaries
sm
docs

The high level view of the rest of the install is:

Un-tar files (done above)
Setup GRUB
Setup initrd
Setup swap
Setup networking
Setup Xen Inventory File
Set up FSTAB
Set up Time Zone, Keyboard, Internationalization
Set up storage

Because my blog software is having fits, I’m going to link to a simple html with the rest of the instructions:
ESXi and Xen Install
I am sorry for having to do that.
-Matt