I've had the X100VI since spring last year and added the X-E5 with the 27mm this summer, mostly so I could go wider or longer without carrying two cameras. Four months in, here's where I've landed.
The X-E5 with the 27 pancake is almost the same size as the X100VI and the image quality is the same as far as I can tell, same 40MP sensor and the same recipes load on both. The X100VI wins on the hybrid viewfinder (I use the OVF way more than I expected!) and the leaf shutter for flash outdoors. The X-E5 wins because I can change lenses, and the film sim dial is honestly delightful.
For JPEG shooters like me they're both perfect. If I had to keep one... the X100VI, but only because I already own it and I'd feel silly selling it after waiting so long for it. If you're buying today and don't have either, get the X-E5 and a pancake. Ask me anything!