Table of contents

Introduction

Hey there! As we continue our journey through the intricate setup of my home lab, I want to spotlight the backbone OS that powers everything: Proxmox. If you’re interested in virtualization or managing a variety of systems under one roof, you might find this discussion particularly enlightening.

Why Proxmox?

After experimenting with other platforms like VMware ESXi and XCP-ng (Xen), I’ve settled on Proxmox, and here’s why:

Trial-01: Experiencing VMware ESXi

VMware ESXi is undeniably robust but comes with its own set of challenges that impacted its viability for my setup:

• Backup Limitations: The backup process in ESXi, which requires exporting VMs to a local machine, felt overly restrictive and cumbersome, hindering operational efficiency.
• Peripheral Passthrough: While I managed to passthrough a GPU and use an attached monitor, the inability to passthrough a host-attached USB without a PCIe card was a significant drawback.
• Cost Concerns: The pricing structure of VMware ESXi turned into a “wallet breaker,” especially when the added costs of necessary hardware for full functionality are considered.
• Storage Handling: NFS and other storage solutions seemed to be treated as secondary, which did not align with my needs for first-class storage integration.
• GPU Limitations: Inserting their GPU driver leads to the infamous code 43 error in Windows if not disabled correctly, complicating the setup further.

Drawbacks Summary

These limitations made ESXi less appealing for a home lab environment where flexibility, cost-effectiveness, and hardware integration are paramount.

Server 2: XCP-ng (Xen)

XCP-ng presented a different set of features and challenges:

• First-Class NFS Support: Unlike VMware, XCP-ng offers excellent NFS support, indicating a stronger focus on integrating diverse storage solutions seamlessly.
• Guest Tools: The lack of comprehensive guest tools made it slightly less user-friendly, particularly for users who thrive in an “open playground” like Proxmox.
• Learning Curve: XCP-ng is well-suited for those with a clear understanding of their virtualization needs and the technical prowess to utilize its capabilities fully.

Special Considerations

For scenarios focused purely on networking and Kubernetes, without the need for GPU or other peripheral support, XCP-ng shines. Its ability to orchestrate single VMs across multiple nodes, without JavaScript installed on every cluster, makes it highly effective, especially in enterprise environments where downtime and security is critically managed.

Comparative Conclusion

While VMware ESXi offers robustness at a high cost, both financially and in terms of flexibility, XCP-ng provides a more specialized platform that excels in certain technical scenarios but lacks the broad usability and tooling that Proxmox offers. For a home lab where experimentation and learning are key, Proxmox’s open-ended, versatile environment, coupled with robust support for a variety of file systems and backup solutions, ultimately makes it the superior choice for my needs.

Choosing Proxmox

Proxmox, on the other hand, aligns perfectly with my technical and creative requirements:

• Robust Backup Solutions: Utilizing Zstandard (zstd) for compression, backups are efficiently stored on my NAS. This was a significant upgrade in terms of both performance and convenience.
• File System Flexibility: Whether you’re a fan of ZFS, Btrfs, or prefer the old school LVM, Proxmox offers support for these without hesitation.
• Simple and Effective Networking: It ‘just works’—a critical feature for any OS handling complex network setups.

Technical Deep Dive into Proxmox

Proxmox is a Debian-based OS that integrates beautifully with a web UI and KVM for virtualization, providing that essential “tinker vibes”:

• AMD GPU Passthrough: Despite some limitations with AMD GPUs where a reset isn’t possible, being on Debian means I can compile my own dpkg with the vendor reset patch, getting my GPU back on track with ease.
• Networking Setup: I’ve mapped the physical network interfaces to virtual bridges which are then utilized in VyOS. This setup exemplifies the versatility and control Proxmox affords in a network-intensive environment.

What’s Next?

Stay tuned for an in-depth look at my networking configurations in Proxmox. I’m excited to share how I manage and optimize the virtual networks, ensuring smooth and secure communications across my devices and services. This post is slated for release next week, so keep your eyes peeled for more practical insights!

Thank you for reading, and as always, I’m open to questions and suggestions. Feel free to reach out via email to discuss more or share your own experiences.