I setup another proxmox server still on a subnet/vlan and was also able to get that working. So its definitely something specific to how I set up the VLAN/Trunk on the switch to the original proxmox or the proxmox networking.
Good lesson learned on trying out simpler setups with fresh/separate devices first.