Agreed. You’re at the mercy of your USB controllers as well as southbridge in that case.
You’re adding more things that can go wrong.
Generally HBA/RAID cards are usually built for enterprise rather than consumer so they’re usually more reliable as well.
I use Homepage for something similar to this