Self hosting a website is more than just paying the $2 for a VPS. The stakes are higher than that.
1. Time - when self hosting you have to set aside time everyday to check the server is up, healthy and secure. This is an everyday task, you cant just assume all is well and ignore the server. And remember time is an expense!
2. 3rd party tools - more often, u gonna need a couple of PAID 3rd party tools to enhance your server security and usability. Think of things like cPanel ($22/m), antivirus, anti spam, mail scanners etc....some come for free but we all know what u get for free.
3. Server Admin - you might need some server admin expertise once in a while to fix those unwelcome, unexpected problems. Imagine if u wake up one day and your server is not sending emails and you don't know how to fix it. We all know how expensive hired short time labor is! If you are already an expert in server admin, refer to point 1.
4. Uptime - your website has to be up and running 247 so self-hosting will definitely eat a sizeable percentage of your time. And this becomes tricky in cases where u have to be "offline" for a few days or even weeks! Will u sleep at night not knowing if your website is up? Imagine u go to rural Mwenezi for one of those annual family gatherings and you are "stuck" there for a week, with no ZESA or Internet access.
5. Backup - most of these VPS guys don't offer any backup whatsoever. So what happens if your server goes boom and u r left with egg on ur face? So to be safe, u now need a backup VPS owned by a difference company, in a different geographic location. More expense...
i could go on and on here. Lets not get carried away guys. unless your website is just one of those online pages that rarely generate important traffic and u don't have any customers, then you can self host and experiment for all you care! if you are in serious business then get a proper web hosting company to host for u and concentrate on sweet-talking clients.