For the several past years I was mainly using cheap 35mm cameras like the Holga 135 and Chinese SLR Great Wall PF-1 but now I prefer using medium format cameras like the Holga 120, Rolleicord II and my most recent Holga WPC (wide pinhole camera). I also own a Lomo LC-A compact which is small and very easy to use, that is why it's my everyday camera.
Many years ago I started using a Soviet Zorki camera and back then it wasn't a cheap as it was one of the best camera on the market. After 1989 I got myself a Yashica SLR. And for 10 years since digital cameras became available I was into digital photography. Then a few years back I decided to come back to film photography especially using cheap cameras.
I think with cheap and simple cameras you'll have more fun. You'll be able to push yourself to be more creative.
For me personally, I think that there isn't much fun using expensive digital cameras and technically perfect digital photos seems boring to me. Everybody is different, so just try whatever it is that you can get your hands to until you'll find the right camera for you.