The cheapest camera that I have and I'm currently using is an Hanimex 35SE which is 35mm film camera. I got it for 5 euro in a thrift store.
One of the reason why I started to do photography is that I have this fear of forgetting. I like to keep a physical memory of every moment of my life, so I won't forget about them. In the process I also learn the necessity to transmit/evoke what I think, what I feel and the way I see the world.
Photography is not mechanical like a computer. Photography is about telling a story. The first photograph in history was taken with a pinhole camera, a simple box with a hole in it, and from there photography evolved into what it is today. I know that if you're just in the first stages of learning photography these days that it's very easy to become obsessed with quality and technology, as we all do. But over time, you'll realize that it is only a small part of what it really means to take/create a photograph. We shouldn't just focus on quality because we must learn to look beyond what we see, to step outside the box, to be aware of the background that makes up a picture as a whole, its history and the emotion behind its creation.