The best way to start keeping chickens is to keep it simple. Start with 2 or 3 chickens, not a flock of 30. Start with relatively cheap, relatively standard chickens that do well in your area instead of a super rare breed.
Chickens are at their most vulnerable when they are young, so while it is fun to raise them from eggs or day-old chicks, it may really be best to get 2-3 juveniles (pullets) or young adults. For many reasons, you should start out with hens and you should buy from a place where you know the chickens you are getting are hens. Probably not the best idea to start in the dead of winter if you live in a cold place.
Space and shelter
Chickens need space to move around, some sort of protected space on the ground, protection from the elements, and a reasonable place to lay. A small commercially available coop is a good way to start. It will be the most expensive part of the process. The funny thing about keeping chickens is that the chickens themselves are basically the least expensive part of it.
Food and water
They need food and water. I get layer pellets at the local farm and garden supply store (Benedict's). Some people geek out over what kind of feed to give them. I just get regular layer pellets (not crumble). I get bedding (wood shavings) too to make the inside of the coop more cozy.
For water, I have found that fancy waterers don't last, and I use buckets. Fancy doors that open and close on timers also don't last. Water is life. If the chickens run out of water, they will die quickly.
Water is life. If the chickens run out of water, they will die quickly.
We give scraps (except chicken) to our flock. I would view scraps as a supplement to a basic diet of layer pellets.
The short version
If you want to start, get a small commercial coop ($100–300) and maybe 3 juvenile or young adult chickens. Make sure you have food out for them (no worry of them over-eating so it can just be out) and always have water.