I started with the Raspberry Pi Model B Board and starter kit and set it up as described in the Raspberry Pi User Guide. You can get the Raspberry Pi as a kit with a USB wifi dongle, keyboard and mouse, or on its own if you already have a mouse and keyboard.
I configured my Pi to run at 800MHz instead of the default 700MHz. I installed the Apache web server, and copied some web pages to Apache's root directory. My Pi is connected to my router with an ethernet cable as this will provide better performance than a wifi connection.
Next, I set up port forwarding in my router so that people can connect to my Raspberry Pi from outside of my firewall. I created an account with no-ip.com, a service which redirects a web domain to my IP address.
I've set up a blog where I'll go into more detail here.