The amount of space available on an SD card is much more limited than the amount of space on a hard disk, so it's worth attaching a hard disk to your Pi. The simplest to expand your Pi's storage capacity is with a USB hard disk.
If you're planning on leaving the same USB hard disk connected to your Raspberry Pi all the time, then you need to make sure that your Pi mounts the drive automatically every time it boots. You can do this by editing the file system table file.
When Linux detects the USB drive, it will create a file in /dev that is used as an interface to your disk. You can list all the disk device files in /dev by typing this command:
$ sudo fdisk -l
Disk /dev/mmcblk0: 3965 MB, 3965190144 bytes
4 heads, 16 sectors/track, 121008 cylinders, total 7744512 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00014d34
Device Boot Start End Blocks Id System
/dev/mmcblk0p1 8192 122879 57344 c W95 FAT32 (LBA)
/dev/mmcblk0p2 122880 7744511 3810816 83 Linux
Disk /dev/sda: 320.1 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders, total 625142448 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x8be4e163
Device Boot Start End Blocks Id System
/dev/sda1 * 63 625137344 312568641 b W95 FAT32
My USB disk is listed in /dev/sda1, and the file system type is Fat32. If your USB drive has an NTFS 3g file system, you will need to install an NTFS driver.
sudo apt-get install ntfs-3g
In order to access the files and folders on a disk, Linux needs to mount it. The contents of the disk will appear as a folder in /media. You can mount disks in other folders, but it's conventional to use /media. You need to create a directory where the mounted disk will appear in the media directory, and change its owner to pi (or any other user account that you might have created).
$ sudo mkdir /media/usbhdd
$ sudo chown pi:pi /media/usbhdd
In order to mount the disk, type this command:
$ sudo mount -t vfat -o uid=pi,gid=pi /dev/sda1 /media/usbhdd
The '-t vfat' tells the mount command that your drive has a fat32 file system. If your drive is formatted with NTFS, you should use '-t ntfs-3g' instead. The '-o uid=pi,gid=pi' part of the command means that the disk will be owned by user pi. You can use this command to unmount the disk:
$ sudo umount /media/usbhdd
Now you need to edit the file system table so that this disk is mounted every time your Raspberry Pi starts up:
$ sudo leafpad /etc/fstab &
You need to use sudo because the fstab file is owned by root. If you don't use sudo, you'll be able to open the file in leafpad, but you won't be able to save changes. The '&' means the command runs in the background, and you can keep using the terminal for other commands while leafpad is running. You should see something like this:
proc /proc proc defaults 0 0
/dev/mmcblk0p1 /boot vfat defaults 0 2
/dev/mmcblk0p2 / ext4 defaults,noatime 0 1
Add the following line, and save the file.
/dev/sda1 /media/usbhdd vfat uid=pi,gid=pi 0 0
Reboot your Pi to and you should be able to access your USB drive via /media/usbhdd.
Computers have hostnames so that they can be addressed by a human readable name as well as by their IP address. The default hostname for your Raspberry Pi is raspberrypi.
You might want to change it if you have more than one Pi, or just to make your Pi your own. Changing the hostname is very easy. You just need to open two files, /etc/hosts and /etc/hostname, and replace the word raspberrypi with your chosen hostname. These files are owned by root, so you need to open them as root. I changed my Pi's hostname to rpi. The contents of /etc/hostname is simply
/etc/hosts should look like this:
::1 localhost ip6-localhost ip6-loopback
Reboot your Pi using this command:
When you open a terminal, the prompt should look like this:
You can share your Raspberry Pi's files and folders across a network using a piece of software called Samba, a Linux implementation of the Server Message Block protocol. You'll need to install this software:
$ sudo apt-get install samba samba-common-bin
Samba contains the SMB protocol, support for the Windows naming service (WINS), and support for joining Windows workgroups. A workgroup is a group of computers on a local network that can access eachother's folders. Samba-common-bin contains a tool that you'll need to register users with Samba. Once these packages have finished installing, you need to edit the Samba configuration file:
$ sudo leafpad /etc/samba/smb.conf &
Find the entries for workgroup and wins support, and set them up as follows:
workgroup = your_workgroup_name
wins support = yes
The name of the workgroup can be anything you want, as long as it only contains alphabetical characters, and it matches the name of the workgroup that you want to join.
You also need to add the following section of code to smb.conf:
comment= Pi Home
Scroll down smb.conf until you see a section called Share Definitions, and add this code there. The path should point to the drive or folder that you want to share. I've set 'only guest' and 'public' to 'no' so that Samba prompts for a password when I visit the folder that I've shared. This means that when I'm using a Windows PC, I can login to the shared folders on my Pi, and I'll have the same read/write permissions that user pi has.
Now type this command in a terminal, and enter pi's password twice:
$ smbpasswd -a pi
If you have a PC or laptop connected to your workgroup, you should be able to see your Raspberry Pi in Windows Explorer under Network.
Being able to back up data to the cloud is very useful. It means that even if your Raspberry Pi dies or your SD card gets corrupted, your data is still safe. It also means that you can access your data from any where in the world.
If you have a Google drive account, you can use the grive program to sync a folder on your Pi with your Google drive.
Start by making sure that your Pi's software is up to date, install some additional packages, and get the code using git:
$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get install git cmake build-essential libgcrypt11-dev libjson0-dev libcurl4-openssl-dev libexpat1-dev libboost-filesystem-dev libboost-program-options-dev binutils-dev libboost-test-dev libqt4-dev libyajl-dev
$ git clone git://github.com/Grive/grive.git
Modify /home/pi/grive/libgrive/src/drive/State.cc to resolve some compiler problems. Lines that use the Add() method need to contain a cast to the correct size of integer. The expression "(boost::uint64_t)" must be added to lines 251, 252 and 256. The updated code should look like this:
void State::Write( const fs::path& filename ) const
Json last_sync ;
last_sync.Add( "sec", Json((boost::uint64_t)m_last_sync.Sec() ) );
last_sync.Add( "nsec", Json((boost::uint64_t)m_last_sync.NanoSec() ) );
Json result ;
result.Add( "last_sync", last_sync ) ;
result.Add( "change_stamp", Json((boost::uint64_t)m_cstamp) ) ;
std::ofstream fs( filename.string().c_str() ) ;
fs << result ;
Now you need to configure the make files by running cmake (note the '.' after the cmake command) and compile the source code by running make:
$ cd ./grive
$ cmake .
You need to create a directory where you can store files that you want to be syncronized with your Google drive, and copy the grive executable to that directory.
$ mkdir ~/google_drive
$ cp ./grive/grive ~/google_drive
The first time you execute grive, you need to use the -a option in order to authenticate with Google. You must be logged into your Google account from your Pi for this to work. If you're logged into your Google account on a PC or laptop, log out, and then log in again using a browser on your Pi. Once you've done this, change to your Google drive directory and run grive.
$ cd ../google_drive/
$ ./grive -a
A link will be printed in the terminal window. Copy the link and paste it in a browser. A page on google.com will appear with a long code that you need to enter in the terminal window. Pasting the code didn't work for me, so you may have to type it in. The authentication process will then take place, and your Pi will sync for the first time. Before you do this, check to see how much space you've used up on your Google drive, and check to see if there's enough space on your Pi's SD card.
Running a backup to the cloud
If you've got this far, you should now be able to sync files and folders in /home/pi/google_drive with your Google drive. The next step is to backup the data on your Pi and upload it to your Google drive.
The following bash script creates a tar archive containing my home directory. I've made sure not to include the Google drive directory in this archive, otherwise previous archives would be included in the backup. The second line compresses the archive into a .tar.gz file. The file is moved to the google drive directory before running grive to upload the new archive to your Google drive on the internet. I've used the date command to embed the current date in the file name.
tar -crvf backup_$(date +%y.%m.%d).tar /home/pi --exclude="/home/pi/google_drive"
gzip backup_$(date +%y.%m.%d).tar
mv backup_$(date +%y.%m.%d).tar.gz ./google_drive
Save this code as grive_backup.sh, and make it executable using the chmod +x command.
When you run the script, it may appear to hang while gzip is running. This may take a few minutes depending on how much data there is in your home directory.
Running a backup script as cron job
This script can eaily be automated by creating an entry in the crontab file. Linux uses the crontab file to schedule tasks. Entries are made in the format
minute hour day_of_month month day_of_week username command
Use this command to open crontab:
$ sudo leafpad /etc/crontab
...and then add this line:
00 03 * * * pi /home/pi/grive_backup.sh
This specifies that you want to run grive_backup.sh at 3.00am every day, and it should be run as user pi. Linux should automatically detect that your crontab file has been modified, so there's no need to reboot.
Your SD card will fill up after a few days or weeks, so it's best to remove old backups. You can do this by adding the following line to the script above:
rm backup_$(date --date="7 days ago" +%y.%m.%d).tar.gz
This time the date command is being used to create a date from 7 days ago, so a back up file from 7 days ago will be deleted.
Many people are using their Raspberry Pi as a home web server. It's a great way to host a small site without having to leave a PC running 24/7. If you want people to see your Raspberry Pi web site on the internet, you need to forward a domain name to your home IP address, and then forward a port on your router to your Pi. In this article I'm going to assume that you've already set up a web site and you need to just need to make it accessible on the web.
The diagram on the right shows a typical network set up for a Raspberry Pi. The Pi is connected to a router by wifi or ethernet. I use ethernet for my server as it's more stable.
The router has an external IP address which it uses to communicate with the internet, and it has another IP address which it uses to
communicate with your home network. Let's say the router's external IP address is 22.214.171.124. The router's internal address acts as a gateway for all the devices on your network. When your PC requests data from the internet, requests are automatically sent to the gateway address, and the router forwards the request to the internet. The gateway address is usually something like 192.168.0.1.
Most home routers have a firewall built-in. This stops people from outside of your home network connecting to your PC, but it allows your PC to make outgoing connections to web servers.
The external IP address of your home router is usually assigned by your ISP's DHCP servers. If you need to restart your router, it will make a new DHCP request to your ISP, and it may be given a new IP address.
How are web requests processed?
When a user visits a web site, they use the site's domain name to address the site. Their browser then sends a request to a DNS server to translate the domain name into an IP address. Once the browser has the site's IP address, it sends a request to the server.
There are two problems with this:
- 1. the external IP address of your home network can change at any time, and
- 2. your Pi is behind a firewall and can't be accessed directly from the internet.
The first problem can be solved by using a dynamic DNS service. These services act as DNS servers that update automactically if your IP address changes. There are numerous dynamic DNS services available. Most of them provide a free service where you can choose a subdomain name.
You need to install a small piece of software on your Pi. This software is supplied by your dynamic DNS provider, and it monitors the external address of your router. If that address changes, the software sends a message to the dynamic DNS server to update its records.
A dynamic DNS system specifically for Raspberry Pi servers is due to be launched in October/November 2013. You can read more about this at rpidns.co.uk.
When a user on the internet wants to see your web site, their browser can get the IP address of your home network from your dynamic DNS service. But requests still need to find a way through your router's firewall.
By default web servers listen for connections on port 80. When a user's request to see a page on your site reaches your router, the router must forward that request to your Pi's IP address. You don't want to forward all incoming traffic to your Pi, just the traffic for the web server which is on port 80. Most routers allow you to set up port forwarding so that any traffic on port 80 will be sent to your Pi. Instructions on how to do this vary from one router to the next, but you can find some pointers at wikiHow.com.
In most routers there are options to use an existing forwarding rule for HTTP traffic on port 80. If the rule doesn't already exist, you'll need to create one. If you are prompted to choose between TCP and UDP, pick TCP. You'll need to specify your Pi's IP address, so you need to give your Pi a static IP address.
Note that some ISPs block port 80 to prevent their customers from running web servers (to reduce bandwidth usage). In this case, you need to configure your server to listen on a different port, say port 87. You would need to set up your router to forward traffic on port 87, and you would need to configure your dynamic DNS service to forward requests on port 80 to port 87.
You may need to restart your router for changes to take effect.
You can test your set-up by opening a web browser and going to your domain name. If all is well, you should see your site appear in your browser.
Once everything's working, you might want to make it easier for people to find it. You can start by setting up an account in Google Webmaster tools and submitting a sitemap. You could also register your site on pirepository.com, a site that lists web sites that run on Raspberry Pi servers.
Sometimes it's useful to automatically run a script on a Pi when it boots up. You might want to start a script that monitors sensors, a Bittorrent client, or a program to back up your Pi to the internet. There are several ways of doing this, and each one has it's benefits.
In the following examples I'm going to start a program called servod which manages a servo. The program runs in the background and uses the Pi's GPIO pins to set the position of a servo.
The simplest method is to put a command in /etc/rc.local. This file is executed while Linux is booting up and before any users have logged in.
When rc.local executes, the path environmanet variable (or any other environment variable) may not have been set up so it's important to use the program or script's full path. It's also important to make sure the program doesn't prompt for user input, as users won't have logged in when rc.local runs. If the program waits for user input, it won't complete, and the boot process will grind to a halt before users can login.
Similar problems can occur with programs that aren't meant to complete like servod, which runs continuously as a service or daemon. The solution in this instance is to run servod in the background using the '&' symbol. This tells the bash shell to execute the program without waiting for it to complete.
The cron daemon is a Linux thread that runs once a minute. It checks a file called the crontab file to see if there are any jobs that need to be executed. Usually entries in crontab specify a time to run tasks, but '@reboot' can be used to tell cron to run a task as soon as your Pi boots. You can edit a crontab file by typing this command at a terminal:
sudo crontab -e
Your crontab file will be opened in the nano editor. Use the arrow keys to move the cursor to the end of the file and add the command that you want to run:
To save changes to the file, press control-o. You will be prompted to save to a temporary file name. Press return, and the crontab program will install the temporary file in the correct place. It's best practice to use a program or script's full path in crontab files.
Sometimes it's necessary to restart a service if it's configuration files have changed. For example, if Apache's configuration files are modified, it needs to be restarted using this command:
sudo /etc/init.d/apache2 restart
This command executes a script in /etc/init.d/ which can start, stop or restart the Apache web server. You can create your own init.d script for programs that you want to run at boot time. It should be put in /etc/init.d. It must start with a shebang (#!/bin/bash) followed by a LSB header, a set of specially formatted comments that contain information about the service.
This is an init script that I created to control servod:
### BEGIN INIT INFO
# Provides: servoblaster
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Example initscript
# Description: This service is used to manage a servo
### END INIT INFO
case "$1" in
echo "Starting servoblaster"
echo "Stopping servoblaster"
echo "Usage: /etc/init.d/servod start|stop"
An example of the header can be found in /etc/init.d/skeleton.
The code that follows the LSB is usually a bash case statment for processing the start/stop command line option. The option is stored in $1, and the case statement tests to see if the option is start or stop. If no option (or the wrong option) was supplied the default case executes, and a usage message is printed. If the start option is used, the script runs servod, and if the option is stop, the script kills the servod process.
Sometimes when a computer is shutdown, services need to be stopped in a controlled way. Programs may need to finish logging data or release lock files before they are terminated. If a script is started using cron or rc.local, it will simply stop when the Pi is shutdown. Using the init.d method to control scripts means that when your Pi shuts down, Linux will gracefully shutdown the script so that it has a chance to finish off any final tasks.
I used chmod to make the script executable:
sudo chmod +x /etc/init.d/servod
I can start the servod program with this command:
sudo /etc/init.d/servod start
...and stop it again with this one:
sudo /etc/init.d/servod stop
In order to make the program run on start up, it's necessary to run this command:
sudo update-rc.d /etc/init.d/servod defaults
This creates a link to /etc/init.d/servod in directories from /etc/rc0.d through to /etc/rc6.d. When Linux boots up or shuts down, it looks in these folders to see if any scripts or programs need to be run. When I restart my Pi the servod program starts automatically.