There are 3 image file types that are very common, gif, jpg and png. And while the differences between them can and do fill a lecture, I'll only go over a few.
GIFs are probably the oldest of the image file types used. It's best for small images with solid colors in them. The file size is usually small allowing the images to load quickly.
Gif images are loss-less. That means that when a Gif is saved it will not get blurry or messy like JPGs can.
This is an image of a Chinese symbol (for luck).
Saved as a Gif | Saved as a JPG |

| 
|
It's hard to see the difference in the two above but below is an enlargement of each time, this clearly shows that the GIF retains it's clarity while the JPG looses it's clarity. (Both closeups are saved as PNG files)
Closeup of Gif | Close up of JPG |

| 
|
Now while the example above shows that JPG's can loose quality when saved, they do excel and was designed for displaying photographs. Gif's can only display 256 colors, while JPG's can display millions of colors. And while yes they do loose something when saved, JPG's quality is still good enough that they are the best way to display images.
A compression setting of about 60% will result in the optimum balance of quality and filesize, without letting the lossy compression do too much damage.
Here is an example picture
The below shows a small section, one saved as a gif, one as a jpg. You can easily see the difference.
As Gif | As JPG |

| 
|
PNGs were created to improve on and replace Gifs for web use. An image with a shadow against a transparent background can be saved as a PNG and retain that shadow while they cannot as a GIF.
However, Internet Explorer 6 doesn't support transparency and so using PNGs is not as popular as it might be. Once IE6 goes away it's likely that there will be more use of png's and complex images.
There are many more image filetypes. Tiff, BMP, Exif, Raw, etc. For a variety of reasons, most of these are not appropriate for web use.
Use jpgs for images or very complex graphics. Use Gif's for clipart, and simple illustrations. Until IE6 goes away, I'd stay away from png's in content.
HTML Source - Image file formats
Wikipedia - Image file formats
FileInfo - Image file formats (all of em)