A tar.gz or tar.bz2 is a tarball with the source code of the application. Source code is the recipie for the software and it has to be compiled into a binary before it can be used. Tarballs are the old way of dealing with software in linux. Back when everyone who used linux was a developer, distributing your stuff as source code made sense.
Today however, most people using linux aren't developers, so they don't need the source code, the process of compiling is hard and it takes too long. RPM is one solution for this.
A RPM package is a container, it holds the binary files needed to run the software, not the source code. This means someone who knows how will compile the software for you, package it in a smart RPM package and distribute it.
When you install a RPM package, it will place the files in your file system so that you can run them. Files are placed in various folders depending on their type, executables (or binaries) are in a folder called "bin", libraries in a folder called "lib", documentation in "doc" and so forth. When you install a RPM package the files are placed in the right folders and a RPM database keeps track of where all the files are, what package they belong to and this makes it easier to keep control of your software.
When you compile and install a tarball, it too will copy the binaries to the right folders, but it won't keep track of the files and which files belong to which package. This means you'll have to manually keep track of your files, which is a lot of work.
The bottom line is, you want to use RPMs, always. Tarballs are an ancient form of installing software, intended for developers and advanced users who want to change stuff in the software. If something you need isn't available as an RPM package, you should ask someone who knows how to build one, or you can ask for an alternative. In Fedora Core, almost all software needed is available as RPM.
To make installation of RPM packages even easier, the process of downloading and installing has been automated using an automatic package tool called YUM. YUM will download and install a package, it's dependencies and install them. Much easier than hunting for packages yourself.
For more, read:
http://foolish.fedorausers.org/rpm/index.html