1. Ambient Lighting
One of the biggest mistakes people make is to take photos during the daytime, which almost always results in ugly reflections on the tank glass. The best time to photograph your fish is in the evening when it's dark outside. It's also wise to close all the curtains and turn off every light source around you except for the aquarium light fixture. This way you won't get any ugly glare in your photos.
2. Over-tank Lighting
After you have gotten all other sources of light extinguished, it's important that your over-tank lighting is as bright as possible. If possible, add a second light fixture on top of your tank. In a pinch you can even drag over a floor or desk lamp and position it directly over the top of your tank. Anything you can do to add extra light directly above the tank and shining down will improve the clarity of your photos. Photos taken under insufficient lighting tend to be more blurry.
3. Macro Setting
You don't need a super high-tech expensive camera to get decent photos. But the camera you use does need to have a good macro setting. This means that it can take very detailed close-up photos. The macro setting is usually indicated by a little flower icon.
4. No Flash
Using flash can make your photos look harsh and cheap because they don't allow the natural shadows on the fish to show. Instead, they highlight the entire fish (making it appear flattened) and create unnatural shadows in the background. Using flash also creates glare on the glass, which is something you're already trying to avoid!
5. Take LOTS of Photos
Good photographers take hundreds of photos just to get a few truly nice shots. Be prepared to spend a lot of time taking photos, only to throw out most of them and have a few really nice ones when it's all said and done. Good photography is just as much chance and luck as it is skill. You need to be ready to capture that picture-perfect moment when it happens, because your fish won't pose on-cue! You can better your chances of getting that perfect photo by taking many photos and throwing out the bad ones. Set a high standard for yourself and keep only the best of the best. I probably throw out about 6-8 photos for every one that I keep.
6. A Steady Hand or a Tripod
My biggest pet peeve is a blurry photo. If you know you don't have a very steady hand, you can always use a tripod. There are small inexpensive ones that can be purchased that work great. I have one with bendy legs that can be wrapped around chairs or other objects. I find a tripod most helpful though for full-tank photos and not as much for close-up photos, since the fish move around so much. It can also help if your camera has an auto-timer, because it defeats the point of using a tripod if you wiggle the camera by pressing down on the button to take the photo. You can set the auto-timer to take a photo after two-seconds, so it's perfectly still when the photo is taken, thus reducing blur.
7. A background That Suits Your Fish
I prefer black backgrounds on a goldfish tank, but you have to think about what background color works best for your fish. Obviously if you have mostly black fish, it will be tough to get good shots of them if you also have a black background. However, I've found that most other colors of fish do photograph really well on a black background. For black fish, I like a light brown or light gray background. Many people choose blue backgrounds for their goldfish because the blue compliments the orange/red colors found in many goldfish. The background you choose will be dependent on the colors of your fish.
8. Shutter Speed
Using a fast shutter speed will reduce the amount of light that the camera "sees", and so will produce less blurry photos.
9. A Photo-editing Program
Once you have done all the above steps and gotten a few really good shots, a photo-editing program can make them even better. Say, for example, you have a really clear shot of a fish in a unique pose, but the photo is crooked. Editing software can help you fix this and other small flaws to make your photos even better. You don't even need to use something like Photoshop, there are free photo-editing programs out there that work just fine for most minor fixes. One example is Windows Live Photo Gallery.
If you want to get better at photographing your fish, the best thing to do is practice! Along the way, you'll learn new tips and tricks that work well for you and your photos will continue to improve. Good luck!