Welcome to The Cheater's Quickie Guide to updating WordPress
(Had you wondering there for a minute, huh?)
Over the course of a few years working with technology stuff, I have learned the value of healthy restraint when confronted with the urge to fix something that ain't broke.
Nonetheless I was very tired of seeing this little notice at the top of the WordPress dashboard on some of my blogs.

Here are the official WordPress instructions detailing the correct way to update http://codex.wordpress.org/Upgrading_WordPress_Extended
But ....I wondered if my Cheater's Quickie Method of updating WordPress would work with the jump from version 2.3.3 to 2.5.1. I can't imagine why it wouldn't, but then ya never know.
It worked.
Here's how to do it:
Prep work
Download the WordPress zip file
Unzip it
Backup the existing WordPress folders and files on your host: Truth is, I have been known to skip this item. You would be well advised to "do as I say, and not as I do". One thought: You will have your FTP program running anyway, and you ~could~ just FTP copies of the older files/folder to your desktop before proceeding. Then delete them after you confirm that the new version is working.
Deactivate any plugins
FTP to your host
Click on your WordPress folder to open it. Note if you have not changed the default setting, this file will be named "wp". If you are using a unique domain name with your wordpress blog, there will be some wordpress related files in the host's root directory. You don't need to touch those.

In the Host panel of your FTP program, with the wordpress folder open, click on the column title Modified, to sort by the files by date - check most recent date on your server panel. In my illustration the newest date is 2/8/2008.

In the My Computer panel of your FTP program sort by date Modified, select all files with a date later than the most recent date in your server panel, in this case, all files dates after 2/2/2008

Upload those newer files to your host, overwriting existing files

Delete wp-includes folder in your host panel
Delete wp-admin folder in your host panel
DO NOT DELETE wp-content folder

Upload the new wp-includes folder
Upload the new wp-admin folder
Close your FTP program
Go to your WordPress Dashboard
The first thing you will probably see it this, click Upgrade WordPress

Then click Continue

Done.
DISCLAIMER This works for me. I do not know if it will work for other users with other systems and/or configurations. Proceed at your own risk.