Over the past few months, I've been using an iPod touch (2nd gen) as a replacement for my previous PDA, a Palm TX. It has been a generally positive transition, though there are things the TX can do that the iPod can't or that worked better on the TX.
I've been using it both long enough and intensely enough to get past the initial adjustment period; I've stopped trying to hit buttons that don't exist, and operations like "turning it on" (which is really just unlocking the screen) are now second nature.
I'd been using some kind of Palm OS PDA since the 1999-vintage Palm V (which replaced an HP 200LX). I ran through a series of Palm OS units until the TX, which is the last Palm OS PDA (and most likely the last PDA of any sort) that Palm will make. I never went for a Palm OS phone, because I don't like having everything tied in to one device (and often one network); the same thing has kept me away from the iPhone. (If they start selling unlocked ones in the US, I'll reconsider. No, third party unlocks don't count.)
A comparison of various aspects of the TX and the iPod touch:
Calendar: The Palm's Datebook application wins this, hands down. Entering a new event is simple and direct (click on a blank line, enter a title, then possibly adjust the start and end times; it'll get the default alarm), where on the iPod touch it takes multiple sub-screens (hit the "+" button, then click on the Title/Location, enter it, hit Save, click on the Starts/Ends button, spin the wheels to set date and time, hit Save, click on Alert, choose from the minimal selection of possible alert times). There's also no option to set alarms to repeat (on the Palm I used "every minute for 10 minutes") or to snooze them when they go off.
This is the biggest weakness of the iPod touch, and presumably also of the iPhone. Most of my events are added on the PDA rather than on the desktop; entry has to get easier than this.
Contacts: The iPod takes this one. The quick-access "initial letters" scrollbar-ish thing? Beautiful. Skimming through records? Simple. The one thing lacking is that the list view doesn't show the primary phone number, like the TX did. (OTOH, with iSync I just sync the important numbers to my cell anyway.)
Mail: I dumped the built-in Versamail on the TX and moved to SnapperMail, so I'll compare that to the iPod's Mail app.
SnapperMail: it's designed for disconnected access, so I can hit "Send/Receive" and it'll pull down all the defined mailboxes for later perusal with just one click. It doesn't do much with HTML email. I have to specify which mailboxes it's syncing.
iPod Mail: it seems faster. It doesn't require paying extra for the "Enterprise" version of a third-party app. It doesn't automatically pull mail from anything but INBOX unless I enter that mailbox while I have connectivity, though it has a nice "Updated" line at the bottom of the screen so I can see how fresh the mailbox is. While it sees all my mailboxes, it doesn't have any way to collapse folders for easier navigation. It renders HTML mail nicely, but a bit too nicely since it will pull down remote images without asking if you open a message. It doesn't handle event invitations in email.
It took some getting used to, but the iPod takes this one. A few improvements and it wouldn't even be a contest.
Syncing (basic apps): This too goes to the iPod. Missing Sync worked for my Palm sync needs, but costs extra (and required a for-pay update to get Leopard compatibility, with a price justified by new features that were almost all Treo-only) and required both plugging in the TX and hitting the HotSync button. The iPod syncs when plugged in, automatically. If I set up MobileMe, it'd even sync wirelessly wherever I have connectivity. (I'm still debating that one; I like the idea, but I also like knowing that my stuff is definitely synced once I've plugged in the cable rather than depending on someone else's cloud to do it.)
Syncing (third-party apps): This includes loading files for e-book readers and the like. For the Palm, I either stuck them in an install directory and HotSynced, or put them on an SD card; with the iPod, each app usually has some kind of over-the-air loading using Bonjour or the ability to pull files off the Web directly. Call this a slight advantage for the Palm, since I can batch stuff up and then sync it all at once with that approach.
Media: I never even bothered with music or video on the TX, since I had an iPod for that. The storage capacity and more media-centric software makes this an easy call: iPod wins.
Book reading: A win for the iPod. The screen is much nicer. The extra built-in storage means I don't have to either swap books off of an SD card or risk wearing out the contacts by leaving the card in all the time. Since most of my books are in eReader format, I have access to even the DRMed ones without any trouble. Navigating the book list is far better in the iPhone OS version of eReader. There are minor disadvantages: it can be harder to hit small links (for footnotes) due to the lack of a stylus, and there are fewer font choices in the iPhone OS version of eReader. The only significant disadvantage is the lack of a Mobipocket DRM-capable reader which may be Amazon trying to kill a Kindle competitor. This xkcd comic applies just as well to e-books as it does to music; thank you, Baen, for not DRMing your books.
Games: iPod all the way; though a few of my old Palm OS favorites aren't available yet, the new games more than make up for it. Great free games include a Rogue port and a reimplementation of the Atari 2600 Adventure; for a few bucks, Galcon, Frenzic, and Mondo Solitaire (regular or the Top 5 version, depending on which games you play) are all recommended.
Web access: Mobile Safari takes Blazer out back and beats it to death without even breaking a sweat. The only thing Blazer does even slightly better is in saving pages for later offline reference, and there are other ways to do that on the iPod.
Reference/Utility: The various Palm apps for airline and MBTA schedules were nice to have, and I hope at least the former will start showing up on the iPhone sooner or later. (The MBTA app is no longer being updated as the vendor went poof, so I don't expect a port; it'd be nice if the T started supporting GTFS though.) There isn't a port of Metro yet, either. On the other hand: Wikipanion rocks, Air Sharing lets me put a lot of reference material in PDF form on the iPod, CSV Touch does the same for tabular data, and Splash ID has replaced the old Strip application as a password vault.
Connectivity/Net-based apps: The big thing the iPod needs is the ability to tether over Bluetooth, or for there to be a legitimately-unlocked GSM iPhone I can buy to use on T-Mobile US without having to go to Hong Kong or Taiwan or something. Apart from that, the iPod has NetNewsWire; great apps for things like Amazon, LJ, BoardGameGeek, and Facebook; fun toys like Urbanspoon and Google Earth; a Maps app that would be a whole lot nicer if it hadn't been nerfed compared to the iPhone version; YouTube support; and last but definitely not least, the amazingly cool Remote.
Conclusion: All in all, the iPod touch is a respectable PDA replacement for the Palm TX. There are still areas that could stand improvement (Calendar, and to a lesser extent Mail, are the big culprits) and the major missing feature of Bluetooth support, but those are outweighed by all kinds of other advantages. It also lets me leave the iPod classic at home (except for long trips), since I can use the iPod touch for day-to-day listening.