If you can't add to it but are satisfied with the games it comes with, then it isn't a bad deal. I know there are some android-based handheld emulators that allow you to download all of the NES ROMs and play them, save progress, etc. Thinkgeek used to carry one, but I couldn't find it just now.
I still don't fully understand why Nintendo doesn't port all of the original NES, Super NES and 64 games to its handheld system, selling them cheap or bundling them together. They could make yet more money off of these games, particularly the ones they own outright like Mario and Zelda.
Well, in the case of 3rd party games, the rights issues (lots of odd ball developers that have been bought, sold, rebranded, etc) and costs are probably a logistical and financial nightmare. In the case of their own games, they probably don't want to cannibalize sales - why sell cheap, when people are buying NES Virtual Console games all the time at $5 a pop? In fact, I'm a little surprised the NES mini has such a large and solid collection of games - $2 a piece is a big discount from the VC store, on top of the hardware.
And yeah, you can pretty easily buy a handheld device intended for this sort of thing. I have an Ipega 9032, which lets you stick a tablet or smartphone in it with emulators on it. Think it was about $20.