Pray for Death (1985) is another ninja revenge option. Totally awful movie, though, and although the plot does not match, it might be close to the OP's movie. Revenge of the Ninja is one of my all time favourite ninja movies, but it doesn't really match the description from the OP at all.
Can the OP provide some more details, like:
- when did you see the movie (rough time period to help narrow the search)
- was it on VHS, DVD or cable?
- was it in English?