it kinda did give the impression that he was somehow abandoned. Although I doubt his father is still alive maybe his mother is, but for some reason had to get rid of him (poverty or bad health maybe...) and he really might have brothers and/or sisters somewhere, possibly older though... don't know why but it seems more likely.
That would be really cool and interesting, not to mention one of the reasons why Sorachi is taking so long to tell us about Gin's past. Makes you wonder what he's building up...