Firstly your dog isn't mentally retarded - he's was abused. It sounds like his life was terrible so he obviously is going to have some issues.
Best advise is to start from scratch. Kennel or Crate him when you can't supervise him, tie him to your belt so he can wander off and do his business. Praise like crazy for doing his stuff outside, startle or scold (verbal only) for going inside.
It might be a longer process but he can learn. Praise works 10x faster than punishment so using a crate/kennel will give you more chances at praise. Take him outside at least once an hour at first so you can catch the act to praise and be sure to stay outside with him.
Based on his history, this could take months to fully set in.