I would be moistening the dry food still, but you can stop with the milk any time now.
Do you have mom and littermates as well, or just the puppy?
I ask because removing the puppy from mom and littermates too early can have long lasting consequences behaviorally- nursing is only a small component of the benefits of keeping them altogether until at least 9-10 weeks.
I would also be in close contact with a veterinarian regarding your vaccination schedule as who knows what sort of maternal immunities this puppy has received.
Is the mother weaning them early or has she a medical issue?