When do you teach your child to distinguish dirt or dirty items? In the toddler years, they should already know what to avoid due to health hazard. It begins with the ground or the soil that you teach the toddler not to eat when the food falls on the ground. And anything that is not edible should not be put in the mouth especially coins and toys. There are times that toddlers start to investigate and experiment on things if it is edible or how does it taste. This can lead to sickness or diseases caused by microbes and germs.
Babies do have an unfortunate habit of investigating everything by putting it in their mouths. That's why we had to put the cat's food and litter trays well out of reach once ours started crawling! Coins and batteries, or anything small enough to be swallowed has to be kept well away from them. The problem is finding a happy medium. If a child doesn't get enough exposure to dirt and germs it will harm their immune system. We've settled for keeping the garden clear of things like animal waste, and teaching him not to eat plants unless we tell him it is alright, and we are with him. He can worry about learning which plants are safe himself once he's a bit older.
It can also lead to things getting stuck in the throat and could cause choking. It isn't easy to teach a child this. the only thing one can do is to keep an eye on them and when you see them doing something, to stop them and to tell them why. Homes should also be made child friendly by keeping things put away and out of reach.
I started teaching my son as soon as he started crawling around and getting curious about things. It's never to early to teach them what is harmful and what is good to be around. Hand washing techniques should be taught early also, to prevent germs on their hands which could be transferred to their mouth.
Same as @AngelaMc, I started to teach my children about that since they started crawling and putting things into their mouth. I didn't "teach on purpose", but whenever they put something inappropriate into their mouth, I told them this is something not to put into the mouth. Slowly, they learned it. Though they might still put that into their mouth, I realized when they wanted to put something into the mouth, especially something you have told them not to, they would look at you first and smile. I guess they were actually trying to get your "approval" or they knew it's something not supposed to be put into the mouth. I also always washed their hands before letting them eat something or after coming back from outside/playing when they were still toddler. When they grew older, I asked them to do the same. So they slowly built the habit.
I think the process begins right from the age when they learn how to hold something in their hands. You always take it off their hands even if they show their disapproval indirectly or directly before leaving that thing. If you keep reminding them time and again the chances are they will know the reason and give up the habit of picking it up in future. Even if you take it from their hands every time you see them picking something unwanted I am sure you'll succeed in your purpose.
It's tough to prevent kids from getting in contact with dirt. I agree with @Taliska that babies investigate everything by putting things in their mouths. As @janemarie said, the best way is keep them away from dirt is to keep our eyes on them, closely watch them. And to make homes child-friendly by putting away things they should not be holding.