I started learning English when I moved from my home country all the way to Baku, Azerbaijan, there I went to my first international school where I first started learning it. I had very few people that I could talk to that knew Spanish and I basically only learnt the language by being surrounded by people who spoke it. I had experienced a short but tough time in my school as I definitely felt excluded from all groups and I couldn't express my ideas very well. Here in Abu Dhabi, you don't really see that exclusion as English is not the main language, but in the U.S, from experiences from my relatives living there, some of them are ignored and not taken seriously when trying to explain/express something.
"Limited" or "Broken" English as Tan describes it, is mostly due because the person has to translate what he/she wants to say from his/her first language to English, and most of the times (if you don't have much experience with the language) they could translate it word for word, similarly to how Tan videotaped one of her mother's conversations. This translating method that a lot of people use, does not make them less educated in general or should be treated any different from others, the ideas are still there, its just that some people can really just judge or make fun of someone because of some grammatical errors. This happens quite a lot in the internet with a number of people getting very upset if you do something like saying "your" instead of "you're" or just miss some words that may affect the flow of the sentence.
Accent is also a major factor that immediately separates you from everyone else, for some people. It is also an example of how your mother tongue has an effect in your identity and the way of expressing yourself. Someone famous who fits the role of being person who is made fun of because of her accent, and not being taken seriously is Sofia Vergara (Colombian actress). Due to her strong accent, some of the words that she may mean to say could mean another thing to someone else who is listening to her, or she may just not pronounce the word well.
Overall, I do not believe people should be treated differently or have any generalization on people's level of education because of their English. Just like Tan's mother, she is quite smart, reads the Forbes report, listens to Wall Street Week, reads all of Shirley MacLaine's books and converses daily with her stockbroker, but she just cant express herself fully with words but instead uses very simple and direct language which shouldn't be classified as "Broken" compared to the "Standard English".