Thank you for your question. Allah is unlimited and boundless in a way that cannot be expressed by comparison. So if we speak in terms of math Allah is not infinity in comparison to the number one. That is that in maths if 10 is the sum of 10 ones, infinity is the sum of unbounded ones. This would mean that Allah is the sum of something which is inaccurate. Interestingly in ahadith we have been discouraged to say Allah is one in the mathematical sense as that puts Him in comparison to two. This is probably what was meant by the person who explained to you that to express Allah in numbers is shirk as He has been limited. That is that although you may have not intended anything except the expression of Allah's limitlessness with the symbol of infinity, in the system of numbers it is the number one that pervades even if it is not affected by different languages.
The ulema have discussed the issue of expressing Allah in words as it is also clear that words are extremely limited and are unable to grasp Allah's infinity. Some ulema have said that we should only express Allah using the words that He has described Himself with in the Qur'an or with the statements of the Infallibles (as). Others have taken the view that as long as the meaning is established for Allah then using a word not found in the scriptural sources is alright. Saying God in English is a translation of Allah and not a new expression of any of the Names of Allah in any way.
May Allah help you in your effort to better understand Allah and the issue of tawhid and may He grant you more success in understanding the intricacies of our beliefs.
'Abd Allah Esmail