I don’t think you understood half of what I wrote correctly. Feel free to skip the next paragraph if you already know about triangulation in the context of cell towers.

Your phone, when connected to a cellular network, can be tracked by cellular towers. Take a single tower to which you are connected. The tower has a multitude of devices which are connected to it. Since latency is a large issue due to distance, communication is less trivial. To prevent waiting for packets from a device for too long, the cell tower will divide a certain time into sectors. In each of these sectors, one device is supposed to send it’s packet. To hit this time window however, the device has to send it’s packet in advance, as to cancel out the latency of the transmission. By how much is determined by the distance between the cell tower and the device. This system requires knowledge of distance. This distance has to be close enough to the real distance, to cancel out latency and maximize the time a device can send data. With obstacles that may reflect the waves, the preciseness of the distance determined is correlated to the stability of the connection. Given that a lot of people can be using a cell tower at once, there are towers which are segmented. These work slightly differently than non segmented ones, in that they have multiple antenna sending and receiving signals in different directions. A segment of a tower with 6 segments would then be responsible for all devices which are in the area of 60 degrees from the tower. The distance together with the general direction already gives us a pretty accurate depiction of the user’s location. However, since most towers are neither segmented, nor would a say 6 segmented one give as accurate of a reading if the device is 3 miles away (that would be a possible curve of pi(≈3) miles which the device could be on, given a perfectly flat ground), let’s make this even more interesting. With a simple, rough time stamp, we can find out the location of a device up to an accuracy of a single point. Add a second cell tower, make the device connect to the towers one at a time and you have the exact position of the device. Well, pretty close to exact at least.

With that said, the reason why a sim card is the issue in this constellation is the unique identifier. If a network of cell towers can determine where you are located and the imsi is tied to your identity, then your location is tied to your identity. And if you think, even for a second, that mobile internet providers would not take advantage of this, think again. A very popular example of this are malls wanting to know where their customers came from.

Now, if you have a device with GrapheneOS (DivestOS too I think, don’t know) for example, then you will have the ability to entirely switch off the cellular antenna of your phone (which one should likely do if they don’t have a sim anyways). This is supported by hardware due to airplane mode, where the airplane requires least interference.

Now that I have explained in more detail, do you understand better why a sim card is a privacy concern?