Tibet was a client state of the Qing dynasty. There’s a reason your referring to the area of Tibet as Tibet instead of South West China. Tibet has a unique culture that differs greatly from Han Chinese culture. It’s like saying Vietnam should be part of China because they have historical claim over the land.

tibet was mostly considered a part of china on and off for like the past 400 years

Tibet was part of the Qing dynasty from 1720 after it had been taken over by the Khoshut Khanate in 1640 and then the Dzungar Khanate, after which it had entered in an agreement with the Qing dynasty who had driven out the invading Dzungars. Previously Tibet had been considered an independent state after the Yuan who conquered Tibet for about 100 years were defeated by the Ming and driven out of the region in 1355. During those 300 years is really the only time since after the 9th century when the Tibetan empire fell that Tibet could be considered an independent state, though its relations with the Ming dynasty during that time is also complicated as it was a tributary state considered by scholars as part of the Ming empire but the exact relation is debated enough that most consider Tibet to have been largely independent during that time. I dont think you could say modern Tibet was colonized in any sense of the word though, it was never settled by Chinese dynasties and it was given large internal autonomy. People like to compare it to Ireland but its not really an apt comparison imo even after Tibet became part of the Qing dynasty.