I agree that the term invasive is not necessarily helpful. It can be used for either native or non-native species; the Douglas-fir in the Willamette Valley are sometimes referred to as "invasive" because (anthropogenic) fire has been removed from the once-prominent oak system. But the term "noxious" is more apt, because it implies harm. That is why most people refer to noxious weeds, not invasive or alien.

As for whether humans are natural or not, that doesn't seem like a relevant part of this discussion. Of course humans are natural. And do we really think that anthropogenic impacts on natural systems are always negative, or that they're something that colonizers introduced?

But there is such a thing as scale. Central to this entire discussion is how quickly systems adapt, and the resilience of systems. As species can travel at incredibly fast speeds, basically as fast as an airplane, new risks arise. It's not that species couldn't travel before. But that we're introducing species constantly at this point. This is a contributing factor for species extermination, because species present in a system, especially a relatively isolated (say, island) system, cannot flee or adapt in time. So whether you "like" native or non-native or whatever, if you like biodiversity, then you do not like the constant, and very very quick, movement of all kinds of species.