Insignificant to what or whom? Though I find that perspective interesting in a vertiginous sort of way, I see it that I cannot be insignificant in that regard as there is nothing there capable of judging my significance. It is, after all, my imagination taking the facts you note and then running with them. I am appearing insignificant to myself, which doesn't really hold. If I were to go far away enough, my house would seem insignificant. When I am sat in it, sheltering from the weather, it is very significant indeed.
If there is something there, capable of observing and of thinking things significant or otherwise, then it is presently insignificant to me, too. At least unless/until it comes here and sucks all our bones out or something.
I am directly significant to some people and a few animals. That's quite enough to be getting on with, for me. Depending on one's views on ethics and politics, I am (or could be, or should be) significant to many more people.
_________________
Of course, it's probably quite a bit more complicated than that.
You know sometimes, between the dames and the horses, I don't even know why I put my hat on.