By all accounts at the beginning of the game/story it appears that not just any normal person can pick up the shards of and assemble the completed “Epoch” item which let us traverse through the timelines.
While it isn’t completely laid out at this point, at least that I have found, it is clear that there is some sort of gift which the “Travelers” hold that has allowed them to accomplish all these feats which we accomplish. Clearly, we are skilled warriors of great magical and martial capability, but there seems to be something more supernatural/preternatural going on with us which has not yet been fully revealed. Though we are certainly not the only one as the NPCs mention numerous other travelers that have come before (and perhaps after? this time travel thing can get sticky in places) us as noted by many of the NPCs such as Yulia and the Forgotten Knight to name a couple of them.
Also, something tells me that while we the “traveler” are strong and are fording our way through the ever strengthening flow of the void we may not truly be immune to its effects; at least for setting story purposes as I believe other travelers and what became of them and their efforts may be revealed more as time goes by.
So right now it seems that our magic and martial prowess coupled with a rather intense sense of purpose and a will to match it that sets us apart.