We've already done it. Traveling forward is easy. The trick is to not think of space and time as separate because they're not. The faster an object travels in "space" the slower it simultaneously moves in "time" in relation to slower objects. If you're going really really really really fast, you're effectively time traveling forward. Astronauts are inadvertent time travelers. Because they orbit earth so fast, they experience time dilation and age slower than everyone on Earth. Voila, dirty time travel.
t's traveling backwards that's seemingly impossible but we know a way to do that too, theoretically. Going faster than light is the secret, so that's probably never actually happening for us. If we do go back, it's not that way.
Gravity and time also have a whole thing going on. Gravity is warped time but also....warps time? It's really weird and I'm a computer scientist not an actual one so that's the best I can do from my amateur interests.
Any civ advanced enough to time travel is advanced enough to take the proper precautions, such as not getting caught by us. Which shouldn't be hard. We still burn fossil fuels and use gunpowder, we're not impressive at all. You're right that we probably don't ever achieve it though. The universe wouldn't work so hard to maintain causality if it wanted to be easy to free yourself from.