Time travel is either easy or impossible. It is easy because essentially time is just another dimension of space-time and travel in the other three dimensions is easy and we are already all travelling in time (just in one direction and locally at the same rate). On the other hand, time travel appears to imply paradoxes. Paradoxes of causality worry me less than issues such as conservation of energy/matter.
Causality worries me less because I suspect cause and effect isn’t everything we might think it is. However, if I disappear from one time period and reappear in the next, then my new time period has more matter in it than it did before and worse, that matter will hang around going forward into the future. Perhaps time travel requires some physical exchange of matter between time periods? If so, then what in the universe is keeping track?
A different issue with time travel is the speed of light. Even approaching (but staying within) the speed of light has some weird temporal consequences. Actual time travel would provide ways of in-effect travelling faster than light and likewise FTL travel also implies a degree of time travel. If the speed of light is a hard unbreakable rule of the universe then it follows that time travel is impossible also.
One method of time travel escapes most of these issues: perception. In Kurt Vonnegut’s Slaughter House 5, Billy Pilgrim becomes unstuck in time due to the allied bombing of Dresden in World War 2. However, the time travel here is how he perceives events. Billy’s body isn’t popping in and out of time periods, instead, he is experiencing his life out of sequential order. This kind of mental time travel avoids issues of causality on the grounds that everything has already happened. Events are what they are but perhaps the order in which we experience them is an illusion of the human mind.
Unfortunately for stories, there aren’t many writers who can make a narrative work where no decisions can possibly matter. Which takes me to a topic for another time: how should time travel stories work?