Where the centre of mass ends up

Step through four paths and see that potential energy depends on where you end, not on the effort spent.