It might not mean much to you now with your current habits, but once Vim's unique mode-based command model becomes deeply ingrained in you, you'll appreciate the noticeable difference between having to move your right hand to the cursor keys, and having the most immediate navigation commands right on the home row. Users coming from different editors may be very fast using Ctrl / Shift combinations together with the cursor keys (and Ctrl + C / V to copy / paste), but in the end, there's a natural speed bump in how fast you can type those sequences. (I've been there, too.)
To me, Vi(m) pushes this to another level with the multitude of navigation commands (
}, and that's just some of the built-in ones; with plugins and customizations, this can become dozens of specialized motions). The limiting factor becomes how quickly you can recall a fitting key combination, not how fast to press keys in succession and adapt the displayed selection to what you need (though that's still possible in Vim - via visual mode).
To me, asking about cursor keys versus
hjkl is the wrong question. The real question is whether to spend the effort to really learn the operating model of Vi(m) - you don't need to, but casual use of Vim won't let you experience the glowing fanaticism and allegiance that many of its users have. If you start with
hjkl on the home row, the other (better) motions are close by, so you're more inclined to use them (and pretty soon will wean yourself off your focus on the cursor keys).
For an optimal start on this journey, also consider the location of the Esc key, another very central key in Vim. Its default placement on the keyboard is far from optimal; many users swap it with either Caps Lock or (left) Ctrl.