I'm not worried about deskilling for myself -- I amuse myself doing programming puzzles, and so far so good.
I am worried that my ADHD will decide that an agent loop doesn't provide enough happy brain chemicals, and thus turn me away from it. Kind of the opposite of the problem some people have, but my brain is weird.