That space is still a shared aisle area, not something the seat owner gets to police. Using it to get in and out, grab a bag, or let someone with a cane pass is totally normal.
If someone pays for extra legroom, they’re paying for legroom, not exclusive rights to block movement. On long haul especially, people need to get up. That’s just how it goes.
Given there’s an older passenger with a cane involved, basic courtesy should override any seat entitlement anyway. You weren’t doing anything wrong.
I’d just be polite but firm and keep doing what you need to do. Cabin crew will back you up if it becomes an issue.
