I went ahead and started with just filling in the groves on that little shaft, one at the front and one at the rear. Looks like that was about 95% of the problem and things line up pretty good now. I lubed it up a bit and think I'll suggest a replacement to the owner in the near future but I think this should buy some time with it. Thanks to all for the help. Link to photo of the finished product below.
https://photos.app.goo.gl/mFcwsTM1QqjCwnVv8Side note: The large lower drive pulley is attached with a large lock nut. Are there torque specs on this lock nut? I installed it with common sense but wasn't sure if there's a specific torque range for it.