The noise in only 1 direction makes me think it needs shims, a bearing is loose, housing cracked, bent or worn shaft...
In 1 direction, the gears are loading up and rotate correctly, in the other, they are finding slop. Some space to slide into or out of that allows the teeth to slip and grind instead of mesh.
Look for wear on the unlikely parts of your gears and housings. I bet thats where the issue lies.