Below is how to fix the accidentals:
- Go to Settings -> Typing
- Then click Advanced keyboard settings
- Language bar options
- Advanced Key Settings
- Finally, turn off those awful other keystroke options
Finding those options is not easy to do. Really, really bad UX Microsoft.