Below is how to fix the accidentals:
- Go to Settings -> Typing
- Then click Advanced keyboard settings
- Language bar options

- Advanced Key Settings

- Finally, turn off those awful other keystroke options

Finding those options is not easy to do. Really, really bad UX Microsoft.
