With a new computer came the need to setup posh-git again. I try to use the SSH connections were possible. This time, I hit an issue. git pull in posh-git returned an error (below), but the GitHub app was working normally.
Using git bash, I checked to see if I could connect to GitHub. The results showed me that posh-git was using the default RSA key file name, id_rsa.
The fix was pretty easy: I just needed to add an entry in my ssh config file. This file is usually located in c:\Users\<your name>\.ssh. After adding the entry, I needed to restart the ssh agent. After that, everything worked fine.