Generally, try not to use SCP. It has been a crufty old program from the Berkeley R-Utilities, but newer OpenSSH releases have rewritten it to use the sftp-server server instead. There will be wildly different behavior between these implementations.

The backend SCP changes are documented here:

https://lwn.net/Articles/835962/

If you need something that SFTP cannot do, then use tar on both sides.

PuTTY has implemented their pscp to prefer the sftp-server for many years, in a long prediction of the eventual abandonment. Their pscp implementation is a better drop-in replacement than the OpenSSH solutions.

The allure of SCP is retry on failure, which is somewhat more difficult with SFTP:

  until scp source.txt user@target:dir/
  do echo target down; sleep 300
  done

I also have an older rhel5 system where I am running tinysshd to use better SSH crypto. Due to upgrades, NFS is now squashing everything to nobody, so I had to disable precisely these checks to let users login with their authorized_keys. I can post the code if anybody is curious.

but also... who has a dir with 777 permissions? Is that something people do nowadays?

Script all the things. double-check your scripts... always be backing up.

I actually think that this assumption is the problem. This assumes a certain problem that, in this example here, was not the real problem. So the whole assumption that openssh refuses a connection in this case, was the wrong assumption to make. This is a design mistake, IMO; I understand the rationale but I disagree with it leading to being unable to connect. I have had similar problems with assumptions before, e. g. "if you are the super-user, we do not allow you to start X; you must be a regular user and use sudo". This is IMO also the wrong design approach - the very idea to restrict what the superuser can do. KDE used to have added an extra #define macro to refuse to be started when the superuser tries to use KDE. This is also the wrong design abstraction - people writing the code not understanding the basic permission system in *nix. (It only were a few #defines in the C++ code, so people could just remove it then recompile the thing and it suddenly worked like pure magic. I had that in some KDE editor, I forgot which one; I think it was kate. Been many years by now.)

If you still have some access (console, password login, another sudo user), this usually fixes it:

    username=bob
    sudo chown "$username:$username" /home/$username
    sudo chmod 700 /home/$username

    sudo install -d -m 700 -o "$username" -g "$username" /home/$username/.ssh
    echo "ssh-ed25519 AAAA....insertyourpubkeyhere" | sudo tee /home/$username/.ssh/authorized_keys >/dev/null
    sudo chown "$username:$username" /home/$username/.ssh/authorized_keys
    sudo chmod 600 /home/$username/.ssh/authorized_keys

    echo "$username ALL=(ALL) NOPASSWD: ALL" | sudo tee /etc/sudoers.d/$username >/dev/null
    sudo chmod 440 /etc/sudoers.d/$username

Shim (old but readable): https://github.com/userify/shim/blob/master/shim.py#L308 (obligatory): https://userify.com

Too many burned fingers to not do this little dance almost every other time.

Actually, I lied, I just use rsync like an insane person.

<bleep> that nonsense!