I found a way to update using drush! I think the key idea I was missing was exactly how and where drush was installed on this appliance.  I deduced two things:

1. drush lives in /usr/local/src/drush on this appliance
2. It appears to just be cloned from the drush git repository and set to track the 8.x branch

With this bit of info, I was able to update drush first CORRECTLY, and then use the latest drush to upgrade basically in the manner suggested in the official drupal 8 upgrade docs.

I wrote a bash script to do this with a couple of tweeks to hopefully make the process smooth. A few things to note about this script:

• It will update your drush by pulling from github and checking out the 8.1.15 branch
• It does a two stage core update, first to 8.4.0 then to the latest 8.4.x (currently 8.4.4)
  • This was suggested as a safer path in a few places on the web if you aren't coming from 8.3.7
  • In the 14.2 version of this appliance, it is running 8.3.1
• It updates the composer vendors (I don't think this is necessary but I figured it was a good idea)
  • To disable this, just remove the 'composer update --with-dependencies' line

#!/bin/bash
if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root"
   exit 1
fi

read -p "First we will update drush [enter to continue]"
cd /usr/local/src/drush
git pull
git checkout 8.1.15

read -p "Ready to archive and enter maintenance mode [enter to continue]"
cd /var/www/drupal8
drush archive-dump
drush ups
drush sset system.maintenance_mode 1
drush cr

read -p "Ready to download and apply core 8.4.0 update [enter to continue]"
drush up drupal-8.4.0
drush updb
drush entup

echo "********************************************************"
echo "* The setDispatcher() error can be ignored             *"
echo "* [see https://github.com/drush-ops/drush/issues/2933] *"
echo "********************************************************"

read -p "Ready to proceed to install latest 8.4.x core [enter to continue]"
drush up drupal
composer update --with-dependencies
drush updb
drush entup

read -p "Ready to finish, check your site for any problems first [enter to continue]"
drush sset system.maintenance_mode 0
drush cr